Using the /common File System
- Connected read/write to all HCC HPC cluster resources – you will see
the same files “in common” on any HCC cluster (i.e. Crane and Tukser).
- 30 TB Per-group quota at no charge – larger quota available for
- No backups are made! Don’t be silly! Precious data should still be
stored / backed up elsewhere such as on Attic. Please.
- No purge! So, while your files just might be lost in the case of
disk failure or user error, they won’t be removed by the purge
/common directory can be accessed via the
$COMMON environment variable, i.e.
How should I use
- Store things that are routinely needed on multiple clusters.
/common is a network attached FS, so limit the number of files per
directory (1 million files in a directory is a very bad idea).
- If you are accessing
/common for a job, you will need to add a
line to your submission script!
- We have each user check out a “license” to access
/common for a
- This allows us to know exactly who is accessing it, and for how
long, in case of the need for a shut down so we can try to avoid
killing jobs whenever possible.
- It also allows us to limit how many jobs can hammer this single
filesystem so it remains healthy and happy.
To gain access to the path on worker
nodes, a job must be submitted
with the following SLURM directive:
SLURM Submit File
If a job lacks the above SLURM directive,
/common will not be accessible from the worker nodes. (Briefly, this construct will allow us
to quickly do maintenance on a single cluster without having to unmount
$COMMON from all HCC resources).
What should I not do when using
- Don’t use it for high I/O work flows, use
/work for that –
should mostly be used to read largely static files or data.
- Do not expect your compiled program binaries to work everywhere!
/common is available on machines with different cpu architecture,
different network connections, and so on. caveat emptor!
- Serial codes will not be optimized for all clusters.
- MPI codes, in particular, will likely not work unless recompiled
for each cluster.
- If you use
module things should be just fine!