Each research group is allocated 50TB of storage in
/work on HCC
clusters. With over 400 active groups, HCC does not have the resources
to provide regular backups of
/work without sacrificing the
performance of the existing filesystem. No matter how careful a user
might be, there is always the risk of file loss due to user error,
natural disasters, or equipment failure.
However, there are a number of solutions available for backing up your data. By carefully considering the benefits and limitations of each, users can select the backup methods that work best for their particular needs. For truly robust file backups, we recommend combining multiple methods. For example, use Git regularly along with manual backups to an external hard-drive at regular intervals such as monthly or biannually.
If you have sufficient hard drive space, regularly backup your
directories to your personal computer. To avoid filling up your personal
hard-drives, consider using an external drive that can easily be placed
in a fireproof safe or at an off-site location for an extra level of
protection. To do this, you can either use Globus
Connect or an
SCP client, such
as Cyberduck or WinSCP.
For help setting up an SCP client, check out our Connecting Guides.
For those worried about personal hard drive crashes, UNL offers the backup service NSave. For a small monthly fee, users can install software that will automatically backup selected files from their personal machine.
Git is a revision control service which can be run locally or can be paired with a repository hosting service, such as GitHub, to provide a remote backup of your files. Git works best with smaller files such as source code and manuscripts. Anyone with an InCommon login can utilize UNL’s GitLab Instance, for free.
HCC offers long-term, near-line data storage through Attic. HCC users with an existing account can apply for an Attic account for a small annual fee that is substantially less than other cloud services.
Many of us are familiar with services such as Google Drive, Dropbox, Box and OneDrive. These cloud-based services provide a convenient portal for accessing your files from any computer. NU offers OneDrive and Box services to all students, staff and faculty. But did you know that you can link your Box account to HCC’s clusters to provide quick and easy access to files stored there? Follow a few set-up steps and you can add files to and access files stored in your Box account directly from HCC clusters. Setup your submit scripts to automatically upload results as they are generated or use it interactively to store important workflow scripts and maintain a backup of your analysis results.
/work files and directories are not backed up, files and
/home are backed up on a daily basis. Due to the
limitations of the
/home filesystem, we strongly recommend that only
source code and compiled programs are backed up to
/home. If you do
/home to backup datasets, please keep a working copy in your
/work directories to prevent negatively impacting the functionality of
/homefiles are automatically backed up daily.
/homeare not subject to the 6 month purge policy that exists on
If you would like more information or assistance in setting up any of these methods, contact us at firstname.lastname@example.org.