ATTIC Guidelines and Best Practices

Introduction

ATTIC is a near-line digital archival storage service for research, scholarly, and creative activity data, provided to those affiliated with the University of Nebraska. A near-line archival storage system is not typically directly accessible (i.e., mounted), and uses storage technology that can be accessed more rapidly than traditional data archival media (e.g., tape). HCC designed ATTIC to meet the requirements of a near-line archival storage system and be as affordable as possible. ATTIC is not designed for active use. Digital storage systems that provide online direct access are designed to be performant, can require specialized hardware, networking, and/or software, and are comparably more expensive to implement and maintain.

ATTIC uses the Zettabyte File System (ZFS), an open-source, widely used, flexible, and resilient filesystem. ATTIC’s storage configuration is designed to follow modern best practices and has multiple redundancies built-in for different types of potential failures. ATTIC is installed in climate-controlled facilities with restricted access and generator-backed redundant power. Copies of every file exist at two distinct geographic locations (Lincoln and Omaha), connected by a high-speed redundant network.

Expectations

• HCC is committed to reducing time-to-science and maximizing value for researchers at the University of Nebraska in the areas of computing, networking, and storage.
• ATTIC is not free; it is a community-supported resource.
  
• HCC depends on the good citizenship of researchers to collectively provide a positive user experience on shared resources, including ATTIC.
• Users are expected to follow best practices when using ATTIC to collectively help prevent excessive load and degraded performance which can be harmful to the larger community.
• HCC reserves the right to curtail problematic use of ATTIC (and other resources) to protect the integrity of the systems.

Guidelines and Best Practices

• Allocations and Billing

  • ATTIC storage allocations are leased to single HCC groups in one terabyte incremental units. Suballocations are not possible.
  • HCC is generally not allowed to lease ATTIC storage allocations outside of the current fiscal year¹. ATTIC users normally lease for a full year with one payment at the beginning of the fiscal year. If an HCC group begins to use ATTIC in the middle of a fiscal year, leasing charges can be prorated to the end of the current fiscal year.

• Data Transfer and Sharing

  • Globus is the preferred tool for transferring and sharing data.
  • It is the responsibility of the researcher/PI to ensure that data is shared only with authorized persons. Public sharing should be used with care.
  • ATTIC should not be relied upon for a permanent archive (referenced in publications). Campus research data librarians should be consulted with for options related to data publication.
  • Archiving software tools such as ZIP and tar should be used, when possible, to combine many small files into one to reduce the total number of files². This has the beneficial effect of increasing computing workflow efficiencies and reducing the burden on ATTIC.

• Backups and Retention

  • The ideal ATTIC data storage use-case is a set of well-curated, archived (written once, read seldom) data files. Automated, frequently synced, rapidly changing, remote system ‘backups’ are generally discouraged as they can produce excessive system load and are outside the scope of ATTIC³.
  • On HCC compute clusters, home directories ($HOME) are backed up automatically as a provided service. The $WORK and $COMMON filesystems are not backed up. Any precious files should be stored on $HOME, ATTIC, or other trusted storage systems.
  • If a researcher unintentionally deletes a file on ATTIC, they can contact HCC support for help in restoring it. Note, the window for recovery is limited so HCC must be informed as soon as possible.
  • When an ATTIC allocation lease is not renewed the researcher must move any files to other storage systems if the researcher wants to keep them.
  • If an ATTIC allocation lease expires, HCC will remove any remaining files ninety (90) calendar days from the expiration date. HCC will make a good faith effort to contact the researcher and/or their department of record before removing any data.

• Sensitive and Protected Data

  • All researchers using ATTIC must adhere to the policies described in the University of Nebraska Executive Memorandum 42.
  • It is the responsibility of the researcher/PI to ensure that no ‘high risk’ data (e.g., HIPAA or other PID data sets) as defined in the University of Nebraska Executive Memorandum 41 are stored on ATTIC.