Anvil: HCC's Cloud

Have your account and ready to go? Visit the Anvil OpenStack web interface at https://anvil.unl.edu.


Overview

Anvil is the Holland Computing Center’s cloud computing resource, based on the OpenStack software.
OpenStack is a free and open-source software platform for cloud computing. Anvil was created to address the needs of NU’s research community that are not well served by a traditional batch-scheduled Linux cluster environment. Examples of use cases that are well suited to Anvil include:

  • A highly interactive environment, especially GUI applications
  • Require root-level access, such as kernel modification or virtualization work
  • Alternate operating systems, such as Windows or other distributions of Linux
  • Test cluster environments for various software frameworks, such as Hadoop or Spark
  • Cluster applications that require a persistent resource, such as a web or database server

Using Anvil, one or more virtual machines (VMs) can be easily be created via a user-friendly web dashboard. Created VMs are then accessible from HCC clusters, or your own workstation once connected to the Anvil Virtual Private Network (VPN). Access is through standard means, typically via SSH for Linux VMs and Remote Desktop for Windows VMs.

Cloud Terms

There are a few terms used within the OpenStack interface and in the instructions below that may be unfamiliar. The following brief definitions may be useful. More detailed information is available in the OpenStack User Guide.

  • Project: A project is the base unit of ownership in OpenStack. Resources (CPUs, RAM, storage, etc.) are allocated and user accounts are associated with a project. Within Anvil, each HCC research group corresponds directly to a project. Similar to resource allocation on HCC clusters, the members of a group share the project’s resources.

  • Image: An image corresponds to everything needed to create a virtual machine for a specific operating system (OS), such as Linux or Windows. HCC creates and maintains basic Windows and Linux images for convenience. Users can also create their own images that can then be uploaded to OpenStack and used within the project.

  • Flavor: A flavor (also known as instance type), defines the parameters (i.e. resources) of the virtual machine. This includes things such as number of CPUs, amount of RAM, storage, etc. There are many instance types available within Anvil, designed to meet a variety of needs.

  • Instance: An instance is a running virtual machine, created by combining an image (the basic OS) with a flavor (resources). That is, Image + Flavor = Instance.

  • Volume: A volume is a means for persistent storage within OpenStack. When an instance is destroyed, any additional data that was on the OS hard drive is lost. A volume can be thought of similar to an external hard drive. It can be attached to an instance and accessed as a second drive. When the instance is destroyed, data on the volume is retained. It can then be attached and accessed from another instance later.

Steps for Access

The guide below outlines the steps needed to begin using Anvil. Please note that Anvil is currently in the beta testing phase. While reasonable precautions are taken against data loss, sole copies of precious or irreproducible data should not be placed or left on Anvil.

  1. Request access to Anvil Access and resources are provided on a per-group basis, similar to HCC clusters. For details, please see What are the per group resource limits? To begin using Anvil, user should fill out the short request form at http://hcc.unl.edu/request-anvil-access. An automated confirmation email will be sent. After group owner approves the request, an HCC staff member will follow-up once access is available.

  2. Create SSH keys OpenStack uses SSH key pairs to identify users and control access to the VMs themselves, as opposed to the traditional username/password combination. SSH key pairs consist of two files, a public key and a private key. The public file can be shared freely; this file will be uploaded to OpenStack and associated with your account. The private key file should be treated the same as a password. Do not share your private key and always keep it in a secure location. Even if you have previously created a key pair for another purpose, it’s best practice to create a dedicated pair for use with Anvil. The process for creating key pairs is different between Windows and Mac. Follow the relevant guide below for your operating system.

    1. Creating SSH key pairs on Windows
    2. Creating SSH key pairs on Mac
  3. Connect to the Anvil VPN The Anvil web portal is accessible from the Internet. On the other hand, for security reasons, the Anvil instances are not generally accessible from the Internet. In order to access the instances from on and off-campus, you will need to first be connected to the Anvil VPN. Follow the instructions below to connect.

    1. Connecting to the Anvil VPN
  4. Add the SSH Key Pair to your account Before creating your first instance, you’ll need to associate the SSH key created in step 2 with your account. Follow the guide below to login to the web dashboard and add the key pair.

    1. Adding SSH Key Pairs
  5. Create an instance Once the setup steps above are completed, you can create an instance within the web dashboard. Follow the guide below to create an instance.

    1. Creating an Instance
  6. Connect to your instance
    After an instance has been created, you can connect (login) and begin to use it. Connecting is done via SSH or X2Go for Linux instances and via Remote Desktop (RDP) for Windows instances. Follow the relevant guide below for your instance and the type of OS you’re connecting from.

    1. Connecting to Windows Instances
    2. Connecting to Linux Instances via SSH from Mac
    3. Connecting to Linux instances via SSH from Windows
    4. Connecting to Linux instances using X2Go (for images with Xfce)
  7. Create and attach a volume to your instance (optional) Volumes are a means within OpenStack for persistent storage. When an instance is destroyed, all data that was placed on the OS hard drive is lost. A volume can be thought of similar to an external hard drive. It can be attached and detached from an instance as needed. Data on the volume will persist until the volume itself is destroyed. Creating a volume is an optional step, but may be useful in certain cases. The process of creating and attaching a volume from the web dashboard is the same regardless of the type (Linux or Windows) of instance it will be attached to. Once the volume is attached, follow the corresponding guide for your instance’s OS to format and make the volume usable within your instance.

    1. Creating and attaching a volume
    2. Formatting and mounting a volume in Windows
    3. Formatting and mounting a volume in Linux
  8. Transferring files to or from your instance (optional) Transferring files to or from an instance is similar to doing so with a personal laptop or workstation. To transfer between an instance and another HCC resource, both SCP and Globus Connect can be used. For transferring between an instance and a laptop/workstation or another instance, standard file sharing utilities such as Dropbox or Box can be used. Globus may also be used, with one stipulation. In order to transfer files between two personal endpoints, a Globus Plus subscription is required. As part of HCC’s Globus Provider Plan, HCC can provide this on a per-user basis free of charge. If you are interested in Globus Plus, please email hcc-support@unl.edu with your request and a brief explanation.

Backups

HCC creates daily backups of images and volume snapshots for disaster recovery. All users’ images, detached volumes, and volume snapshots will be backed up on a daily basis. The ephemeral disks of VMs and attached volumes will NOT be backed up. If you would like your attached volumes to be backed up, make a snapshot by going to the “Volumes” tab, click the down arrow next to the button “Edit Volume” of the volume you want to make a snapshot, then, select “Create Snapshot”.

Please note the backup function is for disaster recovery use only. HCC is unable to restore single files within instances. Further, HCC’s disaster recovery backups should not be the only source of backups for important data. The backup policies are subject to change without prior notice. To retrieve your backups, please contact HCC. If you have special concerns please contact us at hcc-support@unl.edu.