Any researcher performing Open Science in the US can become an OSPool user. The OSPool provides its users with fair-share access (no allocation needed!) to processing and storage capacity contributed by university campuses, government-supported supercomputing institutions and research collaborations. Using state-of-the-art distributed computing technologies the OSPool is designed to support High Throughput workloads that consist of large ensembles of independent computations.
Your jobs must fit a set of criteria in order to be eligible to run on OSG. The list below provides some rule of thumb characteristics that can help us make a decision if using OSG for a given job is a viable option.
| Characteristics of an OSG friendly job |
|---|
| Variable | Suggested Values |
|---|---|
| Memory/Process | <= 40 GB |
| Type of job | serial (i.e. mostly single core) |
| Network traffic (input or output files) |
<= 2GB each side |
| Running Time | Ideal time is 1-10 hours - max is 40 |
| Runtime Disk Usage | <= 10GB |
| Software | Non-licensed, pre-compiled binaries, containers |
| Total CPU Time (of job workflow) | Large, typically >= 1000 hours |
The relatively short runtime is necessary due to job pre-emption. Jobs belonging to resource owners on the machine where your job is running may pre-empt (or kill) your job unexpectedly. When this happens, your job’s progress is not automatically saved, and it will have to start over from the beginning. For this reason, it is good practice to build automatic checkpointing into your job, or break a large job into multiple small jobs if it is at all possible.