Many commonly used software packages and libraries are provided on the
OSG through the module
command. OSG modules are made available
through the OSG Application Software Installation Service (OASIS). The
set of modules provided on OSG can differ from those on the HCC
clusters. To switch to the OSG modules environment on an HCC machine:
[apple@login.crane~]$ source osg_oasis_init
Use the module avail command to see what software and libraries are available:
[apple@login.crane~]$ module avail
------------------- /cvmfs/oasis.opensciencegrid.org/osg/modules/modulefiles/Core --------------------
abyss/2.0.2 gnome_libs/1.0 pegasus/4.7.1
ant/1.9.4 gnuplot/4.6.5 pegasus/4.7.3
ANTS/1.9.4 graphviz/2.38.0 pegasus/4.7.4 (D)
ANTS/2.1.0 (D) grass/6.4.4 phenix/1.10
apr/1.5.1 gromacs/4.6.5 poppler/0.24.1 (D)
aprutil/1.5.3 gromacs/5.0.0 (D) poppler/0.32
arc-lite/2015 gromacs/5.0.5.cuda povray/3.7
atlas/3.10.1 gromacs/5.0.5 proj/4.9.1
atlas/3.10.2 (D) gromacs/5.1.2-cuda proot/2014
autodock/4.2.6 gsl/1.16 protobuf/2.5
module load
command:
[apple@login.crane~]$ module load python/2.7
There are two things required in order to use modules in your HTCondor job.
Include the following requirements in the HTCondor submission script:
Requirements = (HAS_MODULES =?= TRUE)
or
Requirements = [Other requirements ] && (HAS_MODULES =?= TRUE)
The following example will demonstrate how to use modules on OSG with an
R script that implements a Monte-Carlo estimation of Pi (mcpi.R
).
First, create a file called mcpi.R
:
montecarloPi <- function(trials) {
count = 0
for(i in 1:trials) {
if((runif(1,0,1)^2 + runif(1,0,1)^2)<1) {
count = count + 1
}
}
return((count*4)/trials)
}
montecarloPi(1000)
Next, create a wrapper script called R-wrapper.sh
to load the required
modules (R
and libgfortran
), and execute the R script:
#!/bin/bash
EXPECTED_ARGS=1
if [ $# -ne $EXPECTED_ARGS ]; then
echo "Usage: R-wrapper.sh file.R"
exit 1
else
module load R
module load libgfortran
Rscript $1
fi
This script takes the name of the R script (mcpi.R
) as it’s argument
and executes it in batch mode (using the Rscript
command) after
loading the R
and libgfortran
modules.
Make the script executable:
[apple@login.crane~]$ chmod a+x R-script.sh
Finally, create the HTCondor submit script, R.submit
:
universe = vanilla
log = mcpi.log.$(Cluster).$(Process)
error = mcpi.err.$(Cluster).$(Process)
output = mcpi.out.$(Cluster).$(Process)
executable = R-wrapper.sh
transfer_input_files = mcpi.R
arguments = mcpi.R
Requirements = (HAS_MODULES =?= TRUE)
queue 100
This script will queue 100 identical jobs to estimate the value of Pi.
Notice that the wrapper script is transferred automatically with the
job because it is listed as the executable. However, the R script
(mcpi.R
) must be listed after transfer_input_files
in order to be
transferred with the job.
Submit the jobs with the condor_submit
command:
[apple@login.crane~]$ condor_submit R.submit
Check on the status of your jobs with condor_q
:
[apple@login.crane~]$ condor_q
When your jobs have completed, find the average estimate for Pi from all 100 jobs:
[apple@login.crane~]$ grep "[1]" mcpi.out.* | awk '{sum += $2} END { print "Average =", sum/NR}'
Average = 3.13821