Example Batch Scripts
When submitting a job with sbatch, you should specify the resources your job requires using the options below. These flags determine how many nodes, CPUs, memory, and other resources Slurm will reserve for your job. Properly requesting resources ensures that your job runs efficiently and is scheduled appropriately. Include these options at the top of your Slurm batch script as #SBATCH directives.
Basic Example
#!/bin/bash
#SBATCH --job-name=MyJob
#SBATCH --time=24:00:00
#SBATCH --partition=$PARTITION
#SBATCH --nodes=1
#SBATCH --ntasks-per-node=24
#SBATCH --mail-type=end
#SBATCH --mail-user=userid@jhu.edu
module load intel/2020.2 intel-mpi/2020.2
mpirun -n $SLURM_NTASKS ./my_program.x > output.log
OpenMP Job Script
#!/bin/bash -l
#SBATCH --job-name=OpenMP-Job
#SBATCH --output=myLog-file
#SBATCH --partition=$PARTITION
#SBATCH --time=00-01:30:15
#SBATCH --nodes=1
#SBATCH --ntasks-per-node=8
###SBATCH --mem-per-cpu=4GB
#SBATCH --account=$SLURM_ACCOUNT
#SBATCH --export=ALL
ml purge
ml intel intel-mpi intel-mkl
export OMP_NUM_THREADS=8
time ./a.out > MyOutput.log
Hybrid MPI + OpenMP Job Script
#!/bin/bash -l
#SBATCH --job-name=MyLMJob
#SBATCH --output=myLog-file
#SBATCH --partition=$PARTITION
#SBATCH --time=02-01:30:15
#SBATCH --nodes=1
#SBATCH --ntasks-per-node=8
#SBATCH --cpus-per-task=6
#SBATCH --account=$SLURM_ACCOUNT
#SBATCH --export=ALL
ml gcc openmpi
module load hwloc
module load boost
module load gromacs/2016-mpi-plumed
module load plumed/2.3b
export CUDA_AUTO_BOOST=1
mpirun -np 8 bin/gmx_mpi mdrun -deffnm options
GPU Job Script (NAMD, 4 GPUs)
#!/bin/bash -l
#SBATCH --job-name=namd4gpu
#SBATCH --time=48:00:00
#SBATCH --partition=$PARTITION
#SBATCH --qos=qos_gpu
#SBATCH --nodes=1
#SBATCH --ntasks-per-node=48
####SBATCH --mem-per-cpu=4G
#SBATCH --gres=gpu:4
#SBATCH -A Slurm-account_gpu
ml purge
module load namd/2.14-cuda-smp
ml
export CUDA_VISIBLE_DEVICES=0,1,2,3
export CONV_RSH=ssh
GPUS_PER_NODE=4
NODELIST=$SLURM_JOBID.nodes
JOBNODES=$(scontrol show hostname $SLURM_JOB_NODELIST)
echo group main > $NODELIST
for node in $JOBNODES; do
echo host $node >> $NODELIST
done
PPN=$(( $SLURM_NTASKS_PER_NODE / $GPUS_PER_NODE - 1 ))
P=$(( ( $SLURM_NTASKS_PER_NODE - $GPUS_PER_NODE ) * $SLURM_JOB_NUM_NODES ))
COMMAND="$(which namd2) +ignoresharing +idlepoll +p 44 +setcpuaffinity +pemap 1-11,13-23,25-35,37-47 +commap 0,12,24,36 +devices 0,1,2,3"
echo $COMMAND
$COMMAND step1_NPT.conf > npt/npt.log
$COMMAND step2_NVT.conf > nvt/nvt.log
$COMMAND step3_Production.conf > production/production.log
Job Array Script
Job arrays are useful for submitting many similar jobs at once, such as parameter sweeps or batch processing with different input files.
You can submit a job array using:
sbatch --array=0-15%4 script.sh
Runs the job script 16 times (task IDs 0 through 15)
%4 limits the number of concurrently running tasks to 4 (optional)
Within your job script, you can use several environment variables:
$SLURM_ARRAY_JOB_ID: The master job ID (same for all array tasks)$SLURM_ARRAY_TASK_ID: The index of the current array task (0–15 in this example)$SLURM_JOBID: Unique job ID for each task (can differ from array task ID)
To control output file naming for each task:
#SBATCH -o slurm-%A_%a.out
Where:
%Ais replaced with the array job ID%ais replaced with the array task index
This will create uniquely named output files like:
slurm-45000_0.out
slurm-45000_1.out
...
slurm-45000_15.out
You can also customize the array index pattern:
# Run only tasks 1, 3, 5, 7
sbatch --array=1,3,5,7 script.sh
# Run tasks 1 to 7 with a step of 2 (i.e., 1, 3, 5, 7)
sbatch --array=1-7:2 script.sh
Job arrays provide a simple and scalable way to run multiple jobs with minor differences, while maintaining clean control over indexing and output.
#!/bin/bash -l
#SBATCH --job-name=small-array
#SBATCH --time=48:00:00
#SBATCH --partition=$PARTITION
#SBATCH --nodes=1
#SBATCH --ntasks-per-node=1
###SBATCH --mem-per-cpu=4G
#SBATCH --array=1-1000%480
ml purge
module load intel
ml
file=$(ls zmat* | sed -n ${SLURM_ARRAY_TASK_ID}p)
echo $file
newstring="${file:4}"
export basisdir=/scratch16/jcombar1/LC-tests
export workdir=/scratch16/jcombar1/LC-tests
export tmpdir=/scratch16/jcombar1/TMP/$SLURM_JOBID
export PATH=/scratch16/jcombar1/LC/bin:$PATH
export OMP_NUM_THREADS=1
export MKL_NUM_THREADS=1
mkdir -p $tmpdir
cd $tmpdir
cp $workdir/$file ZMAT
cp $basisdir/GENBAS GENBAS
./a.out > $workdir/out.$newstring
cd ..
\rm -rf $tmpdir
Jupyter Notebook Job Script
#!/bin/bash
#SBATCH --ntasks-per-node=1
####SBATCH --mem-per-cpu=4G
#SBATCH --time=1:00:00
#SBATCH --job-name=jupyter-notebook
#SBATCH --output=jupyter-notebook-%J.log
ml anaconda
## or use your own python/conda environment
XDG_RUNTIME_DIR=""
port=$(shuf -i8000-9999 -n1)
echo $port
node=$(hostname -s)
user=$(whoami)
jupyter-notebook --no-browser --port=${port} --ip=${node}
Note
Check the file jupyter-notebook-JOBID.log for your connection details.
SSH tunnel command from your local machine:
ssh -N -L ${port}:${node}:${port} ${user}@dsailogin.arch.jhu.edu
Copy the link provided in the log file into your browser (it starts with
http://127.0.0.1:<PORT>).
Common sbatch Options
Option |
Description |
|---|---|
|
Name of the job (shown in |
|
Walltime (HH:MM:SS) requested for the job |
|
Number of physical nodes to request |
|
Total number of tasks (often used for MPI) |
|
Number of tasks per node (used with MPI) |
|
Number of CPU cores allocated to each task (useful for multi-threading) |
|
Total memory to allocate per node |
|
Memory to allocate per CPU core |
|
Charge the job to the specified allocation account |
|
Assign a specific Quality of Service (QOS) |
|
Send email notification at the end of the job |
|
Email address to send job notifications |
|
Allow job to be requeued if interrupted |
|
Export environment variables (ALL, NONE, or list) |
|
Set the working directory for job execution |
|
Submit a job array with optional concurrency limit (here, max 4 jobs run at a time) |
|
Request nodes with specific features or hardware constraints |