Array jobs

Running a simple array test job

This job is very simple. It just prints a message for each iteration in the array

There are sample job submission scripts available to you in the following path /opt/examples/slurm. We will copy one of these to our home directory, and submit it in this example.

CODE

$ cp /opt/examples/slurm/array-job.sh ~/array-job.sh

New sbatch options for parallel jobs

Argument	Default	Description
`--array=A-Z`	none	The number of times to run this job. An example of --array=1-10 would run the job 10 times with an index of 1-10.

Job Script

CODE

#!/bin/bash

# Which partition/queue does this job need to run in.  Default is 'hsw-fdr'
#SBATCH --partition=hsw-fdr

# How long does my job have to run (HH:MM:SS), # without this option limit is
# 5min
#SBATCH --time=01:00:00

# How many cores should I run my job on, for serial jobs this should be 1,
# which is the default if not specified
#SBATCH --ntasks=1

# This is memory need per task (see above).  If not specified you will get
# 3GB of RAM per cpu.
#SBATCH --mem-per-cpu=1G

# The descriptive name for your job.  This potentially will be visible to other
# users on ACTnowHPC
#SBATCH --job-name=array_test

# The name of the file to write stdout/stderr to.  Use %A as a place holder
# for the current job number, %a is the placeholder for the array index
#SBATCH --output=array_test-%A-%a.out

# Run this job 10 times with an index of 1-10.  The index will be available
# as the variable ${SLURM_ARRAY_TASK_ID} to your job script
#SBATCH --array=1-10

echo "I've started my array job and it's running on"
hostname
echo "I'm array index ${SLURM_ARRAY_TASK_ID}"
sleep 30

Options used in this job submission script


#SBATCH --partition=hsw-fdr	Run in the hsw-fdr partition
#SBATCH --time=01:00:00	Run for 1 hour
#SBATCH --ntasks=60	Run the job on 60 cores
#SBATCH --mem-per-cpu=1G	I will need 1GB of RAM per CPU/CORE for my job (in this example 60GB total)
#SBATCH --job-name=array_test	I'm naming my job "array_test"
#SBATCH --output=array_test-%A-%a.out	Write all the output to a file called `array_test-JOBID-TASKID.out`
#SBATCH --array=1-10	Run this job 10 times, ${SLURM_ARRAY_TASK_ID} will start at 1 and go through 10, incrementing for each job run

To run the job issue the following command

CODE

$ sbatch array-job.sh
Submitted batch job 1522

Check the status of the job

Check the status with squeue (more info Basic SLURM commands).

CODE

$ squeue --job 1522