Grid Engine Job Submission

In order to use the Grid Engine commands you first need to load the "sge" module:

[abc123@login-0-0 abc123]$ module load sge

You can submit a job with qsub. The qsub command has the following syntax:

qsub [ options ] [ scriptfile | -- [ script args ]]

In general, you will need a jobscript to submit jobs to SGE. A jobscript can contain various settings and variables to go with your application. Basically it looks like:

#!/bin/bash        
#$ Script options  # Optional script directives
shellscripts       # Optional shell commands
application        # Application itself

Please see the code examples overview for some commonly used SGE jobscript templates.

Directive	Shell intepretation	SGE intepretation
#	Comment	Comment
#$	Comment	Directive
# $	Comment	Comment

The following table shows a summary of some commonly used directives in SGE. For further information, please refer to the manual pages of SGE ('man qsub').

Directive	Example	Description
-N	#$ -N testjob	jobname to be used in SGE
-M	#$ -M user@domain	sends email notification to user@domain
-m	#$ -m be	mail is be sent at beginning of job and at the end of the job.
-e	#$ -e /tmp	redirects error file to /tmp
-o	#$ -o ~/out	redirects output file to $HOME/out
-q	#$ -q all.q	specific queue, i.e. all.q
-cwd	#$ -cwd	execute from the current working directory
-pe	#$ -pe openmpi 6	request a parallel environment, i.e. openmpi using 6 slots (CPU)
-l h_vmem	#$ -l h_vmem=4G	requesting 4GB of memory for your job

Submitting to a specific queue

To see which queues are available on the Shamu cluster use 'qstat -g c' - NOTE ids.q, millwater.q and plasmon.q are private queues and only lab members are allowed to use them. bigmem.q and gpu.q are open to everyone but requires us to add users to the existing ACLs based on need:

CLUSTER QUEUE CQLOAD USED RES AVAIL TOTAL aoACDS cdsuE
--------------------------------------------------------------------------------
all.q 0.91 1319 0 943 2336 160 64
bigmem.q 0.00 0 0 144 144 0 0
dev.q 0.91 0 0 96 96 0 0
gpu.q 0.00 5 0 27 32 0 0
ids.q 0.00 1 0 63 64 0 0
millwater.q 0.00 0 0 32 32 0 0
plasmon.q 0.00 21 0 155 176 0 0

Then submit the job, e.g. to the bigmem.q:

qsub -q bigmem.q job.sh

Deleting jobs

Removing your job from the system is easy.

qdel jobid

Where the job-id is the number assigned by SGE when you submit the job using qsub. You can only delete your own jobs. Please note that deletion takes place regardless of its state, running or spooled.

Sample job scripts

https://hpcsupport.utsa.edu/foswiki/bin/view/Main/SampleSGESubmitScripts

-- AdminUser - 17 May 2016