ABAQUS on Arc

The ABAQUS software suite from Dassault Systems is used for finite element analysis and computer-aided engineering. The ABAQUS software is used in the Arc HPC environment for projects from a variety of domains, such as petroleum engineering, biomedical engineering, and aerospace engineering.

Requesting Access to ABAQUS

Access to ABAQUS is limited to specific departments which have acquired licensing for their users. To request access or verify your eligibility for access to ABAQUS, submit a service request. See the Getting Help page for assistance with submitting a request.

ABAQUS Installation on Arc

The ABAQUS 2020 executable is available on the compute nodes using Arc’s module system. The environment module system allows users to tailor their shell for a specific application. Loading the ABAQUS module will configure the shell with the appropriate environment variables, such as adjusting the PATH to include the ABAQUS binaries and other supporting files.

After logging onto arc.utsa.edu, the ABAQUS module can be loaded using the command:

module load abaqus/2020

As of the date of this document, only the 2020 version of ABAQUS is available. However, if additional versions are added at some future point, all of the ABAQUS modules can be found using the command:

module spider abaqus

All available modules can be found using the command:

module avail

Running ABAQUS

As with all other software packages, ABAQUS should not be run on the Login nodes. In fact, the ABAQUS modules are not available on the Login nodes. They are only available on the compute nodes.

All ABAQUS invocations must occur within an interactive session via SRUN on the compute nodes or be submitted to the compute nodes as a batch job.

Running ABAQUS interactively

On a compute node accessed through SRUN, users can run the following command to test their access to ABAQUS and also check the available number of license tokens:

[login001: ~]$ srun –pty -t 1:00:00 -n 1 -p compute1 bash
[c003: ~]$ module load abaqus/2020
[c003: ~]$ abqaqus licensing lmstat -a > abaqus_license.txt

The abaqus command can be run in interactive mode while in the bash session:

[c003: ~]$ unset SLURM_GTIDS
[c003: ~]$ abaqus job=testjob input=<filename>.inp interactive
…
…
THE ANALYSIS HAS COMPLETED SUCCESSFULLY
…
Abaqus JOB testjob COMPLETED
[c003: ~]$

The command will block until completion or until it encounters an error.

Running ABAQUS in batch mode

An example of creating and running an ABAQUS job script is provided below. The example files and job script can be copied from:

arc.utsa.edu:/apps/abaqus/2020/examples/

Copy all of the files to your local working directory.

The example runs ABAQUS in parallel, on one node, in the Arc HPC environment. The number of nodes for your problem size and number of cores needed per node can be adjusted per your requirements.

The number of cores will vary by partition. Please refer to the Arc user-guide for further information on these Slurm job parameters. You can find the user-guide for Arc here: https://hpcsupport.utsa.edu/foswiki/bin/view/ARC/WebHome

====== abaqus-testjob.slm ======

#!/bin/bash
#SBATCH -J myabaqusjob
#SBATCH -t 1:00:00
#SBATCH -N 1
#SBATCH -n 4
#SBATCH -o myabaqusjob.o%j
#SBATCH -p compute1

# SBATCH Parameters:
# -J = Job Name
# -t = Limit on run time
# -N = Minimum number of nodes
# -n = Maximum number of tasks
# -o = Stdout/stderr directed to this file, %j is the SLURM Job ID
# -p = Partition

module load abaqus/2020

# Set this to the correct environment file
# for the test we will use some reasonable defaults
abaqus_environment_file=abaqus_v6.env

# Figure out what nodes this job should run on
node_list=$(scontrol show hostname $cores_per_node | sort | uniq)

# This assumes that there will be an even number of tasks across
# each node, if you have configured your job to run with different
# numbers of tasks on each node, you will need to change the task
# calculations below and create a modified mp_host_list.
cores_per_node=$(echo $SLURM_TASKS_PER_NODE | grep -oP '^[^0-9]*\K[0-9]+')

# Make node list and count the number of nodes
core_count=0
mp_host_list="["

for i in ${node_list} ; do
mp_host_list="${mp_host_list}['$i', ${cores_per_node}],"
number_of_nodes=$((number_of_nodes + 1))
done

echo $mp_host_list

# Calculate the amount of cores per node and multiply it by the amount
# of nodes we are running on
core_count=$(($number_of_nodes*$cores_per_node))

echo "Running on nodes: $cores_per_node"
echo "Running on $cores_per_node cores per node for a total of $core_count processes"

mp_host_list=`echo ${mp_host_list} | sed -e "s/,$//"`
mp_host_list="${mp_host_list}]"
echo "mp_host_list=${mp_host_list}" >> $abaqus_environment_file

unset SLURM_GTIDS

abaqus job=test_abaqus_job cpus=$core_count \
   input=./knee_bolster.inp -verbose 3 mp_mode=mpi \
   standard_parallel=all interactive scratch="."

sed -i "/mp_host_list/d" $abaqus_environment_file

After copying the example files and job script, you can run the test with SBATCH:

[login001: testdir]$ sbatch abaqus-testjob.slm

You can view the progress of the job by tailing the output file:

[login001: testdir]$ tail -f myabaqusjob.o2763
The abaqus module version 2020 is loaded.
[['c001', 4],
Running on nodes: 4
Running on 4 cores per node for a total of 4 processes
Processing time = 0 second(s).
Running with MPI
…

Once completed, the directory should contain various output files resulting from the job.

[login001: testdir]$ ls -laF test_abaqus_job.*
test_abaqus_job.abq
test_abaqus_job.com
test_abaqus_job.dat
test_abaqus_job.fil
test_abaqus_job.mdl
test_abaqus_job.msg
test_abaqus_job.odb
test_abaqus_job.pac
test_abaqus_job.prt
test_abaqus_job.res
test_abaqus_job.sel
test_abaqus_job.sta
test_abaqus_job.stt

Licensing Timeout

When an ABAQUS job is submitted and licenses are not currently available, it will wait for licenses to become available for a specified period of time. During this time, it will keep checking if the required number of license tokens have become available. If they don’t become available during the specified time period, the job will be removed from the job queue.

In order to prevent the ABAQUS jobs from consuming the compute resources while waiting for available licenses, users should include the following line in their ABAQUS environment file:

Lmhanglimit=5

This will set a time limit of 5 minutes to wait for license tokens. (Please check the ABAQUS documentation for the latest version of the environment file.)

Compile User Modules on the Login nodes

Users can create ABAQUS libraries in their /home or /work directories. Use the commands below to create the libraries.

[login001: ~]$ cd $WORKDIR
[login001: abc123]$ mkdir abaqus_libs
[login001: abc123]$ abaqus make library=<sourcefile>;

Place resulting files in the abaqus_libs directory. Note that the example here uses $WORKDIR/abaqus_libs, but any path can be used.

[login001: abc123]$ mv <sourcefile-basename>.o abaqus_libs
[login001: abc123]$ mv <sourcefile-basename>.so abaqus_libs

In the ABAQUS environment file, add the following line:

usub_lib_dir=/path-to-directory/abaqus_libs

Normally the usb_lib_dir will be in your scratch, work or home directory.

Troubleshooting ABAQUS Errors

In order to debug ABAQUS errors, users can run ABAQUS commands with the "verbose" option. Verbose can be set to a value of 3 to retrieve all run details:

[c003: ~]$ abaqus cpus=1 job=testjob2 input=<filename>.inp interactive scratch=”.” Verbose=3

References

• Dassault Systems

• Complete Abaqus Environment