2 Installation, Customization, and Execution

2.8 Parallel Q-Chem Jobs

(May 16, 2021)

Parallel execution of Q-Chem can be threaded across multiple processors on a single node, using the OpenMP protocol, or using the message-passing interface (MPI) protocol to parallelize over multiple processor cores and/or multiple compute nodes. A hybrid MPI + OpenMP scheme is also available for certain calculations, in which each MPI process spawns several OpenMP threads. In this hybrid scheme, cross-node communication is handled by the MPI protocol and intra-node communication is done implicitly using OpenMP threading for efficient utilization of shared-memory parallel (SMP) systems. This parallelization strategy reflects current trends towards multi-core architectures in cluster computing.

Table 2.1 summarizes the parallel capabilities of Q-Chem 5.4.

Method OpenMP MPI MPI+OpenMP
HF/DFT energy & gradient yes yes yes
HF/DFT analytical Hessian yes yes no
CDFT/CDFT-CI yes no no
RI-MP2 energy yes no no
Attenuated RI-MP2 energy yes no no
Integral transformation yes no no
CCMAN & CCMAN2 methods yes no no
ADC methods yes no no
CIS/TDDFT energy & gradient yes yes no
CIS/TDDFT analytical Hessian yes no no
Table 2.1: Parallel capabilities of Q-Chem 5.4

To run a Q-Chem calculation with OpenMP threads, specify the number of threads (nthreads) using the qchem command option -nt. Since each thread uses one CPU core, you should not specify more threads than the total number of available CPU cores for performance reason. When unspecified, the number of threads defaults to 1 (serial calculation).

qchem -nt nthreads infile outfile
qchem -nt nthreads infile outfile save
qchem -save -nt nthreads infile outfile save

Similarly, to run parallel calculations with MPI use the option -mpi -np to specify the number of MPI processes to be spawned.

qchem -mpi -np nprocs infile outfile
qchem -mpi -np nprocs infile outfile savename
qchem -save -mpi -np nprocs infile outfile savename

where nprocs is the number of processors to use. If the -np switch is not given, Q-Chem will default to running locally on a single node.

To run hybrid MPI+OpenMP HF/DFT calculations use combined options -mpi -np and -nt together, where -np followed by the number of MPI processes to be spawned and -nt followed by the number of OpenMP threads used in each MPI process.

qchem -mpi -np nprocs -nt nthreads infile outfile
qchem -mpi -np nprocs -nt nthreads infile outfile savename
qchem -save -mpi -np nprocs -nt nthreads infile outfile savename

When the additional argument savename is specified, the temporary files for MPI-parallel Q-Chem are stored in $QCSCRATCH/savename.0 At the start of a job, any existing files will be copied into this directory, and on successful completion of the job, be copied to $QCSCRATCH/savename/ for future use. If the job terminates abnormally, the files will not be copied.

To run parallel Q-Chem via the Slurm job scheduling system, add the -slurm command line argument when starting Q-Chem. For example:

qchem -slurm -nt nthreads infile outfile
qchem -slurm -mpi -np nprocs infile outfile

On computer systems with other batch schedulers such as PBS, users may need to set QCMPIRUN environment variable to point to the mpirun command used in the system. For further details users should read the $QC/README.Parallel file, and contact Q-Chem if any problems are encountered (support@q-chem.com).