Parallel execution of Q-Chem can be threaded across multiple processors on a single node, using the OpenMP protocol, or using the message-passing interface (MPI) protocol to parallelize over multiple processor cores and/or multiple compute nodes. A hybrid MPI + OpenMP scheme is also available for certain calculations, in which each MPI process spawns several OpenMP threads. In this hybrid scheme, cross-node communication is handled by the MPI protocol and intra-node communication is done implicitly using OpenMP threading for efficient utilization of shared-memory parallel (SMP) systems. This parallelization strategy reflects current trends towards multi-core architectures in cluster computing.
Table 2.1 summarizes the parallel capabilities of Q-Chem 5.4.
| Method | OpenMP | MPI | MPI+OpenMP |
|---|---|---|---|
| HF/DFT energy & gradient | yes | yes | yes |
| HF/DFT analytical Hessian | yes | yes | no |
| CDFT/CDFT-CI | yes | no | no |
| RI-MP2 energy | yes | no | no |
| Attenuated RI-MP2 energy | yes | no | no |
| Integral transformation | yes | no | no |
| CCMAN & CCMAN2 methods | yes | no | no |
| ADC methods | yes | no | no |
| CIS/TDDFT energy & gradient | yes | yes | no |
| CIS/TDDFT analytical Hessian | yes | no | no |
To run a Q-Chem calculation with OpenMP threads, specify the number of threads (nthreads) using the qchem command option -nt. Since each thread uses one CPU core, you should not specify more threads than the total number of available CPU cores for performance reason. When unspecified, the number of threads defaults to 1 (serial calculation).
qchem -nt nthreads infile outfile qchem -nt nthreads infile outfile save qchem -save -nt nthreads infile outfile save
Similarly, to run parallel calculations with MPI use the option -mpi -np to specify the number of MPI processes to be spawned.
qchem -mpi -np nprocs infile outfile qchem -mpi -np nprocs infile outfile savename qchem -save -mpi -np nprocs infile outfile savename
where is the number of processors to use. If the -np switch is not given, Q-Chem will default to running locally on a single node.
To run hybrid MPI+OpenMP HF/DFT calculations use combined options -mpi -np and -nt together, where -np followed by the number of MPI processes to be spawned and -nt followed by the number of OpenMP threads used in each MPI process.
qchem -mpi -np nprocs -nt nthreads infile outfile qchem -mpi -np nprocs -nt nthreads infile outfile savename qchem -save -mpi -np nprocs -nt nthreads infile outfile savename
When the additional argument savename is specified, the temporary files for MPI-parallel Q-Chem are stored in $QCSCRATCH/savename.0 At the start of a job, any existing files will be copied into this directory, and on successful completion of the job, be copied to $QCSCRATCH/savename/ for future use. If the job terminates abnormally, the files will not be copied.
To run parallel Q-Chem via the Slurm job scheduling system, add the -slurm command line argument when starting Q-Chem. For example:
qchem -slurm -nt nthreads infile outfile qchem -slurm -mpi -np nprocs infile outfile
On computer systems with other batch schedulers such as PBS, users may need to set QCMPIRUN environment variable to point to the mpirun command used in the system. For further details users should read the $QC/README.Parallel file, and contact Q-Chem if any problems are encountered (support@q-chem.com).