Discover Compilers

Compilers at NCCS

To accommodate the needs of a broad range of user groups, multiple versions of compilers from different vendors are provided on the Discover systems, including:

Vendor	C	C++	Fortran	Finding Module
GNU Compiler	gcc	g++	gfortran	module av comp/gcc
Intel Compiler	icc	icpc	ifort	module av comp/intel
PGI Compiler	pgcc	pgc++	pgf77/pgf90	module av comp/pgi
NAG Compiler	N/A	N/A	nagfor	module av comp/nag

For in depth examples in using modules to select compilers, click here.

Compiler Options

For compiler options, use appropriate man pages. For example:

$ man nagfor
.. or ..
$ man gcc
.. or ..
$ man pgfortran
.. or ..
$ man ifort

MPI Libraries

The NCCS provides support for compiling and linking against various MPI libraries, including Intel MPI, SGI MPT, and HPCX.

Vendor	Supported Compilers	Finding Modules
Intel MPI	GCC/Intel/PGI	module av mpi/impi
SGI MPT	GCC/Intel/PGI	module av mpi/sgi-mpt
HPCX	GCC/Intel/PGI	module av mpi/hpcx

Note: The module system will not display any MPI modules unless you first have a compiler module loaded. Also, because of the OPA interconnect, only “mpi/impi” modules are supported on Skylake nodes.

MPI modules provided by above can be used with all compiler types: GNU, Intel, PGI and NAG.

MPI modules provides an environment that allows usage of MPI libraries and corressponding MPI enabled compilers: mpif90, mpicc, mpicxx.

To learn more about MPI enabled compiler and view the corresponding link lines used by the MPI based compiler, use:

$ mpif90 -show
$ mpicc -show
$ mpicxx -show

Loading MPI libraries

Using Intel compilers:$ module load comp/intel/19.1.3.304 $ module load mpi/impi/19.1.3.304Note: Users are recommended to use the same versions numbers for compilers and MPI.

Using PGI compiler:$ module load comp/pgi/20.4 $ module load mpi/impi/19.1.3.304Note: PGI compilers are no longer being produced, this is the last release.

Note: Due to fortran module dependencies on particular compilers, users are recommended to replace “use mpi” source code line with:include 'mpif.h'

Numerical Libraries

Intel MKL (Math Kernal Library) is the primary numerical library package supported on the NCCS systems. MKL offers highly optimized, thread-safe math routines with good performance.

MKL libraries are already included in the LD_LIBRARY_PATH after you load a module for a recent version of the Intel Compiler (version >= 11). Hence there is no need to load “lib/mkl-*” if you load the module for intel compiler 11 and up.

Please note that you will need to load the module “lib/mkl-*” if you use either PGI or GNU compiler and want to use the MKL.

The table below shows different scenarios to choose the threading and run time libraries (RTL) when linking your code with the MKL (static cases only in the table). All the MKL libraries are located under $MKLPATH.

Compiler	Application Threaded	Threading lib	RTL recommended	Comment
Intel	Does not matter	libmkl_intel_thread.a	libiomp5.so	Dynamically linking RTL is always recommended
PGI	Yes	libmkl_pgi_thread.a or libmkl_sequential.a	None. PGI supplied	Use of libmkl_sequential.a removes threading from Intel MKL calls
PGI	No	libmkl_sequential.a	None	—
GNU	Yes	libmkl_gnu_thread.a	libiomp5.so or GNU OpenMP run time library	libiomp5 offers superior scaling performance
GNU	No	libmkl_sequential.a	None	—
Other	Yes	libmkl_intel_thread.a	libiomp5.so	—
Other	No	libmkl_sequential.a	None	—

To decide which options to use during the link and compile stages when you invoke MKL routines, Intel provided a very helpful Link Advisor website.

Loading MKL with Intel compilers

Intel MKL are bundled all Intel compilers, starting with version 11. It is recommended that you use the most recent compiler version. Use:

$ module load comp/intel/18.0.5.274

The MKL package provides two environment variables: MLKPATH and MLKROOT. The following commands show the value of these environment variables:

$ echo $MKLPATH
[path]/intel/2018/compilers_and_libraries_2018.5.274/linux/mkl/lib/intel64/
$ echo $MKLROOT
[path]/intel/2018/compilers_and_libraries_2018.5.274/linux/mkll

Users can find example code using the following path, once they have loaded the appropriate module:

$ cd $MKLROOT/examples

Examples for compiling Basic Linear Algebra Subprograms(BLAS) programs using different kind of compilers:

Intel compiler

Set your LIBS variable to:

LIBS = -L$MKLPATH -lmkl_intel_lp64 -lmkl_intel_thread -lmkl_core -liomp5 -lpthread -ldl

Assuming your source files are sasumx.f common_func.f, ad executable is sasumx.out; run:

$ ifort sasumx.f common_func.f $LIBS -o sasumx.out

GNU compiler (such as gfortran)

Users may use the default GNU compiler or load a module using “module av comp/gcc”

MKL functionality is available through lib modules, which can be listed using “module av lib”

An example of loading a lib module:

$ module load lib/mkl/18.0.5.274

Note: Loading the module above will add the following to the LD_LIBRARY_PATH to:

/usr/local/intel/2018/compilers_and_libraries_2018.5.274/linux/mkl/lib/intel64
/usr/local/intel/2018/compilers_and_libraries_2018.5.274/linux/compiler/lib/intel64_lin

Set the $LIBS1 and LIBS2 variable to:

LIBS1="-L/usr/local/intel/2018/compilers_and_libraries_2018.5.274/linux/mkl/lib/intel64 -lmkl_gf_lp64 -lmkl_intel_thread -lmkl_core"
LIBS2="-L/usr/local/intel/2018/compilers_and_libraries_2018.5.274/linux/compiler/lib/intel64_lin -liomp5 -lpthread -ldl"

Assuming your source files are sasumx.f common_func.f, ad executable is sasumx.out; run:

$ gfortran -m64 -fcray-pointer sasumx.f common_func.f $LIBS1 $LIBS2 -o sasumx.out

PGI compiler

Once a PGI compiler module is loaded, set the LIBS1 and LIBS2 variable to:

LIBS1="-L/usr/local/intel/2018/compilers_and_libraries_2018.5.274/linux/mkl/lib/intel64 -lmkl_intel_lp64 -lmkl_intel_thread -lmkl_core"
LIBS2="-L/usr/local/intel/2018/compilers_and_libraries_2018.5.274/linux/compiler/lib/intel64_lin -liomp5 -lpthread -ldl"

Assuming your source files are sasumx.f common_func.f, ad executable is sasumx.out; run:

$ pgf95 sasumx.f common_func.f $LIBS1 $LIBS2 -o sasumx.out