Niflheim

Information about the Niflheim cluster can be found at https://wiki.fysik.dtu.dk/niflheim.

Installing GPAW on all three architectures

Here, we install the development versions of ASE and GPAW in ~/ase and ~/gpaw. First, make sure you have this in your ~/.bashrc:

if [ -f /etc/bashrc ]; then
    . /etc/bashrc
fi
if [ -f /home/opt/modulefiles/modulefiles_el6.sh ]; then
    source /home/opt/modulefiles/modulefiles_el6.sh
fi
GPAW=~/gpaw
ASE=~/ase
if [ -z $FYS_PLATFORM ]
then
    module load GPAW
    module load matplotlib
    PLATFORM=linux-x86_64-$CPU_ARCH-el7-3.5
    alias gpaw-sbatch=$GPAW/doc/platforms/Linux/Niflheim/gpaw-sbatch.py
    if [ $CPU_ARCH = broadwell ]; then
        export GPAW_MPI_OPTIONS="-mca pml cm -mca mtl psm2"
    elif [ $CPU_ARCH = sandybridge ]; then
        export GPAW_MPI_OPTIONS=""
    elif [ $CPU_ARCH = nehalem ]; then
        export GPAW_MPI_OPTIONS=""
    fi
else
    cd $PBS_O_WORKDIR
    module load GPAW
    module load NUMPY/1.7.1-1
    module load SCIPY/0.12.0-1
    module load fftw
    PLATFORM=linux-x86_64-$FYS_PLATFORM-2.6
    alias gpaw-qsub=$GPAW/doc/platforms/Linux/Niflheim/gpaw-qsub.py
fi

export PATH=$GPAW/tools:$GPAW/build/bin.$PLATFORM:$PATH
export PYTHONPATH=$GPAW:$GPAW/build/lib.$PLATFORM:$PYTHONPATH
export PATH=$ASE/tools:$PATH
export PYTHONPATH=$ASE:$PYTHONPATH

Now, get the source code for ASE and GPAW:

$ cd
$ rm -rf gpaw ase
$ git clone https://gitlab.com/ase/ase.git
$ git clone https://gitlab.com/gpaw/gpaw.git

and compile GPAW’s C-extension using the compile.sh script:

$ cd gpaw
$ sh doc/platforms/Linux/Niflheim/compile.sh

Submit jobs to the queue with:

$ gpaw sbatch -- -p xeon8 -N 2 -n 16 my-script.py

Type gpaw sbatch -h for help.

Using more than one version of GPAW

Here we install an additional version of GPAW for, say, production runs:

$ cd
$ mkdir production
$ cd production
$ git clone https://gitlab.com/gpaw/gpaw.git
$ cd gpaw
$ git checkout 1.0.1
$ sh doc/platforms/Linux/Niflheim/compile.sh

Now you can submit jobs that use this production version with:

???