Sophia cluster at HPC Risø¶
This document explains how to compile a developer version of GPAW on Sophia.
See also
Information about the Sophia cluster can be found at https://docs-devel.hpc.ait.dtu.dk/hardware/sophia/. Please note that the Sophia web-page is only accessible from the DTU network.
Creating the venv¶
Download the sophia-venv.sh
script and run it like this:
$ ./sophia-venv.sh <venv-name>
...
After a few minutes, you will have a <venv-name>
folder with
a GPAW installation inside.
Using the venv¶
In the following, we will assume that your venv folder is ~/venv1/
.
The venv needs to be activated like this:
$ source ~/venv1/bin/activate
and you can deactivate it when you no longer need to use it:
$ deactivate
You will want the activation to happen automatically for the jobs you submit to Sophia. Here are three ways to do it:
If you always want to use one venv then just put the activation command in your
~/.bashrc
.If you only want jobs running inside a certain folder to use the venv, then add this to your
~/.bashrc
:if [[ $SLURM_SUBMIT_DIR/ = $HOME/project-1* ]]; then source ~/venv1/bin/activate fi
Now, SLURM-jobs submitted inside your
~/project-1/
folder will use the venv.Use the “automatic discovery of venv’s” feature of MyQueue:
$ cd ~/project-1 $ ln -s ~/venv1 venv $ mq submit job.py
MyQueue will look for
venv/
folders (or soft-links as in the example) in one of the parent folders and activate the venv automatically when your job starts running.
Full script¶
#!/usr/bin/bash
set -e # stop if there are errors
NAME=$1
FOLDER=$PWD
echo '
export EASYBUILD_PREFIX=/groups/physics/modules
module use $EASYBUILD_PREFIX/modules/all
module purge
unset PYTHONPATH
module load matplotlib
module load spglib-python
module load libxc
module load libvdwxc
module load ScaLAPACK
module load GPAW-setups
' > modules.sh
. modules.sh
python3 -m venv $NAME
cd $NAME
VENV=$PWD
. bin/activate
PIP="python3 -m pip"
$PIP install --upgrade pip -qq
mv bin/activate old
mv ../modules.sh bin/activate
cat old >> bin/activate
rm old
git clone https://gitlab.com/ase/ase.git
$PIP install -e ase
git clone https://gitlab.com/asr-dev/asr.git
$PIP install -e asr
git clone https://gitlab.com/gpaw/gpaw.git
echo "
from os import environ
from pathlib import Path
scalapack = True
fftw = True
libraries = ['openblas', 'fftw3', 'readline', 'gfortran',
'scalapack', 'xc', 'vdwxc']
libxc = Path(environ['EBROOTLIBXC'])
include_dirs.append(libxc / 'include')
libvdwxc = Path(environ['EBROOTLIBVDWXC'])
include_dirs.append(libvdwxc / 'include')
library_dirs = environ['LD_LIBRARY_PATH'].split(':')
" > gpaw/siteconfig.py
pip install -e gpaw
$PIP install myqueue graphviz qeh ase-ext
gpaw install-data --basis --version=20000 . --no-register
export GPAW_SETUP_PATH=$GPAW_SETUP_PATH:$VENV/gpaw-basis-pvalence-0.9.20000
echo "export GPAW_SETUP_PATH=$GPAW_SETUP_PATH" >> bin/activate
ase completion >> bin/activate
gpaw completion >> bin/activate
mq completion >> bin/activate
$PIP completion --bash >> bin/activate
echo '
if [[ $SLURM_SUBMIT_DIR ]]; then
export MPLBACKEND=Agg
else
export MPLBACKEND=TkAgg
fi' >> bin/activate
mq --version
ase info
gpaw test