nanolab.cnf.cornell.edu

Here you find information about the system http://www.cnf.cornell.edu/cnf5_tool.taf?_function=detail&eq_id=111.

The installation of user’s packages on nanolab EL4, 32-bit described below uses modules, and assumes bash shell:

  • packages are installed under ~/CAMd:

    mkdir ~/CAMd
cd ~/CAMd

  • module files are located under ~/CAMd/modulefiles

  • download the customize_nanolab_EL4.py file:

    wget https://svn.fysik.dtu.dk/projects/gpaw/trunk/doc/platforms/Linux/customize_nanolab_EL4.py

    scalapack = False

extra_compile_args = ['-O3', '-std=c99', '-fpic']

compiler = 'gcc'
mpicompiler = '/usr/local/openmpi-1.3.3/bin/mpicc'
mpilinker = mpicompiler

libraries = ['mkl_lapack', 'mkl_core', 'mkl_sequential', 'mkl_gf', 'iomp5']

mkl_lib_path = '/opt/intel/mkl/10.2.1.017/lib/32'
ompi_lib_path = '/usr/local/openmpi-1.3.3/lib'

library_dirs = [mkl_lib_path, ompi_lib_path]

extra_link_args =['-Wl,-rpath='+mkl_lib_path+',-rpath='+ompi_lib_path]

define_macros += [('GPAW_NO_UNDERSCORE_CBLACS', '1')]
define_macros += [('GPAW_NO_UNDERSCORE_CSCALAPACK', '1')]

  • download packages with download_nanolab.sh, buy running sh download_nanolab.sh:

    export APPS="/home/karsten/CAMd"
export MODULEFILES="${APPS}/modulefiles"

# download packages
nose_version=0.11.3
wget http://python-nose.googlecode.com/files/nose-${nose_version}.tar.gz
numpy_version=1.1.1
wget http://downloads.sourceforge.net/numpy/numpy-${numpy_version}.tar.gz
numpy_version=1.5.0
wget http://downloads.sourceforge.net/numpy/numpy-${numpy_version}.tar.gz
ase_version=3.4.1.1765
wget https://wiki.fysik.dtu.dk/ase-files/python-ase-${ase_version}.tar.gz
gpaw_version=0.7.2.6974
wget https://wiki.fysik.dtu.dk/gpaw-files/gpaw-${gpaw_version}.tar.gz
gpaw_setups_version=0.6.6300
wget http://wiki.fysik.dtu.dk/gpaw-files/gpaw-setups-${gpaw_setups_version}.tar.gz

  • from nanolab.cnf.cornell.edu login to one of c-nodes (Red Hat 4, 32-bit):

    ssh c7.cnf.cornell.edu

  • install packages, deploy modules and test with install_nanolab_EL4.sh, buy running sh install_nanolab_EL4.sh:

    export APPS="/home/karsten/CAMd"
export MODULEFILES="${APPS}/modulefiles"

# build packages

numpy_version=1.1.1
tar zxf numpy-${numpy_version}.tar.gz
cd  numpy-${numpy_version}
# disable compiling with atlas
sed -i "s/_lib_atlas =.*/_lib_atlas = ['ignore_atlas']/g" numpy/distutils/system_info.py
python setup.py install --root=${APPS}/numpy-${numpy_version}-1
cd ..

ase_version=3.4.1.1765
tar zxf python-ase-${ase_version}.tar.gz
# patch cPickle
sed -i "s/cPickle as //g" python-ase-${ase_version}/ase/io/trajectory.py

gpaw_version=0.7.2.6974
tar zxf gpaw-${gpaw_version}.tar.gz

gpaw_setups_version=0.6.6300
tar zxf gpaw-setups-${gpaw_setups_version}.tar.gz

mkdir -p ${MODULEFILES}/numpy
cat <<EOF > ${MODULEFILES}/numpy/${numpy_version}-1
#%Module1.0
set apps_path ${APPS}
prepend-path    PATH                \$apps_path/numpy-${numpy_version}-1/usr/bin
prepend-path    PYTHONPATH          \$apps_path/numpy-${numpy_version}-1/usr/lib/python2.3/site-packages/
unset apps_path
EOF

mkdir -p ${MODULEFILES}/campos-ase3
cat <<EOF > ${MODULEFILES}/campos-ase3/${ase_version}-1
#%Module1.0
set apps_path ${APPS}
prereq numpy
prepend-path    PATH                \$apps_path/python-ase-${ase_version}/tools
prepend-path    PYTHONPATH          \$apps_path/python-ase-${ase_version}/
unset apps_path
EOF

mkdir -p ${MODULEFILES}/campos-gpaw-setups
cat <<EOF > ${MODULEFILES}/campos-gpaw-setups/${gpaw_setups_version}-1
#%Module1.0
set apps_path ${APPS}
prepend-path    GPAW_SETUP_PATH     \$apps_path/gpaw-setups-${gpaw_setups_version}
unset apps_path
EOF

mkdir -p ${MODULEFILES}/campos-gpaw
cat <<EOF > ${MODULEFILES}/campos-gpaw/${gpaw_version}-1
#%Module1.0
set apps_path ${APPS}
prereq numpy
prereq campos-ase3
prereq campos-gpaw-setups
prereq intel_compilers/11.1
prereq openmpi/1.3.3
prepend-path    PATH                \$apps_path/gpaw-${gpaw_version}/tools
prepend-path    PATH                \$apps_path/gpaw-${gpaw_version}/build/bin.linux-i686-2.3
prepend-path    PYTHONPATH          \$apps_path/gpaw-${gpaw_version}/
prepend-path    PYTHONPATH          \$apps_path/gpaw-${gpaw_version}/build/lib.linux-i686-2.3
setenv OMP_NUM_THREADS 1
unset apps_path
EOF

module avail

module use --append ${MODULEFILES}
module load numpy
module load campos-ase3
# test numpy
python -c "import numpy; numpy.test()"
# test ase
mkdir -p testase
cd testase
testase.py --no-display 2>&1 | tee testase.log
cd ..
# build gpaw
cd gpaw-${gpaw_version}
python setup.py build_ext --customize=../customize_nanolab_EL4.py --remove-default-flags
cd ..
module load campos-gpaw-setups
module load intel_compilers/11.1
module load openmpi/1.3.3
module load campos-gpaw
mkdir -p testgpaw
cd testgpaw
mpiexec -np 4 gpaw-python `which gpaw-test` 2>&1 | tee testgpaw.log

    Note that every time you wish to install a new version of a package, and deploy new module file, better keep the old module file.

  • submit the test job:

    qsub submit.sh

    using the following submit.sh:

    TODO

  • to enable the installation permanently add the following to ~/.bashrc:

    module use --append /home/karsten/CAMd/modulefiles
module load numpy
module load campos-ase3
module load campos-gpaw-setups
module load intel_compilers/11.1
module load openmpi/1.3.3
module load campos-gpaw