Diff for "Cluster_software_-_RPMS" - Niflheim Linux supercomputer cluster

Differences between revisions 1 and 23 (spanning 22 versions)

This page describes the necessary steps for installing the login nodes fjorm or thul.

Install external packages

As root:

create yum repository definitions (do not enable them):

# atrpms
echo '[atrpms]' > /etc/yum.repos.d/atrpms.repo
echo 'name=name=CentOS $releasever - $basearch - ATrpms' >> /etc/yum.repos.d/atrpms.repo
echo 'baseurl=http://dl.atrpms.net/el$releasever-$basearch/atrpms/stable' >> /etc/yum.repos.d/atrpms.repo
echo '#baseurl=http://mirrors.ircam.fr/pub/atrpms/el$releasever-$basearch/atrpms/stable' >> /etc/yum.repos.d/atrpms.repo
echo 'gpgkey=http://ATrpms.net/RPM-GPG-KEY.atrpms' >> /etc/yum.repos.d/atrpms.repo
echo 'gpgcheck=1' >> /etc/yum.repos.d/atrpms.repo
echo 'enabled=0' >> /etc/yum.repos.d/atrpms.repo
# epel
echo '[epel]' > /etc/yum.repos.d/epel.repo
echo 'name=name=CentOS $releasever - $basearch - EPEL' >> /etc/yum.repos.d/epel.repo
echo 'baseurl=http://download.fedora.redhat.com/pub/epel/$releasever/$basearch' >> /etc/yum.repos.d/epel.repo
echo 'gpgkey=http://download.fedora.redhat.com/pub/epel/RPM-GPG-KEY-EPEL' >> /etc/yum.repos.d/epel.repo
echo 'gpgcheck=1' >> /etc/yum.repos.d/epel.repo
echo 'enabled=0' >> /etc/yum.repos.d/epel.repo

install, as root:

yum install yum-utils
# /var directories must be created
yum search --enablerepo=atrpms arpack-devel
yum search --enablerepo=epel jmol

configure rpmbuild (as rpmbuild):

enable modules - add to ~/.bashrc:

if [ -r "/home/camp/modulefiles.sh" ]; then
    source /home/camp/modulefiles.sh
fi

set the FYS_PLATFORM variable:
```
export FYS_PLATFORM=Intel-Nehalem-el5 # thul
export FYS_PLATFORM=AMD-Opteron-el5 # fjorm
```
Note that this variable will be set automatically by /home/camp/modulefiles.sh after the environment-modules package is installed.

use the following ~rpmbuild/.rpmmacros:

%disttag        el5.fys

%packager       rpmbuild@fysik.dtu.dk
%distribution   Fysik RPMS
%vendor         Fysik RPMS <rpm@fysik.dtu.dk>

%_signature     gpg
%_gpg_path      ~/.gnupg
%_gpg_name      Fysik RPMS

%_topdir        /home/camp/rpmbuild/%(echo $FYS_PLATFORM)
%_rpmdir        %{_topdir}/RPMS
%_srcrpmdir     %{_topdir}/SRPMS
%_svndir        /home/camp/rpmbuild/rpmbuild
%_specdir       %{_svndir}/SPECS
%_sourcedir     %{_svndir}/SOURCES
%_rpmfilename   %%{NAME}-%%{VERSION}-%%{RELEASE}.%%{ARCH}.rpm
%_builddir      /tmp
%_tmppath      %{_topdir}/BUILD

# no debuginfo
%debug_package %{nil}

# don't strip (this does not fully work)
# https://bugzilla.redhat.com/show_bug.cgi?id=219731
%define __strip /bin/true

%niflheim       1

as rpmbuild create directories:

mkdir -p ~/${FYS_PLATFORM}/RPMS
mkdir -p ~/${FYS_PLATFORM}/SRPMS
mkdir -p ~/${FYS_PLATFORM}/BUILD
mkdir -p ~/${FYS_PLATFORM}/SPECS # needed only by openmpi
mkdir -p ~/${FYS_PLATFORM}/SOURCES # needed only by openmpi

install official packages, as rpmbuild:

cd ~/${FYS_PLATFORM}/RPMS
mkdir external; cd external
yumdownloader --resolve gcc-gfortran gcc43-c++ gcc43-gfortran blas-devel lapack-devel python-devel
yumdownloader --resolve gnuplot libXi-devel xorg-x11-fonts-100dpi pexpect tetex-latex tkinter qt-devel
yumdownloader --resolve openmpi openmpi-devel openmpi-libs compat-dapl libibverbs librdmacm openib
yumdownloader --resolve pygtk2-devel gtk2-devel tk-devel agg ghostscript libtiff-devel
yum localinstall * # as root

install atrpms packages, as rpmbuild (vtk-python is currently unavailable 16 Apr 2009):

cd ~/${FYS_PLATFORM}/RPMS/external
yumdownloader --resolve --enablerepo=atrpms vtk-python arpack-devel graphviz
wget http://ATrpms.net/RPM-GPG-KEY.atrpms
rpm --import RPM-GPG-KEY.atrpms # as root
yum localinstall * # as root

install the packages from epel, as rpmbuild:

~/${FYS_PLATFORM}/RPMS/external
yumdownloader --resolve --enablerepo=epel jmol
yumdownloader --resolve --enablerepo=epel environment-modules suitesparse-devel
wget http://download.fedora.redhat.com/pub/epel/RPM-GPG-KEY-EPEL
rpm --import RPM-GPG-KEY-EPEL # as root
yum localinstall * # as root

remove default openmpi:
```
yum remove openmpi openmpi-libs
```

edit /etc/yum.conf so it contains:

exclude=netcdf-* netcdf3-* fftw-* fftw2-* fftw3-* python-numeric openmpi-*

logout and login (as rpmbuild) again to activate modules settings from ~/.bashrc.

It's time to build custom RPMS

As rpmbuild:

cd ~/rpmbuild/SPECS

Install icc/ifort compilers, enable them (only on thul):

. /opt/intel/Compiler/11.0/083/bin/intel64/ifortvars_intel64.sh
. /opt/intel/Compiler/11.0/083/bin/intel64/iccvars_intel64.sh

Build intel compatibility packages (only on thul):

rpmbuild -bb --with modules --with default_version intel-redist.spec

Note: do not install the resulting RPMS on the login node. They need to be installed only on compute nodes. On the login node only the module file needs to be deployed (as root):

mkdir -p /etc/modulefiles/intel
cp /tmp/intel-11.0.083/11.0.083-1.intel64.el5.fys /etc/modulefiles/intel

Note: the above module file should contain at least:

prepend-path    LD_LIBRARY_PATH      /opt/intel/Compiler/11.0/083/lib/intel64
prepend-path    PATH                 /opt/intel/Compiler/11.0/083/bin/intel64
prepend-path    MANPATH              /opt/intel/Compiler/11.0/083/man

Install mkl, and build mkl compatibility package (only on thul):

cd ~/rpmbuild/SOURCES
mkdir 10.1.3.027
cp -r /opt/intel/mkl/10.1.3.027/lib 10.1.3.027
cp -r /opt/intel/mkl/10.1.3.027/doc 10.1.3.027
tar zcf intel-redist-mkl-10.1.3.027.tar.gz 10.1.3.027

cd ~/rpmbuild/SPECS
rpmbuild -bb --with modules --with default_version intel-redist-mkl.spec

Note: do not install the resulting RPM on the login node. It needs to be installed only on compute nodes. On the login node only the module file needs to be deployed (as root):

mkdir -p /etc/modulefiles/mkl
cp /tmp/intel-mkl-10.1.3.027/10.1.3.027-1.el5.fys.em64t /etc/modulefiles/mkl

Note: the above module file should contain at least:

prepend-path    LD_LIBRARY_PATH      /opt/intel/mkl/10.1.3.027/lib/em64t

Build a custom openmpi, using torque support:

export rpmtopdir=${HOME}/${FYS_PLATFORM} # set this to _topdir value from ~/.rpmmacros
wget http://www.open-mpi.org/software/ompi/v1.3/downloads/openmpi-1.3.2.tar.bz2 \
     -O ~/rpmbuild/SOURCES/openmpi-1.3.2.tar.bz2
sh ./buildrpm-1.3.2-1.gfortran.sh ../SOURCES/openmpi-1.3.2.tar.bz2 2>&1 | tee buildrpm-1.3.2-1.gfortran.sh.log.${FYS_PLATFORM}
sh ./buildrpm-1.3.2-1.gfortran43.sh ../SOURCES/openmpi-1.3.2.tar.bz2 2>&1 | tee buildrpm-1.3.2-1.gfortran43.sh.log.${FYS_PLATFORM}
sh ./buildrpm-1.3.2-1.pathscale.sh ../SOURCES/openmpi-1.3.2.tar.bz2 2>&1 | tee buildrpm-1.3.2-1.pathscale.sh.log.${FYS_PLATFORM}
sh ./buildrpm-1.3.2-1.ifort.sh ../SOURCES/openmpi-1.3.2.tar.bz2 2>&1 | tee buildrpm-1.3.2-1.ifort.sh.log.${FYS_PLATFORM} # thul only

Note: intel openmpi needs to be installed ignoring dependencies:

rpm -ivh --nodeps --oldpackage ~rpmbuild/${FYS_PLATFORM}/RPMS/openmpi-1.3.2-1.ifort.el5.fys.x86_64.rpm

If scripts that contain ALL build/install/uninstall commands (global_install.sh and global_uninstall.sh) need to be created, every time after an RPM is successfully built, do:

grep -v "#\!" install.sh >> ~/${FYS_PLATFORM}/global_install.sh
cat uninstall.sh ~/${FYS_PLATFORM}/global_uninstall.sh | grep -v "#\!" >> ~/${FYS_PLATFORM}/global_uninstall.sh.tmp && mv -f ~/${FYS_PLATFORM}/global_uninstall.sh.tmp ~/${FYS_PLATFORM}/global_uninstall.sh
# ignore "cat: .../global_uninstall.sh: No such ..." error when running first time

Note that global_uninstall.sh won't remove built RPM files, just will uninstall the packages.

Build the following for dacapo:

set the disttag variable for convenience:
```
export disttag="el5.fys"
```

acml:

rpmbuild -bb --with compiler=gfortran --with version1=0 --with version2=1 --with modules --with default_version acml.spec
rpmbuild -bb --with compiler=pathscale --with version1=0 --with version2=1 --with modules --with default_version acml.spec

rpmbuild -bb --with compiler=gfortran43 --with version1=1 --with version2=0 --with modules --with default_version acml.spec # fjorm only
rpmbuild -bb --with compiler=pathscale --with version1=1 --with version2=0 --with modules --with default_version acml.spec

rpmbuild -bb --with compiler=gfortran43 --with version1=2 --with version2=0 --with modules --with default_version acml.spec
rpmbuild -bb --with compiler=pathscale --with version1=2 --with version2=0 --with modules --with default_version acml.spec
rpmbuild -bb --with compiler=ifort --with version1=2 --with version2=0 --with modules --with default_version acml.spec # thul only

goto (only on fjorm):

rpmbuild --bb --with compiler=gfortran --with modules=1 --with default_version=1 \
              --with prefix=/opt/goto/1.26/1.${disttag}.gfortran.smp goto.spec
rpmbuild --bb --with compiler=gfortran43 --with modules=1 --with default_version=1 \
              --with prefix=/opt/goto/1.26/1.${disttag}.gfortran43.smp goto.spec
rpmbuild --bb --with compiler=pathscale --with compiler_bindir=/opt/pathscale/bin --with compiler_libdir=/opt/pathscale/lib/3.2 \
              --with modules=1 --with default_version=1 --with prefix=/opt/goto/1.26/1.${disttag}.pathscale.smp goto.spec

Note - 1.26 version fails on Nehalem with:

../../../param.h:1195:21: error: division by zero in #if

atlas:

rpmbuild --bb  --with compiler=gfortran --with compiler_bindir=/usr/bin --with compiler_libdir=/usr/lib64 \
         -with modules=1 --with default_version=1 --with prefix=/opt/atlas/3.8.3/1.gfortran.${disttag} atlas.spec

rpmbuild --bb  --with compiler=gfortran43 --with compiler_bindir=/usr/bin --with compiler_libdir=/usr/lib64 \
         -with modules=1 --with default_version=1 --with prefix=/opt/atlas/3.8.3/1.gfortran43.${disttag} atlas.spec

rpmbuild --bb  --with compiler=pathscale --with compiler_bindir=/opt/pathscale/bin --with compiler_libdir=/opt/pathscale/lib/3.2 \
         -with modules=1 --with default_version=1 --with prefix=/opt/atlas/3.8.3/1.pathscale.${disttag} atlas.spec

campos-dacapo-pseudopotentials:

rpmbuild -bb --with modules --with default_version campos-dacapo-pseudopotentials.spec

rasmol:

rpmbuild -bb --with modules --with default_version --with prefix=/opt/RasMol/2.7.3/3.${disttag} RasMol.spec

cblas:

rpmbuild --bb  --with blas_version=3.0.37.el5 --with modules=1 --with default_version=1 \
               --with prefix=/opt/cblas/2.23.3/1.${disttag}.gfortran.blas.3.0.37.el5 cblas.spec

rpmbuild --bb  --with blas=acml --with blas_version=4.0.1 --with blasdir=/opt/acml/4.0.1/gfortran64/lib \
               --with modules=1 --with default_version=1 --with prefix=/opt/cblas/2.23.3/1.${dist}.gfortran.acml.4.0.1 cblas.spec

python-setuptools:

rpmbuild -bb --with modules --with default_version --with prefix=/opt/python-setuptools/0.6c9/1.${disttag}.python2.4 python-setuptools.spec

python-nose:

rpmbuild -bb --with modules --with default_version --with prefix=/opt/python-nose/0.10.4/1.${disttag}.python2.4 python-nose.spec
module load python-setuptools
module load python-nose

numpy:

rpmbuild --bb --with cblas_prefix=none \ # dotblas fails with acml
              --with blas=acml --with blas_version=4.0.1 --with blasdir=/opt/acml/4.0.1/gfortran64/lib \
              --with lapack=acml --with lapackdir=/opt/acml/4.0.1/gfortran64/lib \
              --with compiler=gfortran --with modules=1 --with default_version=1 \
              --with prefix=/opt/numpy/1.3.0/1.${disttag}.gfortran.python2.4.acml.4.0.1.acml numpy.spec

rpmbuild --bb --with cblas_prefix=/opt/cblas/2.23.3/1.${disttag}.gfortran.blas.3.0.37.el5 --with blas_version=3.0.37.el5 \
              --with compiler=gfortran --with modules=1 --with default_version=1 \
              --with prefix=/opt/numpy/1.3.0/1.${disttag}.gfortran.python2.4.blas.3.0.37.el5.lapack numpy.spec

Test with:

module load numpy/1.3.0-1.el5.fys.gfortran.python2.4.blas.3.0.37.el5.lapack
python -c "import numpy; from numpy.core.multiarray import dot; b = numpy.ones(13, numpy.complex); dot(b, b)"
python -c "import numpy; numpy.test()"
module unload numpy/1.3.0-1.el5.fys.gfortran.python2.4.blas.3.0.37.el5.lapack

module load acml-gfortran64/4.0.1-1.el5.fys
module load numpy/1.3.0-1.el5.fys.gfortran.python2.4.acml.4.0.1.acml
python -c "import numpy; from numpy.core.multiarray import dot; b = numpy.ones(13, numpy.complex); dot(b, b)"
python -c "import numpy; numpy.test()"
module unload numpy/1.3.0-1.el5.fys.gfortran.python2.4.acml.4.0.1.acml
module unload acml-gfortran64/4.0.1-1.el5.fys

Load the default numpy:

module load numpy/1.3.0-1.el5.fys.gfortran.python2.4.acml.4.0.1.acml

gnuplot-py:

rpmbuild -bb --with modules --with default_version --with prefix=/opt/gnuplot-py/1.8.1/1.${disttag}.python2.4 gnuplot-py.spec

module load gnuplot-py

python-numeric (we must install 24.2 version, and we keep the default version):

cd ~/${FYS_PLATFORM}/RPMS/external
rpm -e --nodeps python-numeric # as root
yumdownloader --resolve --disableexcludes=main python-numeric
cd ~/rpmbuild/SPECS

rpmbuild --bb --with cblas_prefix=/opt/cblas/2.23.3/1.${disttag}.gfortran.acml.4.0.1 \
              --with blas=acml --with blas_version=4.0.1 --with blasdir=/opt/acml/4.0.1/gfortran64/lib \
              --with lapack=acml --with lapackdir=/opt/acml/4.0.1/gfortran64/lib \
              --with compiler=gfortran --with modules=1 --with default_version=1 \
              --with prefix=/opt/python-numeric/24.2/4.${disttag}.gfortran.python2.4.acml.4.0.1.acml python-numeric.spec

rpmbuild --bb --with cblas_prefix=/opt/cblas/2.23.3/1.${disttag}.gfortran.blas.3.0.37.el5 --with blas_version=3.0.37.el5 \
              --with compiler=gfortran --with modules=1 --with default_version=1 \
              --with prefix=/opt/python-numeric/24.2/4.${disttag}.gfortran.python2.4.blas.3.0.37.el5.lapack python-numeric.spec

Note: (16 Apr 2009) currently Numeric's test.py results in (we ignore this error):

glibc detected *** python: free(): invalid next size (normal): 0x09aee970 ***

After installing python-numeric make a very rough check:

module load python-numeric/24.2-4.el5.fys.gfortran.python2.4.blas.3.0.37.el5.lapack
python -c "import lapack_lite"
ldd `rpm -ql python-numeric-24.2-4.el5.fys.gfortran.python2.4.blas.3.0.37.el5.lapack | grep lapack_lite.so`
ldd `rpm -ql python-numeric-24.2-4.el5.fys.gfortran.python2.4.blas.3.0.37.el5.lapack | grep _dotblas.so`
module unload python-numeric/24.2-4.el5.fys.gfortran.python2.4.blas.3.0.37.el5.lapack

module load acml-gfortran64/4.0.1-1.el5.fys
module load python-numeric/24.2-4.el5.fys.gfortran.python2.4.acml.4.0.1.acml
python -c "import lapack_lite"
ldd `rpm -ql python-numeric-24.2-4.el5.fys.gfortran.python2.4.acml.4.0.1.acml | grep lapack_lite.so`
ldd `rpm -ql python-numeric-24.2-4.el5.fys.gfortran.python2.4.acml.4.0.1.acml | grep _dotblas.so`
module unload python-numeric/24.2-4.el5.fys.gfortran.python2.4.acml.4.0.1.acml
module unload acml-gfortran64/4.0.1-1.el5.fys

and reinstall the default version:

rpm -ivh --oldpackage ~rpmbuild/${FYS_PLATFORM}/RPMS/external/python-numeric-23.*.rpm

load the default Numeric version:

module load acml-gfortran64/4.0.1-1.el5.fys
module load python-numeric/24.2-4.el5.fys.gfortran.python2.4.acml.4.0.1.acml

netcdf:

rpmbuild --bb  --with compiler=gfortran --with modules=1 --with default_version=1 \
               --with prefix=/opt/netcdf4/4.0.1/1.gfortran.${disttag} netcdf4.spec
rpmbuild --bb  --with compiler=gfortran43 --with modules=1 --with default_version=1 \
               --with prefix=/opt/netcdf4/4.0.1/1.gfortran43.${disttag} netcdf4.spec
rpmbuild --bb  --with compiler=pathscale --with compiler_bindir=/opt/pathscale/bin --with compiler_libdir=/opt/pathscale/lib/3.2 \
               --with modules=1 --with default_version=1 --with prefix=/opt/netcdf4/4.0.1/1.pathscale.${disttag} netcdf4.spec

ScientificPython:

rpmbuild --bb  --with Numeric_includedir=none --with numpy=numpy --with compiler=gfortran --with netcdf=netcdf4 \
               --with netcdf_includedir=/opt/netcdf4/4.0.1/1.gfortran.${disttag}/include \
               --with netcdf_libdir=/opt/netcdf4/4.0.1/1.gfortran.${disttag}/lib64 \
               --with modules=1 --with default_version=1 \
               --with prefix=/opt/ScientificPython/2.8/1.${disttag}.python2.4.serial_version.numpy ScientificPython.spec

rpmbuild --bb  --with numpy=numeric --with compiler=gfortran --with netcdf=netcdf4 \
               --with netcdf_includedir=/opt/netcdf4/4.0.1/1.gfortran.${disttag}/include \
               --with netcdf_libdir=/opt/netcdf4/4.0.1/1.gfortran.${disttag}/lib64 \
               --with modules=1 --with default_version=1 --with parallel=1 --with openmpi=openmpi \
               --with ompi_bindir=/opt/openmpi/1.3.2-1.gfortran/bin \
               --with ompi_includedir=/opt/openmpi/1.3.2-1.gfortran/include --with ompi_libdir=/opt/openmpi/1.3.2-1.gfortran/lib64 \
               --with prefix=/opt/ScientificPython/2.6.2/1.${disttag}.python2.4.openmpi.numeric ScientificPython.spec

python-docutils:

rpmbuild -bb --with modules --with default_version --with prefix=/opt/python-docutils/0.5/1.${disttag}.python2.4 python-docutils.spec

pytz:

rpmbuild -bb --with modules --with default_version --with prefix=/opt/pytz/2008g/1.${disttag}.python2.4 pytz.spec

python-docutils:

rpmbuild -bb --with modules --with default_version --with prefix=/opt/python-dateutil/1.4.1/3.${disttag}.python2.4 python-dateutil.spec

python-matplotlib:

module load pytz
module load python-docutils
module load python-dateutil
rpmbuild -bb --with modules --with default_version --with prefix=/opt/python-matplotlib/0.98.5.2/1.${disttag}.python2.4 python-matplotlib.spec

campos-ase2:

rpmbuild -bb --with modules --with default_version --with prefix=/opt/campos-ase2/2.3.13/1.${disttag}.python2.4 campos-ase2.spec

campos-dacapo-python:

rpmbuild -bb --with modules --with default_version --with prefix=/opt/campos-dacapo-python/0.9.4/1.${disttag}.python2.4 campos-dacapo-python.spec

fftw2:

rpmbuild --bb  --with compiler=gfortran --with default_version=1 --with modules=1 --with prefix=/opt/fftw2/2.1.5/12.gfortran.${disttag} fftw2.spec
rpmbuild --bb  --with compiler=gfortran43 --with default_version=1 --with modules=1 --with prefix=/opt/fftw2/2.1.5/12.gfortran43.${disttag} fftw2.spec
rpmbuild --bb  --with compiler=pathscale --with compiler_bindir=/opt/pathscale/bin --with compiler_libdir=/opt/pathscale/lib/3.2 \
               --with default_version=1 --with modules=1 --with prefix=/opt/fftw2/2.1.5/12.pathscale.${disttag} fftw2.spec

campos-dacapo:

rpmbuild --bb  --with blas=acml --with blas_version=4.2.0 --with blasdir=/opt/acml/4.2.0/gfortran4364/lib \
               --with compiler=gfortran43 --with compiler_bindir=/usr/bin --with compiler_libdir=/usr/lib64 \
               --with fftw=fftw2 --with fftw_libdir=/opt/fftw2/2.1.5/12.gfortran43.${disttag}/lib64 \
               --with lapack=acml --with lapackdir=/opt/acml/4.2.0/gfortran4364/lib --with netcdf=netcdf4 \
               --with netcdf_includedir=/opt/netcdf4/4.0.1/1.gfortran43.${disttag}/include \
               --with netcdf_libdir=/opt/netcdf4/4.0.1/1.gfortran43.${disttag}/lib64 \
               --with modules=1 --with default_version=1 --with parallel=1 --with openmpi=openmpi \
               --with ompi_bindir=/opt/openmpi/1.3.2-1.gfortran43/bin \
               --with ompi_includedir=/opt/openmpi/1.3.2-1.gfortran43/include --with ompi_libdir=/opt/openmpi/1.3.2-1.gfortran43/lib64 \
               --with prefix=/opt/campos-dacapo/2.7.16/1.${disttag}.gfortran43.openmpi.acml.4.2.0.acml campos-dacapo.spec

rpmbuild --bb  --with blas=acml --with blas_version=4.0.1 --with blasdir=/opt/acml/4.0.1/gfortran64/lib \
               --with compiler=gfortran --with compiler_bindir=/usr/bin --with compiler_libdir=/usr/lib64 \
               --with fftw=fftw2 --with fftw_libdir=/opt/fftw2/2.1.5/12.gfortran.${disttag}/lib64 \
               --with lapack=acml --with lapackdir=/opt/acml/4.0.1/gfortran64/lib --with netcdf=netcdf4 \
               --with netcdf_includedir=/opt/netcdf4/4.0.1/1.gfortran.${disttag}/include \
               --with netcdf_libdir=/opt/netcdf4/4.0.1/1.gfortran.${disttag}/lib64 \
               --with modules=1 --with default_version=1 --with parallel=1 --with openmpi=openmpi \
               --with ompi_bindir=/opt/openmpi/1.3.2-1.gfortran/bin \
               --with ompi_includedir=/opt/openmpi/1.3.2-1.gfortran/include --with ompi_libdir=/opt/openmpi/1.3.2-1.gfortran/lib64 \
               --with prefix=/opt/campos-dacapo/2.7.16/1.${disttag}.gfortran.openmpi.acml.4.0.1.acml campos-dacapo.spec

rpmbuild --bb  --with blas=blas --with blas_version=3.0.37.el5 --with blasdir=/usr/lib64 \
               --with compiler=gfortran43 --with compiler_bindir=/usr/bin --with compiler_libdir=/usr/lib64 \
               --with fftw=fftw2 --with fftw_libdir=/opt/fftw2/2.1.5/12.gfortran43.${disttag}/lib64 \
               --with lapack=lapack --with lapackdir=/usr/lib64 --with netcdf=netcdf4 \
               --with netcdf_includedir=/opt/netcdf4/4.0.1/1.gfortran43.${disttag}/include \
               --with netcdf_libdir=/opt/netcdf4/4.0.1/1.gfortran43.${disttag}/lib64 \
               --with modules=1 --with default_version=1 --with parallel=1 --with openmpi=openmpi \
               --with ompi_bindir=/opt/openmpi/1.3.2-1.gfortran43/bin \
               --with ompi_includedir=/opt/openmpi/1.3.2-1.gfortran43/include --with ompi_libdir=/opt/openmpi/1.3.2-1.gfortran43/lib64 \
               --with prefix=/opt/campos-dacapo/2.7.16/1.${disttag}.gfortran43.openmpi.blas.3.0.37.el5.lapack campos-dacapo.spec

rpmbuild --bb  --with blas=blas --with blas_version=3.0.37.el5 --with blasdir=/usr/lib64 \
               --with compiler=gfortran --with compiler_bindir=/usr/bin --with compiler_libdir=/usr/lib64 \
               --with fftw=fftw2 --with fftw_libdir=/opt/fftw2/2.1.5/12.gfortran.${disttag}/lib64 \
               --with lapack=lapack --with lapackdir=/usr/lib64 --with netcdf=netcdf4 \
               --with netcdf_includedir=/opt/netcdf4/4.0.1/1.gfortran.${disttag}/include \
               --with netcdf_libdir=/opt/netcdf4/4.0.1/1.gfortran.${disttag}/lib64 \
               --with modules=1 --with default_version=1 --with parallel=1 --with openmpi=openmpi \
               --with ompi_bindir=/opt/openmpi/1.3.2-1.gfortran/bin \
               --with ompi_includedir=/opt/openmpi/1.3.2-1.gfortran/include --with ompi_libdir=/opt/openmpi/1.3.2-1.gfortran/lib64 \
               --with prefix=/opt/campos-dacapo/2.7.16/1.${disttag}.gfortran.openmpi.blas.3.0.37.el5.lapack campos-dacapo.spec

Build following for gpaw:

fftw3:

rpmbuild --bb --with major_version=3 --with version1=2 --with version2=1 --with compiler=gfortran \
              --with default_version=1 --with modules=1 --with prefix=/opt/fftw3/3.2.1/12.gfortran.${disttag} fftw2.spec
rpmbuild --bb --with major_version=3 --with version1=2 --with version2=1 --with compiler=gfortran43 \
              --with default_version=1 --with modules=1 --with prefix=/opt/fftw3/3.2.1/12.gfortran43.${disttag} fftw2.spec

scipy:

module unload numpy
module load numpy/1.3.0-1.el5.fys.gfortran.python2.4.blas.3.0.37.el5.lapack

rpmbuild --bb --with cblas_prefix=/opt/cblas/2.23.3/1.${disttag}.gfortran.blas.3.0.37.el5 \
              --with blas=blas --with blas_version=3.0.37.el5 --with blasdir=/usrlib64 \
              --with lapack=lapack --with lapackdir=/usr/lib64 \
              --with ufsparse_libdir=/usr/lib64 --with ufsparse_includedir=/usr/include/suitesparse \
              --with fftw_libdir=/opt/fftw3/3.2.1/12.gfortran.${disttag}/lib64 \
              --with compiler=gfortran --with default_version=1 --with modules=1 \
              --with prefix=/opt/scipy/0.7.0/1.${disttag}.gfortran.python2.4.blas.3.0.37.el5.lapack scipy.spec

module unload numpy
module load numpy/1.3.0-1.el5.fys.gfortran.python2.4.acml.4.0.1.acml

rpmbuild --bb --with cblas_prefix=/opt/cblas/2.23.3/1.${disttag}.gfortran.acml.4.0.1 \
              --with blas=acml --with blas_version=4.0.1 --with blasdir=/opt/acml/4.0.1/gfortran64/lib \
              --with lapack=acml --with lapackdir=/opt/acml/4.0.1/gfortran64/lib \
              --with ufsparse_libdir=/usr/lib64 --with ufsparse_includedir=/usr/include/suitesparse \
              --with fftw_libdir=/opt/fftw3/3.2.1/12.gfortran.${disttag}/lib64 \
              --with compiler=gfortran --with default_version=1 --with modules=1 \
              --with prefix=/opt/scipy/0.7.0/1.${disttag}.gfortran.python2.4.acml.4.0.1.acml scipy.spec

campos-gpaw-setups:

rpmbuild -bb --with default_version --with modules campos-gpaw-setups.spec

campos-ase3:

rpmbuild -bb --with modules --with default_version --with prefix=/opt/campos-ase3/3.1.0.846/1.${disttag}.python2.4 campos-ase3.spec

povray:

rpmbuild -bb --with modules --with default_version --with prefix=/opt/povray/3.6.1/3.${disttag} povray.spec

python-jinja2:

rpmbuild -bb --with modules --with default_version --with prefix=/opt/python-jinja2/2.1.1/1.${disttag}.python2.4 python-jinja2.spec

python-pygments:

rpmbuild -bb --with modules --with default_version --with prefix=/opt/python-pygments/1.0/1.${disttag}.python2.4 python-pygments.spec

babel:

rpmbuild -bb --with modules --with default_version --with prefix=/opt/babel/0.9.4/1.${disttag}.python2.4 babel.spec

python-sphinx:

module load python-jinja2
rpmbuild -bb --with modules --with default_version --with prefix=/opt/python-sphinx/0.6.1/1.${disttag}.python2.4 python-sphinx.spec

blacs:

rpmbuild --bb --with compiler=gfortran43 --with compiler_bindir=/usr/bin --with compiler_libdir=/usr/lib64 \
               --with modules=1 --with default_version=1 --with parallel=1 --with openmpi=openmpi \
               --with ompi_bindir=/opt/openmpi/1.3.2-1.gfortran43/bin \
               --with ompi_includedir=/opt/openmpi/1.3.2-1.gfortran43/include --with ompi_libdir=/opt/openmpi/1.3.2-1.gfortran43/lib64 \
               --with prefix=/opt/blacs/1.1/24.${disttag}.gfortran43.openmpi blacs.spec

rpmbuild --bb  --with compiler=gfortran --with compiler_bindir=/usr/bin --with compiler_libdir=/usr/lib64 \
               --with modules=1 --with default_version=1 --with parallel=1 --with openmpi=openmpi \
               --with ompi_bindir=/opt/openmpi/1.3.2-1.gfortran/bin \
               --with ompi_includedir=/opt/openmpi/1.3.2-1.gfortran/include --with ompi_libdir=/opt/openmpi/1.3.2-1.gfortran/lib64 \
               --with prefix=/opt/blacs/1.1/24.${disttag}.gfortran.openmpi blacs.spec

scalapack:

rpmbuild --bb  --with blacs=blacs --with blacsdir=/opt/blacs/1.1/24.${disttag}.gfortran43.openmpi/lib64 \
               --with blas=acml --with blas_version=4.2.0 --with blasdir=/opt/acml/4.2.0/gfortran4364/lib \
               --with compiler=gfortran43 --with compiler_bindir=/usr/bin --with compiler_libdir=/usr/lib64 \
               --with default_version=1 --with modules=1 --with openmpi=openmpi \
               --with lapack=acml --with lapackdir=/opt/acml/4.2.0/gfortran4364/lib \
               --with ompi_bindir=/opt/openmpi/1.3.2-1.gfortran43/bin \
               --with ompi_includedir=/opt/openmpi/1.3.2-1.gfortran43/include --with ompi_libdir=/opt/openmpi/1.3.2-1.gfortran43/lib64 \
               --with prefix=/opt/scalapack/1.8.0/1.${disttag}.gfortran43.openmpi.acml.4.2.0.acml scalapack.spec

rpmbuild --bb  --with blacs=blacs --with blacsdir=/opt/blacs/1.1/24.${disttag}.gfortran.openmpi/lib64 \
               --with blas=acml --with blas_version=4.0.1 --with blasdir=/opt/acml/4.0.1/gfortran64/lib \
               --with compiler=gfortran --with compiler_bindir=/usr/bin --with compiler_libdir=/usr/lib64 \
               --with default_version=1 --with modules=1 --with openmpi=openmpi \
               --with lapack=acml --with lapackdir=/opt/acml/4.0.1/gfortran64/lib \
               --with ompi_bindir=/opt/openmpi/1.3.2-1.gfortran/bin \
               --with ompi_includedir=/opt/openmpi/1.3.2-1.gfortran/include --with ompi_libdir=/opt/openmpi/1.3.2-1.gfortran43/lib \
               --with prefix=/opt/scalapack/1.8.0/1.${disttag}.gfortran.openmpi.acml.4.0.1.acml scalapack.spec

rpmbuild --bb  --with blacs=blacs --with blacsdir=/opt/blacs/1.1/24.${disttag}.gfortran43.openmpi/lib64 \
               --with blas=blas --with blas_version=3.0.37.el5 --with blasdir=/usr/lib64 \
               --with compiler=gfortran43 --with compiler_bindir=/usr/bin --with compiler_libdir=/usr/lib64 \
               --with default_version=1 --with modules=1 --with openmpi=openmpi \
               --with lapack=lapack --with lapackdir=/usr/lib64 \
               --with ompi_bindir=/opt/openmpi/1.3.2-1.gfortran43/bin \
               --with ompi_includedir=/opt/openmpi/1.3.2-1.gfortran43/include --with ompi_libdir=/opt/openmpi/1.3.2-1.gfortran43/lib64 \
               --with prefix=/opt/scalapack/1.8.0/1.${disttag}.gfortran43.openmpi.blas.3.0.37.el5.lapack scalapack.spec

rpmbuild --bb  --with blacs=blacs --with blacsdir=/opt/blacs/1.1/24.${disttag}.gfortran.openmpi/lib64 \
               --with blas=blas --with blas_version=3.0.37.el5 --with blasdir=/usr/lib64 \
               --with compiler=gfortran --with compiler_bindir=/usr/bin --with compiler_libdir=/usr/lib64 \
               --with default_version=1 --with modules=1 --with openmpi=openmpi \
               --with lapack=lapack --with lapackdir=/usr/lib64 \
               --with ompi_bindir=/opt/openmpi/1.3.2-1.gfortran/bin \
               --with ompi_includedir=/opt/openmpi/1.3.2-1.gfortran/include --with ompi_libdir=/opt/openmpi/1.3.2-1.gfortran/lib64 \
               --with prefix=/opt/scalapack/1.8.0/1.${disttag}.gfortran.openmpi.blas.3.0.37.el5.lapack scalapack.spec

campos-gpaw:

rpmbuild --bb  --with blacs=blacs --with blacsdir=/opt/blacs/1.1/24.${disttag}.gfortran.openmpi/lib64 \
               --with blas=acml --with blas_version=4.0.1 --with blasdir=/opt/acml/4.0.1/gfortran64/lib \
               --with cmr_repository=/home/niflheim/repository/db \
               --with compiler=gfortran --with compiler_bindir=/usr/bin --with compiler_libdir=/usr/lib64 \
               --with default_version=1 --with modules=1 \
               --with lapack=acml --with lapackdir=/opt/acml/4.0.1/gfortran64/lib \
               --with ompi_bindir=/opt/openmpi/1.3.2-1.gfortran/bin \
               --with ompi_includedir=/opt/openmpi/1.3.2-1.gfortran/include --with ompi_libdir=/opt/openmpi/1.3.2-1.gfortran/lib64 \
               --with openmpi=openmpi --with parallel=1 \
               --with scalapack=scalapack --with scalapackdir=/opt/scalapack/1.8.0/1.${disttag}.gfortran.openmpi.acml.4.0.1.acml/lib64 \
               --with prefix=/opt/campos-gpaw/0.6.3862/1.${disttag}.gfortran.python2.4.openmpi.acml.4.0.1.acml campos-gpaw.spec

rpmbuild --bb  --with blacs=blacs --with blacsdir=/opt/blacs/1.1/24.${disttag}.gfortran43.openmpi/lib64 \
               --with blas=acml --with blas_version=4.2.0 --with blasdir=/opt/acml/4.2.0/gfortran4364/lib \
               --with cmr_repository=/home/niflheim/repository/db \
               --with compiler=gfortran43 --with compiler_bindir=/usr/bin --with compiler_libdir=/usr/lib64 \
               --with default_version=1 --with modules=1 \
               --with lapack=acml --with lapackdir=/opt/acml/4.2.0/gfortran4364/lib \
               --with ompi_bindir=/opt/openmpi/1.3.2-1.gfortran43/bin \
               --with ompi_includedir=/opt/openmpi/1.3.2-1.gfortran43/include --with ompi_libdir=/opt/openmpi/1.3.2-1.gfortran43/lib64 \
               --with openmpi=openmpi --with parallel=1 \
               --with scalapack=scalapack --with scalapackdir=/opt/scalapack/1.8.0/1.${disttag}.gfortran43.openmpi.acml.4.2.0.acml/lib64 \
               --with prefix=/opt/campos-gpaw/0.6.3862/1.${disttag}.gfortran43.python2.4.openmpi.acml.4.2.0.acml campos-gpaw.spec

Testing packages

Test dacapo installation (as normal user!).

If you use modules:

module load openmpi
module load campos-dacapo-pseudopotentials
module load python-numeric
module load campos-dacapo-python
module load ScientificPython
module load gnuplot-py
module load RasMol
module load campos-ase2
module load campos-dacapo
ulimit -s 65000 # dacapo needs a large stack

Test with (make sure that /scratch/$USER exists):

cp -r `rpm -ql campos-dacapo-python | grep "share/campos-dacapo-python$"` /tmp
cd /tmp/campos-dacapo-python/Tests
python test.py 2>&1 | tee test.log

It can take up to 1 day. Please consider disabling these "long" tests in test.py:

tests.remove('../Examples/Wannier-ethylene.py')
tests.remove('../Examples/Wannier-Pt4.py')
tests.remove('../Examples/Wannier-Ptwire.py')
tests.remove('../Examples/Wannier-Fe-bcc.py')
tests.remove('../Examples/transport_1dmodel.py')

Note all vtk related tests will fail.

Test gpaw installation (as normal user!):

If you use modules:

module load openmpi
module load campos-ase3
module load campos-gpaw-setups
module load campos-gpaw

Test with:

cp -r `rpm -ql campos-gpaw | grep "share/campos-gpaw/test$"` /tmp/test.gpaw.$$
cd /tmp/test.gpaw.*
python test.py 2>&1 | tee test.log

It takes about 20 minutes.

On "Golden Client"

ssh n001

Enable nfs mount of the server home directory - follow 'Enable nfs mount on the "Golden Client"' from configuring NFS. After this do:

cd /home/dulak-server/rpm/campos

rpm -ivh campos-dacapo-2*

If getting:

package example_package.el5.i386 is already installed

remove these packages with:

rpm -e --nodeps example_package

to allow the installation to proceed.

Make sure that both python-numeric versions are installed:

rpm -q python-numeric

This command will show a list of packages that need to be installed to fulfill dacapo dependencies. All these packages should be already under /home/dulak-server/rpm. Remember to test the dacapo and gpaw installations on the "Golden Client" too.

If you are installing workstation only, your setup is ready for testing - go to benchmarking and maintenance.

If you are building a cluster go back to installing and configuring systemimager,

Niflheim: Cluster_software_-_RPMS (last edited 2015-07-02 11:28:20 by OleHolmNielsen)

-  ⇤ ← Revision 1 as of 2009-06-06 12:05:58 → 
  Size: 11167
  Editor: MarcinDulak
  Comment:
+   ← Revision 23 as of 2009-06-23 06:22:48 → ⇥
  Size: 37744
  Editor: MarcinDulak
  Comment:
-Deletions are marked like this.
+Additions are marked like this.
 Line 1:
-This page describes the necessary steps for installing
`gpaw <https://wiki.fysik.dtu.dk/gpaw>`_ and `dacapo <https://wiki.fysik.dtu.dk/dacapo>`_ programs.

On the server
-------------
+This page describes the necessary steps for installing the login nodes **fjorm** or **thul**.
-Line 30:
+Line 26:
-- install::
+- install, as root::
-Line 33:
+Line 29:
-- if not yet done, go to `configuring rpmbuild <Building_a_Cluster_-_Tutorial/configuring_rpmbuild>`_,

- **Skip this step if not installing on "dulak-server"**: create the ``/home/dulak-server/rpm/external`` directory (to keep external RPMS)::

   mkdir /home/dulak-server/rpm/external; cd /home/dulak-server/rpm/external

- install official packages::
+   # /var directories must be created
   yum search --enablerepo=atrpms arpack-devel
   yum search --enablerepo=epel jmol

- configure rpmbuild (as rpmbuild):

  - enable modules - add to `~/.bashrc`::

     if [ -r "/home/camp/modulefiles.sh" ]; then
         source /home/camp/modulefiles.sh
     fi

  - set the `FYS_PLATFORM` variable::

     export FYS_PLATFORM=Intel-Nehalem-el5 # thul
     export FYS_PLATFORM=AMD-Opteron-el5 # fjorm

    **Note** that this variable will be set automatically by `/home/camp/modulefiles.sh` after
    the `environment-modules` package is installed.

  - use the following ~rpmbuild/.rpmmacros::

     %disttag        el5.fys

     %packager       rpmbuild@fysik.dtu.dk
     %distribution   Fysik RPMS
     %vendor         Fysik RPMS <rpm@fysik.dtu.dk>

     %_signature     gpg
     %_gpg_path      ~/.gnupg
     %_gpg_name      Fysik RPMS

     %_topdir        /home/camp/rpmbuild/%(echo $FYS_PLATFORM)
     %_rpmdir        %{_topdir}/RPMS
     %_srcrpmdir     %{_topdir}/SRPMS
     %_svndir        /home/camp/rpmbuild/rpmbuild
     %_specdir       %{_svndir}/SPECS
     %_sourcedir     %{_svndir}/SOURCES
     %_rpmfilename   %%{NAME}-%%{VERSION}-%%{RELEASE}.%%{ARCH}.rpm
     %_builddir      /tmp
     %_tmppath      %{_topdir}/BUILD

     # no debuginfo
     %debug_package %{nil}

     # don't strip (this does not fully work)
     # https://bugzilla.redhat.com/show_bug.cgi?id=219731
     %define __strip /bin/true

     %niflheim       1

  - as rpmbuild create directories::

     mkdir -p ~/${FYS_PLATFORM}/RPMS
     mkdir -p ~/${FYS_PLATFORM}/SRPMS
     mkdir -p ~/${FYS_PLATFORM}/BUILD
     mkdir -p ~/${FYS_PLATFORM}/SPECS # needed only by openmpi
     mkdir -p ~/${FYS_PLATFORM}/SOURCES # needed only by openmpi

- install official packages, as rpmbuild::

   cd ~/${FYS_PLATFORM}/RPMS
   mkdir external; cd external
-Line 45:
+Line 95:
-   yum localinstall *

- install `atrpms` packages (``vtk-python`` is currently unavailable 16 Apr 2009)::

   yumdownloader --resolve --enablerepo=atrpms vtk-python fftw2 fftw2-devel netcdf netcdf-devel arpack-devel graphviz
+   yumdownloader --resolve pygtk2-devel gtk2-devel tk-devel agg ghostscript libtiff-devel
   yum localinstall * # as root

- install `atrpms` packages, as rpmbuild (``vtk-python`` is currently unavailable 16 Apr 2009)::

   cd ~/${FYS_PLATFORM}/RPMS/external
   yumdownloader --resolve --enablerepo=atrpms vtk-python arpack-devel graphviz
-Line 51:
+Line 103:
-   rpm --import RPM-GPG-KEY.atrpms
   yum localinstall *

- install the packages from `epel`::

   yumdownloader --resolve --enablerepo=epel fftw3 fftw3-devel python-matplotlib python-docutils jmol
+   rpm --import RPM-GPG-KEY.atrpms # as root
   yum localinstall * # as root

- install the packages from `epel`, as rpmbuild::

   ~/${FYS_PLATFORM}/RPMS/external
   yumdownloader --resolve --enablerepo=epel jmol
-Line 59:
+Line 112:
-   rpm --import RPM-GPG-KEY-EPEL
   yum localinstall *
   source /etc/profile.d/modules.sh
+   rpm --import RPM-GPG-KEY-EPEL # as root
   yum localinstall * # as root

- remove default openmpi::

   yum remove openmpi openmpi-libs
-Line 66:
+Line 122:
+- logout and login (as rpmbuild) again to activate modules settings from `~/.bashrc`.
-Line 70:
+Line 128:
-As root::
+As rpmbuild::
-Line 74:
+Line 132:
-**Skip this step if not installing on "dulak-server"**: create the ``/home/dulak-server/rpm/campos`` directory (to keep custom built RPMS)::

  mkdir /home/dulak-server/rpm/campos

Preferably build a custom openmpi, using the latest gcc/gfortran and torque support::

  wget http://www.open-mpi.org/software/ompi/v1.3/downloads/openmpi-1.3.1.tar.bz2 \
       -O ~/rpmbuild/SOURCES/openmpi-1.3.1.tar.bz2
  export rpmtopdir=${HOME} # set this to _topdir value from ~/.rpmmacros
  sh ./buildrpm-1.3.1-1.gfortran.sh ../SOURCES/openmpi-1.3.1.tar.bz2 2>&1 | tee buildrpm-1.3.1-1.gfortran.sh.log
  rpm -ivh ~/RPMS/*/openmpi-*.rpm
  cp ~/RPMS/*/openmpi-*.rpm /home/dulak-server/rpm/campos
+Install icc/ifort compilers, enable them (only on **thul**)::

 . /opt/intel/Compiler/11.0/083/bin/intel64/ifortvars_intel64.sh
 . /opt/intel/Compiler/11.0/083/bin/intel64/iccvars_intel64.sh

Build intel compatibility packages (only on **thul**)::

  rpmbuild -bb --with modules --with default_version intel-redist.spec

**Note**: do not install the resulting RPMS on the login node. They need to be installed **only** on compute nodes.
On the login node only the module file needs to be deployed (as root)::

  mkdir -p /etc/modulefiles/intel
  cp /tmp/intel-11.0.083/11.0.083-1.intel64.el5.fys /etc/modulefiles/intel

**Note**: the above module file should contain at least::

  prepend-path    LD_LIBRARY_PATH      /opt/intel/Compiler/11.0/083/lib/intel64
  prepend-path    PATH                 /opt/intel/Compiler/11.0/083/bin/intel64
  prepend-path    MANPATH              /opt/intel/Compiler/11.0/083/man

Install mkl, and build mkl compatibility package (only on **thul**)::

  cd ~/rpmbuild/SOURCES
  mkdir 10.1.3.027
  cp -r /opt/intel/mkl/10.1.3.027/lib 10.1.3.027
  cp -r /opt/intel/mkl/10.1.3.027/doc 10.1.3.027
  tar zcf intel-redist-mkl-10.1.3.027.tar.gz 10.1.3.027

  cd ~/rpmbuild/SPECS
  rpmbuild -bb --with modules --with default_version intel-redist-mkl.spec

**Note**: do not install the resulting RPM on the login node. It needs to be installed **only** on compute nodes.
On the login node only the module file needs to be deployed (as root)::

  mkdir -p /etc/modulefiles/mkl
  cp /tmp/intel-mkl-10.1.3.027/10.1.3.027-1.el5.fys.em64t /etc/modulefiles/mkl

**Note**: the above module file should contain at least::

  prepend-path    LD_LIBRARY_PATH      /opt/intel/mkl/10.1.3.027/lib/em64t     

Build a custom openmpi, using torque support::

  export rpmtopdir=${HOME}/${FYS_PLATFORM} # set this to _topdir value from ~/.rpmmacros
  wget http://www.open-mpi.org/software/ompi/v1.3/downloads/openmpi-1.3.2.tar.bz2 \
       -O ~/rpmbuild/SOURCES/openmpi-1.3.2.tar.bz2
  sh ./buildrpm-1.3.2-1.gfortran.sh ../SOURCES/openmpi-1.3.2.tar.bz2 2>&1 | tee buildrpm-1.3.2-1.gfortran.sh.log.${FYS_PLATFORM}
  sh ./buildrpm-1.3.2-1.gfortran43.sh ../SOURCES/openmpi-1.3.2.tar.bz2 2>&1 | tee buildrpm-1.3.2-1.gfortran43.sh.log.${FYS_PLATFORM}
  sh ./buildrpm-1.3.2-1.pathscale.sh ../SOURCES/openmpi-1.3.2.tar.bz2 2>&1 | tee buildrpm-1.3.2-1.pathscale.sh.log.${FYS_PLATFORM}
  sh ./buildrpm-1.3.2-1.ifort.sh ../SOURCES/openmpi-1.3.2.tar.bz2 2>&1 | tee buildrpm-1.3.2-1.ifort.sh.log.${FYS_PLATFORM} # thul only

**Note**: intel openmpi needs to be installed ignoring dependencies::

  rpm -ivh --nodeps --oldpackage ~rpmbuild/${FYS_PLATFORM}/RPMS/openmpi-1.3.2-1.ifort.el5.fys.x86_64.rpm
-Line 90:
+Line 191:
-    grep -v "#\!" install.sh >> ~/global_install.sh
    cat uninstall.sh ~/global_uninstall.sh | grep -v "#\!" >> ~/global_uninstall.sh.tmp && mv -f ~/global_uninstall.sh.tmp ~/global_uninstall.sh
    # ignore "cat: /root/global_uninstall.sh: No such ..." error when running first time
+    grep -v "#\!" install.sh >> ~/${FYS_PLATFORM}/global_install.sh
    cat uninstall.sh ~/${FYS_PLATFORM}/global_uninstall.sh | grep -v "#\!" >> ~/${FYS_PLATFORM}/global_uninstall.sh.tmp && mv -f ~/${FYS_PLATFORM}/global_uninstall.sh.tmp ~/${FYS_PLATFORM}/global_uninstall.sh
    # ignore "cat: .../global_uninstall.sh: No such ..." error when running first time
-Line 97:
+Line 198:
+- set the disttag variable for convenience::

   export disttag="el5.fys"

- acml::

    rpmbuild -bb --with compiler=gfortran --with version1=0 --with version2=1 --with modules --with default_version acml.spec
    rpmbuild -bb --with compiler=pathscale --with version1=0 --with version2=1 --with modules --with default_version acml.spec

    rpmbuild -bb --with compiler=gfortran43 --with version1=1 --with version2=0 --with modules --with default_version acml.spec # fjorm only
    rpmbuild -bb --with compiler=pathscale --with version1=1 --with version2=0 --with modules --with default_version acml.spec

    rpmbuild -bb --with compiler=gfortran43 --with version1=2 --with version2=0 --with modules --with default_version acml.spec
    rpmbuild -bb --with compiler=pathscale --with version1=2 --with version2=0 --with modules --with default_version acml.spec
    rpmbuild -bb --with compiler=ifort --with version1=2 --with version2=0 --with modules --with default_version acml.spec # thul only

 - goto (only on **fjorm**)::

    rpmbuild --bb --with compiler=gfortran --with modules=1 --with default_version=1 \
                  --with prefix=/opt/goto/1.26/1.${disttag}.gfortran.smp goto.spec
    rpmbuild --bb --with compiler=gfortran43 --with modules=1 --with default_version=1 \
                  --with prefix=/opt/goto/1.26/1.${disttag}.gfortran43.smp goto.spec
    rpmbuild --bb --with compiler=pathscale --with compiler_bindir=/opt/pathscale/bin --with compiler_libdir=/opt/pathscale/lib/3.2 \
                  --with modules=1 --with default_version=1 --with prefix=/opt/goto/1.26/1.${disttag}.pathscale.smp goto.spec

   **Note** - **1.26** version fails on Nehalem with::

     ../../../param.h:1195:21: error: division by zero in #if

- atlas::

    rpmbuild --bb  --with compiler=gfortran --with compiler_bindir=/usr/bin --with compiler_libdir=/usr/lib64 \
             -with modules=1 --with default_version=1 --with prefix=/opt/atlas/3.8.3/1.gfortran.${disttag} atlas.spec

    rpmbuild --bb  --with compiler=gfortran43 --with compiler_bindir=/usr/bin --with compiler_libdir=/usr/lib64 \
             -with modules=1 --with default_version=1 --with prefix=/opt/atlas/3.8.3/1.gfortran43.${disttag} atlas.spec

    rpmbuild --bb  --with compiler=pathscale --with compiler_bindir=/opt/pathscale/bin --with compiler_libdir=/opt/pathscale/lib/3.2 \
             -with modules=1 --with default_version=1 --with prefix=/opt/atlas/3.8.3/1.pathscale.${disttag} atlas.spec
-Line 100:
+Line 241:
-    python campos_installer.py --machine='dulak-cluster' --create_scripts campos-dacapo-pseudopotentials
+    rpmbuild -bb --with modules --with default_version campos-dacapo-pseudopotentials.spec
-Line 104:
+Line 245:
-    python campos_installer.py --machine='dulak-cluster' --create_scripts RasMol
    cp ~/RPMS/*/RasMol-*.rpm /home/dulak-server/rpm/campos
+    rpmbuild -bb --with modules --with default_version --with prefix=/opt/RasMol/2.7.3/3.${disttag} RasMol.spec

- `cblas <https://wiki.fysik.dtu.dk/niflheim/Cluster_software_-_RPMS#cblas>`_::

    rpmbuild --bb  --with blas_version=3.0.37.el5 --with modules=1 --with default_version=1 \
                   --with prefix=/opt/cblas/2.23.3/1.${disttag}.gfortran.blas.3.0.37.el5 cblas.spec

    rpmbuild --bb  --with blas=acml --with blas_version=4.0.1 --with blasdir=/opt/acml/4.0.1/gfortran64/lib \
                   --with modules=1 --with default_version=1 --with prefix=/opt/cblas/2.23.3/1.${dist}.gfortran.acml.4.0.1 cblas.spec

- python-setuptools::

       rpmbuild -bb --with modules --with default_version --with prefix=/opt/python-setuptools/0.6c9/1.${disttag}.python2.4 python-setuptools.spec

- python-nose::

       rpmbuild -bb --with modules --with default_version --with prefix=/opt/python-nose/0.10.4/1.${disttag}.python2.4 python-nose.spec
       module load python-setuptools
       module load python-nose

- numpy::

    rpmbuild --bb --with cblas_prefix=none \ # dotblas fails with acml
                  --with blas=acml --with blas_version=4.0.1 --with blasdir=/opt/acml/4.0.1/gfortran64/lib \
                  --with lapack=acml --with lapackdir=/opt/acml/4.0.1/gfortran64/lib \
                  --with compiler=gfortran --with modules=1 --with default_version=1 \
                  --with prefix=/opt/numpy/1.3.0/1.${disttag}.gfortran.python2.4.acml.4.0.1.acml numpy.spec

    rpmbuild --bb --with cblas_prefix=/opt/cblas/2.23.3/1.${disttag}.gfortran.blas.3.0.37.el5 --with blas_version=3.0.37.el5 \
                  --with compiler=gfortran --with modules=1 --with default_version=1 \
                  --with prefix=/opt/numpy/1.3.0/1.${disttag}.gfortran.python2.4.blas.3.0.37.el5.lapack numpy.spec

  Test with::

    module load numpy/1.3.0-1.el5.fys.gfortran.python2.4.blas.3.0.37.el5.lapack
    python -c "import numpy; from numpy.core.multiarray import dot; b = numpy.ones(13, numpy.complex); dot(b, b)"
    python -c "import numpy; numpy.test()"
    module unload numpy/1.3.0-1.el5.fys.gfortran.python2.4.blas.3.0.37.el5.lapack
     
    module load acml-gfortran64/4.0.1-1.el5.fys
    module load numpy/1.3.0-1.el5.fys.gfortran.python2.4.acml.4.0.1.acml
    python -c "import numpy; from numpy.core.multiarray import dot; b = numpy.ones(13, numpy.complex); dot(b, b)"
    python -c "import numpy; numpy.test()"
    module unload numpy/1.3.0-1.el5.fys.gfortran.python2.4.acml.4.0.1.acml
    module unload acml-gfortran64/4.0.1-1.el5.fys

  Load the default numpy::

    module load numpy/1.3.0-1.el5.fys.gfortran.python2.4.acml.4.0.1.acml
-Line 109:
+Line 297:
-    python campos_installer.py --machine='dulak-cluster' --create_scripts gnuplot-py
    cp ~/RPMS/*/gnuplot-py-*.rpm /home/dulak-server/rpm/campos

  if you use modules::
+    rpmbuild -bb --with modules --with default_version --with prefix=/opt/gnuplot-py/1.8.1/1.${disttag}.python2.4 gnuplot-py.spec
-Line 115:
+Line 300:
-    echo "module load gnuplot-py" >> ~/global_install.sh

  otherwise logout and login again!

- `cblas <https://wiki.fysik.dtu.dk/niflheim/Cluster_software_-_RPMS#cblas>`_::

    python campos_installer.py --machine='dulak-cluster' --create_scripts cblas
    cp ~/RPMS/*/cblas-*.rpm /home/dulak-server/rpm/campos
-Line 126:
+Line 303:
-    cd
    rpm -e --nodeps python-numeric
+    cd ~/${FYS_PLATFORM}/RPMS/external
    rpm -e --nodeps python-numeric # as root
-Line 129:
+Line 306:
-    cp python-numeric-*.rpm /home/dulak-server/rpm/external # **Skip this step if not installing on "dulak-server"**
-Line 131:
+Line 307:
-    python campos_installer.py --machine='dulak-cluster' --create_scripts python-numeric
    cp ~/RPMS/*/python-numeric-*.rpm /home/dulak-server/rpm/campos
+    rpmbuild --bb --with cblas_prefix=/opt/cblas/2.23.3/1.${disttag}.gfortran.acml.4.0.1 \
                  --with blas=acml --with blas_version=4.0.1 --with blasdir=/opt/acml/4.0.1/gfortran64/lib \
                  --with lapack=acml --with lapackdir=/opt/acml/4.0.1/gfortran64/lib \
                  --with compiler=gfortran --with modules=1 --with default_version=1 \
                  --with prefix=/opt/python-numeric/24.2/4.${disttag}.gfortran.python2.4.acml.4.0.1.acml python-numeric.spec

    rpmbuild --bb --with cblas_prefix=/opt/cblas/2.23.3/1.${disttag}.gfortran.blas.3.0.37.el5 --with blas_version=3.0.37.el5 \
                  --with compiler=gfortran --with modules=1 --with default_version=1 \
                  --with prefix=/opt/python-numeric/24.2/4.${disttag}.gfortran.python2.4.blas.3.0.37.el5.lapack python-numeric.spec
-Line 138:
+Line 323:
-  If you use modules::

    module load python-numeric
    echo "module load python-numeric" >> ~/global_install.sh

  otherwise logout and login again!
-Line 147:
+Line 325:
+    module load python-numeric/24.2-4.el5.fys.gfortran.python2.4.blas.3.0.37.el5.lapack
-Line 148:
+Line 327:
-    ldd `rpm -ql python-numeric | grep lapack_lite.so`
    ldd `rpm -ql python-numeric | grep _dotblas.so`
+    ldd `rpm -ql python-numeric-24.2-4.el5.fys.gfortran.python2.4.blas.3.0.37.el5.lapack | grep lapack_lite.so`
    ldd `rpm -ql python-numeric-24.2-4.el5.fys.gfortran.python2.4.blas.3.0.37.el5.lapack | grep _dotblas.so`
    module unload python-numeric/24.2-4.el5.fys.gfortran.python2.4.blas.3.0.37.el5.lapack

    module load acml-gfortran64/4.0.1-1.el5.fys
    module load python-numeric/24.2-4.el5.fys.gfortran.python2.4.acml.4.0.1.acml
    python -c "import lapack_lite"
    ldd `rpm -ql python-numeric-24.2-4.el5.fys.gfortran.python2.4.acml.4.0.1.acml | grep lapack_lite.so`
    ldd `rpm -ql python-numeric-24.2-4.el5.fys.gfortran.python2.4.acml.4.0.1.acml | grep _dotblas.so`
    module unload python-numeric/24.2-4.el5.fys.gfortran.python2.4.acml.4.0.1.acml
    module unload acml-gfortran64/4.0.1-1.el5.fys
-Line 153:
+Line 341:
-    rpm -ivh --oldpackage ~/python-numeric-*.rpm
+    rpm -ivh --oldpackage ~rpmbuild/${FYS_PLATFORM}/RPMS/external/python-numeric-23.*.rpm

  load the default `Numeric` version::

    module load acml-gfortran64/4.0.1-1.el5.fys
    module load python-numeric/24.2-4.el5.fys.gfortran.python2.4.acml.4.0.1.acml

- netcdf::

     rpmbuild --bb  --with compiler=gfortran --with modules=1 --with default_version=1 \
                    --with prefix=/opt/netcdf4/4.0.1/1.gfortran.${disttag} netcdf4.spec
     rpmbuild --bb  --with compiler=gfortran43 --with modules=1 --with default_version=1 \
                    --with prefix=/opt/netcdf4/4.0.1/1.gfortran43.${disttag} netcdf4.spec
     rpmbuild --bb  --with compiler=pathscale --with compiler_bindir=/opt/pathscale/bin --with compiler_libdir=/opt/pathscale/lib/3.2 \
                    --with modules=1 --with default_version=1 --with prefix=/opt/netcdf4/4.0.1/1.pathscale.${disttag} netcdf4.spec
-Line 157:
+Line 359:
-    python campos_installer.py --machine='dulak-cluster' --create_scripts ScientificPython
    cp ~/RPMS/*/ScientificPython-*.rpm /home/dulak-server/rpm/campos
+    rpmbuild --bb  --with Numeric_includedir=none --with numpy=numpy --with compiler=gfortran --with netcdf=netcdf4 \
                   --with netcdf_includedir=/opt/netcdf4/4.0.1/1.gfortran.${disttag}/include \
                   --with netcdf_libdir=/opt/netcdf4/4.0.1/1.gfortran.${disttag}/lib64 \
                   --with modules=1 --with default_version=1 \
                   --with prefix=/opt/ScientificPython/2.8/1.${disttag}.python2.4.serial_version.numpy ScientificPython.spec

    rpmbuild --bb  --with numpy=numeric --with compiler=gfortran --with netcdf=netcdf4 \
                   --with netcdf_includedir=/opt/netcdf4/4.0.1/1.gfortran.${disttag}/include \
                   --with netcdf_libdir=/opt/netcdf4/4.0.1/1.gfortran.${disttag}/lib64 \
                   --with modules=1 --with default_version=1 --with parallel=1 --with openmpi=openmpi \
                   --with ompi_bindir=/opt/openmpi/1.3.2-1.gfortran/bin \
                   --with ompi_includedir=/opt/openmpi/1.3.2-1.gfortran/include --with ompi_libdir=/opt/openmpi/1.3.2-1.gfortran/lib64 \
                   --with prefix=/opt/ScientificPython/2.6.2/1.${disttag}.python2.4.openmpi.numeric ScientificPython.spec

- python-docutils::

    rpmbuild -bb --with modules --with default_version --with prefix=/opt/python-docutils/0.5/1.${disttag}.python2.4 python-docutils.spec

- pytz::

    rpmbuild -bb --with modules --with default_version --with prefix=/opt/pytz/2008g/1.${disttag}.python2.4 pytz.spec

- python-docutils::

    rpmbuild -bb --with modules --with default_version --with prefix=/opt/python-dateutil/1.4.1/3.${disttag}.python2.4 python-dateutil.spec

- python-matplotlib::

    module load pytz
    module load python-docutils
    module load python-dateutil
    rpmbuild -bb --with modules --with default_version --with prefix=/opt/python-matplotlib/0.98.5.2/1.${disttag}.python2.4 python-matplotlib.spec
-Line 162:
+Line 394:
-    python campos_installer.py --machine='dulak-cluster' --create_scripts campos-ase2
    cp ~/RPMS/*/campos-ase2-*.rpm /home/dulak-server/rpm/campos
+    rpmbuild -bb --with modules --with default_version --with prefix=/opt/campos-ase2/2.3.13/1.${disttag}.python2.4 campos-ase2.spec
-Line 167:
+Line 398:
-    python campos_installer.py --machine='dulak-cluster' --create_scripts campos-dacapo-python
+    rpmbuild -bb --with modules --with default_version --with prefix=/opt/campos-dacapo-python/0.9.4/1.${disttag}.python2.4 campos-dacapo-python.spec

- fftw2::

    rpmbuild --bb  --with compiler=gfortran --with default_version=1 --with modules=1 --with prefix=/opt/fftw2/2.1.5/12.gfortran.${disttag} fftw2.spec
    rpmbuild --bb  --with compiler=gfortran43 --with default_version=1 --with modules=1 --with prefix=/opt/fftw2/2.1.5/12.gfortran43.${disttag} fftw2.spec
    rpmbuild --bb  --with compiler=pathscale --with compiler_bindir=/opt/pathscale/bin --with compiler_libdir=/opt/pathscale/lib/3.2 \
                   --with default_version=1 --with modules=1 --with prefix=/opt/fftw2/2.1.5/12.pathscale.${disttag} fftw2.spec
-Line 171:
+Line 409:
-    python campos_installer.py --machine='dulak-cluster' --create_scripts --compiler=gfortran43 campos-dacapo
    cp ~/RPMS/*/campos-dacapo-*.rpm /home/dulak-server/rpm/campos

  logout and login again!

build following for gpaw:
+     rpmbuild --bb  --with blas=acml --with blas_version=4.2.0 --with blasdir=/opt/acml/4.2.0/gfortran4364/lib \
                    --with compiler=gfortran43 --with compiler_bindir=/usr/bin --with compiler_libdir=/usr/lib64 \
                    --with fftw=fftw2 --with fftw_libdir=/opt/fftw2/2.1.5/12.gfortran43.${disttag}/lib64 \
                    --with lapack=acml --with lapackdir=/opt/acml/4.2.0/gfortran4364/lib --with netcdf=netcdf4 \
                    --with netcdf_includedir=/opt/netcdf4/4.0.1/1.gfortran43.${disttag}/include \
                    --with netcdf_libdir=/opt/netcdf4/4.0.1/1.gfortran43.${disttag}/lib64 \
                    --with modules=1 --with default_version=1 --with parallel=1 --with openmpi=openmpi \
                    --with ompi_bindir=/opt/openmpi/1.3.2-1.gfortran43/bin \
                    --with ompi_includedir=/opt/openmpi/1.3.2-1.gfortran43/include --with ompi_libdir=/opt/openmpi/1.3.2-1.gfortran43/lib64 \
                    --with prefix=/opt/campos-dacapo/2.7.16/1.${disttag}.gfortran43.openmpi.acml.4.2.0.acml campos-dacapo.spec

     rpmbuild --bb  --with blas=acml --with blas_version=4.0.1 --with blasdir=/opt/acml/4.0.1/gfortran64/lib \
                    --with compiler=gfortran --with compiler_bindir=/usr/bin --with compiler_libdir=/usr/lib64 \
                    --with fftw=fftw2 --with fftw_libdir=/opt/fftw2/2.1.5/12.gfortran.${disttag}/lib64 \
                    --with lapack=acml --with lapackdir=/opt/acml/4.0.1/gfortran64/lib --with netcdf=netcdf4 \
                    --with netcdf_includedir=/opt/netcdf4/4.0.1/1.gfortran.${disttag}/include \
                    --with netcdf_libdir=/opt/netcdf4/4.0.1/1.gfortran.${disttag}/lib64 \
                    --with modules=1 --with default_version=1 --with parallel=1 --with openmpi=openmpi \
                    --with ompi_bindir=/opt/openmpi/1.3.2-1.gfortran/bin \
                    --with ompi_includedir=/opt/openmpi/1.3.2-1.gfortran/include --with ompi_libdir=/opt/openmpi/1.3.2-1.gfortran/lib64 \
                    --with prefix=/opt/campos-dacapo/2.7.16/1.${disttag}.gfortran.openmpi.acml.4.0.1.acml campos-dacapo.spec

     rpmbuild --bb  --with blas=blas --with blas_version=3.0.37.el5 --with blasdir=/usr/lib64 \
                    --with compiler=gfortran43 --with compiler_bindir=/usr/bin --with compiler_libdir=/usr/lib64 \
                    --with fftw=fftw2 --with fftw_libdir=/opt/fftw2/2.1.5/12.gfortran43.${disttag}/lib64 \
                    --with lapack=lapack --with lapackdir=/usr/lib64 --with netcdf=netcdf4 \
                    --with netcdf_includedir=/opt/netcdf4/4.0.1/1.gfortran43.${disttag}/include \
                    --with netcdf_libdir=/opt/netcdf4/4.0.1/1.gfortran43.${disttag}/lib64 \
                    --with modules=1 --with default_version=1 --with parallel=1 --with openmpi=openmpi \
                    --with ompi_bindir=/opt/openmpi/1.3.2-1.gfortran43/bin \
                    --with ompi_includedir=/opt/openmpi/1.3.2-1.gfortran43/include --with ompi_libdir=/opt/openmpi/1.3.2-1.gfortran43/lib64 \
                    --with prefix=/opt/campos-dacapo/2.7.16/1.${disttag}.gfortran43.openmpi.blas.3.0.37.el5.lapack campos-dacapo.spec

     rpmbuild --bb  --with blas=blas --with blas_version=3.0.37.el5 --with blasdir=/usr/lib64 \
                    --with compiler=gfortran --with compiler_bindir=/usr/bin --with compiler_libdir=/usr/lib64 \
                    --with fftw=fftw2 --with fftw_libdir=/opt/fftw2/2.1.5/12.gfortran.${disttag}/lib64 \
                    --with lapack=lapack --with lapackdir=/usr/lib64 --with netcdf=netcdf4 \
                    --with netcdf_includedir=/opt/netcdf4/4.0.1/1.gfortran.${disttag}/include \
                    --with netcdf_libdir=/opt/netcdf4/4.0.1/1.gfortran.${disttag}/lib64 \
                    --with modules=1 --with default_version=1 --with parallel=1 --with openmpi=openmpi \
                    --with ompi_bindir=/opt/openmpi/1.3.2-1.gfortran/bin \
                    --with ompi_includedir=/opt/openmpi/1.3.2-1.gfortran/include --with ompi_libdir=/opt/openmpi/1.3.2-1.gfortran/lib64 \
                    --with prefix=/opt/campos-dacapo/2.7.16/1.${disttag}.gfortran.openmpi.blas.3.0.37.el5.lapack campos-dacapo.spec

Build following for gpaw:

- fftw3::

    rpmbuild --bb --with major_version=3 --with version1=2 --with version2=1 --with compiler=gfortran \
                  --with default_version=1 --with modules=1 --with prefix=/opt/fftw3/3.2.1/12.gfortran.${disttag} fftw2.spec
    rpmbuild --bb --with major_version=3 --with version1=2 --with version2=1 --with compiler=gfortran43 \
                  --with default_version=1 --with modules=1 --with prefix=/opt/fftw3/3.2.1/12.gfortran43.${disttag} fftw2.spec

- scipy::

    module unload numpy
    module load numpy/1.3.0-1.el5.fys.gfortran.python2.4.blas.3.0.37.el5.lapack

    rpmbuild --bb --with cblas_prefix=/opt/cblas/2.23.3/1.${disttag}.gfortran.blas.3.0.37.el5 \
                  --with blas=blas --with blas_version=3.0.37.el5 --with blasdir=/usrlib64 \
                  --with lapack=lapack --with lapackdir=/usr/lib64 \
                  --with ufsparse_libdir=/usr/lib64 --with ufsparse_includedir=/usr/include/suitesparse \
                  --with fftw_libdir=/opt/fftw3/3.2.1/12.gfortran.${disttag}/lib64 \
                  --with compiler=gfortran --with default_version=1 --with modules=1 \
                  --with prefix=/opt/scipy/0.7.0/1.${disttag}.gfortran.python2.4.blas.3.0.37.el5.lapack scipy.spec

    module unload numpy
    module load numpy/1.3.0-1.el5.fys.gfortran.python2.4.acml.4.0.1.acml

    rpmbuild --bb --with cblas_prefix=/opt/cblas/2.23.3/1.${disttag}.gfortran.acml.4.0.1 \
                  --with blas=acml --with blas_version=4.0.1 --with blasdir=/opt/acml/4.0.1/gfortran64/lib \
                  --with lapack=acml --with lapackdir=/opt/acml/4.0.1/gfortran64/lib \
                  --with ufsparse_libdir=/usr/lib64 --with ufsparse_includedir=/usr/include/suitesparse \
                  --with fftw_libdir=/opt/fftw3/3.2.1/12.gfortran.${disttag}/lib64 \
                  --with compiler=gfortran --with default_version=1 --with modules=1 \
                  --with prefix=/opt/scipy/0.7.0/1.${disttag}.gfortran.python2.4.acml.4.0.1.acml scipy.spec
-Line 180:
+Line 488:
-    python campos_installer.py --machine='dulak-cluster' --create_scripts campos-gpaw-setups
+    rpmbuild -bb --with default_version --with modules campos-gpaw-setups.spec
-Line 184:
+Line 492:
-    python campos_installer.py --machine='dulak-cluster' --create_scripts campos-ase3
    cp ~/RPMS/*/campos-ase3-*.rpm /home/dulak-server/rpm/campos
+    rpmbuild -bb --with modules --with default_version --with prefix=/opt/campos-ase3/3.1.0.846/1.${disttag}.python2.4 campos-ase3.spec

- povray::

    rpmbuild -bb --with modules --with default_version --with prefix=/opt/povray/3.6.1/3.${disttag} povray.spec

- python-jinja2::

    rpmbuild -bb --with modules --with default_version --with prefix=/opt/python-jinja2/2.1.1/1.${disttag}.python2.4 python-jinja2.spec

- python-pygments::

    rpmbuild -bb --with modules --with default_version --with prefix=/opt/python-pygments/1.0/1.${disttag}.python2.4 python-pygments.spec

- babel::

    rpmbuild -bb --with modules --with default_version --with prefix=/opt/babel/0.9.4/1.${disttag}.python2.4 babel.spec

- python-sphinx::

    module load python-jinja2
    rpmbuild -bb --with modules --with default_version --with prefix=/opt/python-sphinx/0.6.1/1.${disttag}.python2.4 python-sphinx.spec

- blacs::

     rpmbuild --bb --with compiler=gfortran43 --with compiler_bindir=/usr/bin --with compiler_libdir=/usr/lib64 \
                    --with modules=1 --with default_version=1 --with parallel=1 --with openmpi=openmpi \
                    --with ompi_bindir=/opt/openmpi/1.3.2-1.gfortran43/bin \
                    --with ompi_includedir=/opt/openmpi/1.3.2-1.gfortran43/include --with ompi_libdir=/opt/openmpi/1.3.2-1.gfortran43/lib64 \
                    --with prefix=/opt/blacs/1.1/24.${disttag}.gfortran43.openmpi blacs.spec

     rpmbuild --bb  --with compiler=gfortran --with compiler_bindir=/usr/bin --with compiler_libdir=/usr/lib64 \
                    --with modules=1 --with default_version=1 --with parallel=1 --with openmpi=openmpi \
                    --with ompi_bindir=/opt/openmpi/1.3.2-1.gfortran/bin \
                    --with ompi_includedir=/opt/openmpi/1.3.2-1.gfortran/include --with ompi_libdir=/opt/openmpi/1.3.2-1.gfortran/lib64 \
                    --with prefix=/opt/blacs/1.1/24.${disttag}.gfortran.openmpi blacs.spec

- scalapack::

     rpmbuild --bb  --with blacs=blacs --with blacsdir=/opt/blacs/1.1/24.${disttag}.gfortran43.openmpi/lib64 \
                    --with blas=acml --with blas_version=4.2.0 --with blasdir=/opt/acml/4.2.0/gfortran4364/lib \
                    --with compiler=gfortran43 --with compiler_bindir=/usr/bin --with compiler_libdir=/usr/lib64 \
                    --with default_version=1 --with modules=1 --with openmpi=openmpi \
                    --with lapack=acml --with lapackdir=/opt/acml/4.2.0/gfortran4364/lib \
                    --with ompi_bindir=/opt/openmpi/1.3.2-1.gfortran43/bin \
                    --with ompi_includedir=/opt/openmpi/1.3.2-1.gfortran43/include --with ompi_libdir=/opt/openmpi/1.3.2-1.gfortran43/lib64 \
                    --with prefix=/opt/scalapack/1.8.0/1.${disttag}.gfortran43.openmpi.acml.4.2.0.acml scalapack.spec

     rpmbuild --bb  --with blacs=blacs --with blacsdir=/opt/blacs/1.1/24.${disttag}.gfortran.openmpi/lib64 \
                    --with blas=acml --with blas_version=4.0.1 --with blasdir=/opt/acml/4.0.1/gfortran64/lib \
                    --with compiler=gfortran --with compiler_bindir=/usr/bin --with compiler_libdir=/usr/lib64 \
                    --with default_version=1 --with modules=1 --with openmpi=openmpi \
                    --with lapack=acml --with lapackdir=/opt/acml/4.0.1/gfortran64/lib \
                    --with ompi_bindir=/opt/openmpi/1.3.2-1.gfortran/bin \
                    --with ompi_includedir=/opt/openmpi/1.3.2-1.gfortran/include --with ompi_libdir=/opt/openmpi/1.3.2-1.gfortran43/lib \
                    --with prefix=/opt/scalapack/1.8.0/1.${disttag}.gfortran.openmpi.acml.4.0.1.acml scalapack.spec

     rpmbuild --bb  --with blacs=blacs --with blacsdir=/opt/blacs/1.1/24.${disttag}.gfortran43.openmpi/lib64 \
                    --with blas=blas --with blas_version=3.0.37.el5 --with blasdir=/usr/lib64 \
                    --with compiler=gfortran43 --with compiler_bindir=/usr/bin --with compiler_libdir=/usr/lib64 \
                    --with default_version=1 --with modules=1 --with openmpi=openmpi \
                    --with lapack=lapack --with lapackdir=/usr/lib64 \
                    --with ompi_bindir=/opt/openmpi/1.3.2-1.gfortran43/bin \
                    --with ompi_includedir=/opt/openmpi/1.3.2-1.gfortran43/include --with ompi_libdir=/opt/openmpi/1.3.2-1.gfortran43/lib64 \
                    --with prefix=/opt/scalapack/1.8.0/1.${disttag}.gfortran43.openmpi.blas.3.0.37.el5.lapack scalapack.spec

     rpmbuild --bb  --with blacs=blacs --with blacsdir=/opt/blacs/1.1/24.${disttag}.gfortran.openmpi/lib64 \
                    --with blas=blas --with blas_version=3.0.37.el5 --with blasdir=/usr/lib64 \
                    --with compiler=gfortran --with compiler_bindir=/usr/bin --with compiler_libdir=/usr/lib64 \
                    --with default_version=1 --with modules=1 --with openmpi=openmpi \
                    --with lapack=lapack --with lapackdir=/usr/lib64 \
                    --with ompi_bindir=/opt/openmpi/1.3.2-1.gfortran/bin \
                    --with ompi_includedir=/opt/openmpi/1.3.2-1.gfortran/include --with ompi_libdir=/opt/openmpi/1.3.2-1.gfortran/lib64 \
                    --with prefix=/opt/scalapack/1.8.0/1.${disttag}.gfortran.openmpi.blas.3.0.37.el5.lapack scalapack.spec
-Line 189:
+Line 569:
-    python campos_installer.py --machine='dulak-cluster' --create_scripts --compiler=gfortran43 campos-gpaw
    cp ~/RPMS/*/campos-gpaw-*.rpm /home/dulak-server/rpm/campos

  logout and login again!
+     rpmbuild --bb  --with blacs=blacs --with blacsdir=/opt/blacs/1.1/24.${disttag}.gfortran.openmpi/lib64 \
                    --with blas=acml --with blas_version=4.0.1 --with blasdir=/opt/acml/4.0.1/gfortran64/lib \
                    --with cmr_repository=/home/niflheim/repository/db \
                    --with compiler=gfortran --with compiler_bindir=/usr/bin --with compiler_libdir=/usr/lib64 \
                    --with default_version=1 --with modules=1 \
                    --with lapack=acml --with lapackdir=/opt/acml/4.0.1/gfortran64/lib \
                    --with ompi_bindir=/opt/openmpi/1.3.2-1.gfortran/bin \
                    --with ompi_includedir=/opt/openmpi/1.3.2-1.gfortran/include --with ompi_libdir=/opt/openmpi/1.3.2-1.gfortran/lib64 \
                    --with openmpi=openmpi --with parallel=1 \
                    --with scalapack=scalapack --with scalapackdir=/opt/scalapack/1.8.0/1.${disttag}.gfortran.openmpi.acml.4.0.1.acml/lib64 \
                    --with prefix=/opt/campos-gpaw/0.6.3862/1.${disttag}.gfortran.python2.4.openmpi.acml.4.0.1.acml campos-gpaw.spec

     rpmbuild --bb  --with blacs=blacs --with blacsdir=/opt/blacs/1.1/24.${disttag}.gfortran43.openmpi/lib64 \
                    --with blas=acml --with blas_version=4.2.0 --with blasdir=/opt/acml/4.2.0/gfortran4364/lib \
                    --with cmr_repository=/home/niflheim/repository/db \
                    --with compiler=gfortran43 --with compiler_bindir=/usr/bin --with compiler_libdir=/usr/lib64 \
                    --with default_version=1 --with modules=1 \
                    --with lapack=acml --with lapackdir=/opt/acml/4.2.0/gfortran4364/lib \
                    --with ompi_bindir=/opt/openmpi/1.3.2-1.gfortran43/bin \
                    --with ompi_includedir=/opt/openmpi/1.3.2-1.gfortran43/include --with ompi_libdir=/opt/openmpi/1.3.2-1.gfortran43/lib64 \
                    --with openmpi=openmpi --with parallel=1 \
                    --with scalapack=scalapack --with scalapackdir=/opt/scalapack/1.8.0/1.${disttag}.gfortran43.openmpi.acml.4.2.0.acml/lib64 \
                    --with prefix=/opt/campos-gpaw/0.6.3862/1.${disttag}.gfortran43.python2.4.openmpi.acml.4.2.0.acml campos-gpaw.spec