This page describes the necessary steps for installing the login nodes **fjorm** or **thul**. Install external packages ========================= As root: - create yum repository definitions (do **not** enable them):: # atrpms echo '[atrpms]' > /etc/yum.repos.d/atrpms.repo echo 'name=name=CentOS $releasever - $basearch - ATrpms' >> /etc/yum.repos.d/atrpms.repo echo 'baseurl=http://dl.atrpms.net/el$releasever-$basearch/atrpms/stable' >> /etc/yum.repos.d/atrpms.repo echo '#baseurl=http://mirrors.ircam.fr/pub/atrpms/el$releasever-$basearch/atrpms/stable' >> /etc/yum.repos.d/atrpms.repo echo 'gpgkey=http://ATrpms.net/RPM-GPG-KEY.atrpms' >> /etc/yum.repos.d/atrpms.repo echo 'gpgcheck=1' >> /etc/yum.repos.d/atrpms.repo echo 'enabled=0' >> /etc/yum.repos.d/atrpms.repo # epel echo '[epel]' > /etc/yum.repos.d/epel.repo echo 'name=name=CentOS $releasever - $basearch - EPEL' >> /etc/yum.repos.d/epel.repo echo 'baseurl=http://download.fedora.redhat.com/pub/epel/$releasever/$basearch' >> /etc/yum.repos.d/epel.repo echo 'gpgkey=http://download.fedora.redhat.com/pub/epel/RPM-GPG-KEY-EPEL' >> /etc/yum.repos.d/epel.repo echo 'gpgcheck=1' >> /etc/yum.repos.d/epel.repo echo 'enabled=0' >> /etc/yum.repos.d/epel.repo - install, as root:: yum install yum-utils # /var directories must be created yum search --enablerepo=atrpms arpack-devel yum search --enablerepo=epel jmol - configure rpmbuild (as rpmbuild): - enable modules - add to `~/.bashrc`:: if [ -r "/home/camp/modulefiles.sh" ]; then source /home/camp/modulefiles.sh fi - set the `FYS_PLATFORM` variable:: export FYS_PLATFORM=Intel-Nehalem-el5 # thul export FYS_PLATFORM=AMD-Opteron-el5 # fjorm **Note** that this variable will be set automatically by `/home/camp/modulefiles.sh` after the `environment-modules` package is installed. - use the following ~rpmbuild/.rpmmacros:: %disttag el5.fys %packager rpmbuild@fysik.dtu.dk %distribution Fysik RPMS %vendor Fysik RPMS %_signature gpg %_gpg_path ~/.gnupg %_gpg_name Fysik RPMS %_topdir /home/camp/rpmbuild/%(echo $FYS_PLATFORM) %_rpmdir %{_topdir}/RPMS %_srcrpmdir %{_topdir}/SRPMS %_svndir /home/camp/rpmbuild/rpmbuild %_specdir %{_svndir}/SPECS %_sourcedir %{_svndir}/SOURCES %_rpmfilename %%{NAME}-%%{VERSION}-%%{RELEASE}.%%{ARCH}.rpm %_builddir /tmp %_tmppath %{_topdir}/BUILD # no debuginfo %debug_package %{nil} # don't strip %define __strip /bin/true %niflheim 1 - as rpmbuild create directories:: mkdir -p ~/${FYS_PLATFORM}/RPMS mkdir -p ~/${FYS_PLATFORM}/SRPMS mkdir -p ~/${FYS_PLATFORM}/BUILD mkdir -p ~/${FYS_PLATFORM}/SPECS # needed only by openmpi mkdir -p ~/${FYS_PLATFORM}/SOURCES # needed only by openmpi - install official packages, as rpmbuild:: cd ~/${FYS_PLATFORM}/RPMS mkdir external; cd external yumdownloader --resolve gcc-gfortran gcc43-c++ gcc43-gfortran blas-devel lapack-devel python-devel yumdownloader --resolve gnuplot libXi-devel xorg-x11-fonts-100dpi pexpect tetex-latex tkinter qt-devel yumdownloader --resolve openmpi openmpi-devel openmpi-libs compat-dapl libibverbs librdmacm openib yumdownloader --resolve pygtk2-devel gtk2-devel tk-devel agg ghostscript libtiff-devel yum localinstall * # as root - install `atrpms` packages, as rpmbuild (``vtk-python`` is currently unavailable 16 Apr 2009):: cd ~/${FYS_PLATFORM}/RPMS/external yumdownloader --resolve --enablerepo=atrpms vtk-python arpack-devel graphviz wget http://ATrpms.net/RPM-GPG-KEY.atrpms rpm --import RPM-GPG-KEY.atrpms # as root yum localinstall * # as root - install the packages from `epel`, as rpmbuild:: ~/${FYS_PLATFORM}/RPMS/external yumdownloader --resolve --enablerepo=epel jmol yumdownloader --resolve --enablerepo=epel environment-modules suitesparse-devel wget http://download.fedora.redhat.com/pub/epel/RPM-GPG-KEY-EPEL rpm --import RPM-GPG-KEY-EPEL # as root yum localinstall * # as root - remove default openmpi:: yum remove openmpi openmpi-libs - edit ``/etc/yum.conf`` so it contains:: exclude=netcdf-* netcdf3-* fftw-* fftw2-* fftw3-* python-numeric openmpi-* - logout and login (as rpmbuild) again to activate modules settings from `~/.bashrc`. It's time to build custom RPMS ============================== As rpmbuild:: cd ~/rpmbuild/SPECS Install icc/ifort compilers, enable them (only on **thul**):: . /opt/intel/Compiler/11.0/083/bin/intel64/ifortvars_intel64.sh . /opt/intel/Compiler/11.0/083/bin/intel64/iccvars_intel64.sh Build intel compatibility packages (only on **thul**):: rpmbuild -bb --with modules --with default_version intel-redist.spec **Note**: do not install the resulting RPMS on the login node. They need to be installed **only** on compute nodes. On the login node only the module file needs to be deployed (as root):: mkdir -p /etc/modulefiles/intel cp /tmp/intel-11.0.083/11.0.083-1.intel64.el5.fys /etc/modulefiles/intel **Note**: the above module file should contain at least:: prepend-path LD_LIBRARY_PATH /opt/intel/Compiler/11.0/083/lib/intel64 prepend-path PATH /opt/intel/Compiler/11.0/083/bin/intel64 prepend-path MANPATH /opt/intel/Compiler/11.0/083/man Install mkl, and build mkl compatibility package (only on **thul**):: cd ~/rpmbuild/SOURCES mkdir 10.1.3.027 cp -r /opt/intel/mkl/10.1.3.027/lib 10.1.3.027 cp -r /opt/intel/mkl/10.1.3.027/doc 10.1.3.027 tar zcf intel-redist-mkl-10.1.3.027.tar.gz 10.1.3.027 cd ~/rpmbuild/SPECS rpmbuild -bb --with modules --with default_version intel-redist-mkl.spec **Note**: do not install the resulting RPM on the login node. It needs to be installed **only** on compute nodes. On the login node only the module file needs to be deployed (as root):: mkdir -p /etc/modulefiles/mkl cp /tmp/intel-mkl-10.1.3.027/10.1.3.027-1.el5.fys.em64t /etc/modulefiles/mkl **Note**: the above module file should contain at least:: prepend-path LD_LIBRARY_PATH /opt/intel/mkl/10.1.3.027/lib/em64t Build a custom openmpi, using torque support:: export rpmtopdir=${HOME}/${FYS_PLATFORM} # set this to _topdir value from ~/.rpmmacros wget http://www.open-mpi.org/software/ompi/v1.3/downloads/openmpi-1.3.2.tar.bz2 \ -O ~/rpmbuild/SOURCES/openmpi-1.3.2.tar.bz2 sh ./buildrpm-1.3.2-1.gfortran.sh ../SOURCES/openmpi-1.3.2.tar.bz2 2>&1 | tee buildrpm-1.3.2-1.gfortran.sh.log.${FYS_PLATFORM} sh ./buildrpm-1.3.2-1.gfortran43.sh ../SOURCES/openmpi-1.3.2.tar.bz2 2>&1 | tee buildrpm-1.3.2-1.gfortran43.sh.log.${FYS_PLATFORM} sh ./buildrpm-1.3.2-1.pathscale.sh ../SOURCES/openmpi-1.3.2.tar.bz2 2>&1 | tee buildrpm-1.3.2-1.pathscale.sh.log.${FYS_PLATFORM} sh ./buildrpm-1.3.2-1.ifort.sh ../SOURCES/openmpi-1.3.2.tar.bz2 2>&1 | tee buildrpm-1.3.2-1.ifort.sh.log.${FYS_PLATFORM} # thul only **Note**: intel openmpi needs to be installed ignoring dependencies:: rpm -ivh --nodeps --oldpackage ~rpmbuild/${FYS_PLATFORM}/RPMS/openmpi-1.3.2-1.ifort.el5.fys.x86_64.rpm If scripts that contain **ALL** build/install/uninstall commands (``global_install.sh`` and ``global_uninstall.sh``) need to be created, every time after an RPM is successfully built, do:: grep -v "#\!" install.sh >> ~/${FYS_PLATFORM}/global_install.sh cat uninstall.sh ~/${FYS_PLATFORM}/global_uninstall.sh | grep -v "#\!" >> ~/${FYS_PLATFORM}/global_uninstall.sh.tmp && mv -f ~/${FYS_PLATFORM}/global_uninstall.sh.tmp ~/${FYS_PLATFORM}/global_uninstall.sh # ignore "cat: .../global_uninstall.sh: No such ..." error when running first time **Note** that `global_uninstall.sh` won't remove built RPM files, just will uninstall the packages. Build the following for dacapo: - set the disttag variable for convenience:: export disttag="el5.fys" - acml:: rpmbuild -bb --with compiler=gfortran --with version1=0 --with version2=1 --with modules --with default_version acml.spec rpmbuild -bb --with compiler=pathscale --with version1=0 --with version2=1 --with modules --with default_version acml.spec rpmbuild -bb --with compiler=gfortran43 --with version1=1 --with version2=0 --with modules --with default_version acml.spec # fjorm only rpmbuild -bb --with compiler=pathscale --with version1=1 --with version2=0 --with modules --with default_version acml.spec rpmbuild -bb --with compiler=gfortran43 --with version1=2 --with version2=0 --with modules --with default_version acml.spec rpmbuild -bb --with compiler=pathscale --with version1=2 --with version2=0 --with modules --with default_version acml.spec rpmbuild -bb --with compiler=ifort --with version1=2 --with version2=0 --with modules --with default_version acml.spec # thul only - goto (only on **fjorm**):: rpmbuild --bb --with compiler=gfortran --with modules=1 --with default_version=1 \ --with prefix=/opt/goto/1.26/1.${disttag}.gfortran.smp goto.spec rpmbuild --bb --with compiler=gfortran43 --with modules=1 --with default_version=1 \ --with prefix=/opt/goto/1.26/1.${disttag}.gfortran43.smp goto.spec rpmbuild --bb --with compiler=pathscale --with compiler_bindir=/opt/pathscale/bin --with compiler_libdir=/opt/pathscale/lib/3.2 \ --with modules=1 --with default_version=1 --with prefix=/opt/goto/1.26/1.${disttag}.pathscale.smp goto.spec **Note** - **1.26** version fails on Nehalem with:: ../../../param.h:1195:21: error: division by zero in #if - atlas:: rpmbuild --bb --with compiler=gfortran --with compiler_bindir=/usr/bin --with compiler_libdir=/usr/lib64 \ -with modules=1 --with default_version=1 --with prefix=/opt/atlas/3.8.3/1.gfortran.${disttag} atlas.spec rpmbuild --bb --with compiler=gfortran43 --with compiler_bindir=/usr/bin --with compiler_libdir=/usr/lib64 \ -with modules=1 --with default_version=1 --with prefix=/opt/atlas/3.8.3/1.gfortran43.${disttag} atlas.spec rpmbuild --bb --with compiler=pathscale --with compiler_bindir=/opt/pathscale/bin --with compiler_libdir=/opt/pathscale/lib/3.2 \ -with modules=1 --with default_version=1 --with prefix=/opt/atlas/3.8.3/1.pathscale.${disttag} atlas.spec - `campos-dacapo-pseudopotentials `_:: rpmbuild -bb --with modules --with default_version campos-dacapo-pseudopotentials.spec - `rasmol `_:: rpmbuild -bb --with modules --with default_version --with prefix=/opt/RasMol/2.7.3/3.${disttag} RasMol.spec - `cblas `_:: rpmbuild --bb --with blas_version=3.0.37.el5 --with modules=1 --with default_version=1 \ --with prefix=/opt/cblas/2.23.3/1.${disttag}.gfortran.blas.3.0.37.el5 cblas.spec rpmbuild --bb --with blas=acml --with blas_version=4.0.1 --with blasdir=/opt/acml/4.0.1/gfortran64/lib \ --with modules=1 --with default_version=1 --with prefix=/opt/cblas/2.23.3/1.${dist}.gfortran.acml.4.0.1 cblas.spec - python-setuptools:: rpmbuild -bb --with modules --with default_version --with prefix=/opt/python-setuptools/0.6c9/1.${disttag}.python2.4 python-setuptools.spec - python-nose:: rpmbuild -bb --with modules --with default_version --with prefix=/opt/python-nose/0.10.4/1.${disttag}.python2.4 python-nose.spec module load python-setuptools module load python-nose - numpy:: rpmbuild --bb --with cblas_prefix=none \ # dotblas fails with acml --with blas=acml --with blas_version=4.0.1 --with blasdir=/opt/acml/4.0.1/gfortran64/lib \ --with lapack=acml --with lapackdir=/opt/acml/4.0.1/gfortran64/lib \ --with compiler=gfortran --with modules=1 --with default_version=1 \ --with prefix=/opt/numpy/1.3.0/1.${disttag}.gfortran.python2.4.acml.4.0.1.acml numpy.spec rpmbuild --bb --with cblas_prefix=/opt/cblas/2.23.3/1.${disttag}.gfortran.blas.3.0.37.el5 --with blas_version=3.0.37.el5 \ --with compiler=gfortran --with modules=1 --with default_version=1 \ --with prefix=/opt/numpy/1.3.0/1.${disttag}.gfortran.python2.4.blas.3.0.37.el5.lapack numpy.spec Test with:: module load numpy/1.3.0-1.el5.fys.gfortran.python2.4.blas.3.0.37.el5.lapack python -c "import numpy; from numpy.core.multiarray import dot; b = numpy.ones(13, numpy.complex); dot(b, b)" python -c "import numpy; numpy.test()" module unload numpy/1.3.0-1.el5.fys.gfortran.python2.4.blas.3.0.37.el5.lapack module load acml-gfortran64/4.0.1-1.el5.fys module load numpy/1.3.0-1.el5.fys.gfortran.python2.4.acml.4.0.1.acml python -c "import numpy; from numpy.core.multiarray import dot; b = numpy.ones(13, numpy.complex); dot(b, b)" python -c "import numpy; numpy.test()" module unload numpy/1.3.0-1.el5.fys.gfortran.python2.4.acml.4.0.1.acml module unload acml-gfortran64/4.0.1-1.el5.fys Load the default numpy:: module load numpy/1.3.0-1.el5.fys.gfortran.python2.4.acml.4.0.1.acml - `gnuplot-py `_:: rpmbuild -bb --with modules --with default_version --with prefix=/opt/gnuplot-py/1.8.1/1.${disttag}.python2.4 gnuplot-py.spec module load gnuplot-py - `python-numeric `_ (we must install **24.2** version, and we keep the default version):: cd ~/${FYS_PLATFORM}/RPMS/external rpm -e --nodeps python-numeric # as root yumdownloader --resolve --disableexcludes=main python-numeric cd ~/rpmbuild/SPECS rpmbuild --bb --with cblas_prefix=/opt/cblas/2.23.3/1.${disttag}.gfortran.acml.4.0.1 \ --with blas=acml --with blas_version=4.0.1 --with blasdir=/opt/acml/4.0.1/gfortran64/lib \ --with lapack=acml --with lapackdir=/opt/acml/4.0.1/gfortran64/lib \ --with compiler=gfortran --with modules=1 --with default_version=1 \ --with prefix=/opt/python-numeric/24.2/4.${disttag}.gfortran.python2.4.acml.4.0.1.acml python-numeric.spec rpmbuild --bb --with cblas_prefix=/opt/cblas/2.23.3/1.${disttag}.gfortran.blas.3.0.37.el5 --with blas_version=3.0.37.el5 \ --with compiler=gfortran --with modules=1 --with default_version=1 \ --with prefix=/opt/python-numeric/24.2/4.${disttag}.gfortran.python2.4.blas.3.0.37.el5.lapack python-numeric.spec **Note**: (16 Apr 2009) currently Numeric's `test.py` results in (we ignore this error):: glibc detected *** python: free(): invalid next size (normal): 0x09aee970 *** After installing python-numeric make a very rough check:: module load python-numeric/24.2-4.el5.fys.gfortran.python2.4.blas.3.0.37.el5.lapack python -c "import lapack_lite" ldd `rpm -ql python-numeric-24.2-4.el5.fys.gfortran.python2.4.blas.3.0.37.el5.lapack | grep lapack_lite.so` ldd `rpm -ql python-numeric-24.2-4.el5.fys.gfortran.python2.4.blas.3.0.37.el5.lapack | grep _dotblas.so` module unload python-numeric/24.2-4.el5.fys.gfortran.python2.4.blas.3.0.37.el5.lapack module load acml-gfortran64/4.0.1-1.el5.fys module load python-numeric/24.2-4.el5.fys.gfortran.python2.4.acml.4.0.1.acml python -c "import lapack_lite" ldd `rpm -ql python-numeric-24.2-4.el5.fys.gfortran.python2.4.acml.4.0.1.acml | grep lapack_lite.so` ldd `rpm -ql python-numeric-24.2-4.el5.fys.gfortran.python2.4.acml.4.0.1.acml | grep _dotblas.so` module unload python-numeric/24.2-4.el5.fys.gfortran.python2.4.acml.4.0.1.acml module unload acml-gfortran64/4.0.1-1.el5.fys and reinstall the default version:: rpm -ivh --oldpackage ~rpmbuild/${FYS_PLATFORM}/RPMS/external/python-numeric-23.*.rpm load the default `Numeric` version:: module load acml-gfortran64/4.0.1-1.el5.fys module load python-numeric/24.2-4.el5.fys.gfortran.python2.4.acml.4.0.1.acml - netcdf:: rpmbuild --bb --with compiler=gfortran --with modules=1 --with default_version=1 \ --with prefix=/opt/netcdf4/4.0.1/1.gfortran.${disttag} netcdf4.spec rpmbuild --bb --with compiler=gfortran43 --with modules=1 --with default_version=1 \ --with prefix=/opt/netcdf4/4.0.1/1.gfortran43.${disttag} netcdf4.spec rpmbuild --bb --with compiler=pathscale --with compiler_bindir=/opt/pathscale/bin --with compiler_libdir=/opt/pathscale/lib/3.2 \ --with modules=1 --with default_version=1 --with prefix=/opt/netcdf4/4.0.1/1.pathscale.${disttag} netcdf4.spec - `ScientificPython `_:: rpmbuild --bb --with Numeric_includedir=none --with numpy=numpy --with compiler=gfortran --with netcdf=netcdf4 \ --with netcdf_includedir=/opt/netcdf4/4.0.1/1.gfortran.${disttag}/include \ --with netcdf_libdir=/opt/netcdf4/4.0.1/1.gfortran.${disttag}/lib64 \ --with modules=1 --with default_version=1 \ --with prefix=/opt/ScientificPython/2.8/1.${disttag}.python2.4.serial_version.numpy ScientificPython.spec rpmbuild --bb --with numpy=numeric --with compiler=gfortran --with netcdf=netcdf4 \ --with netcdf_includedir=/opt/netcdf4/4.0.1/1.gfortran.${disttag}/include \ --with netcdf_libdir=/opt/netcdf4/4.0.1/1.gfortran.${disttag}/lib64 \ --with modules=1 --with default_version=1 --with parallel=1 --with openmpi=openmpi \ --with ompi_bindir=/opt/openmpi/1.3.2-1.gfortran/bin \ --with ompi_includedir=/opt/openmpi/1.3.2-1.gfortran/include --with ompi_libdir=/opt/openmpi/1.3.2-1.gfortran/lib64 \ --with prefix=/opt/ScientificPython/2.6.2/1.${disttag}.python2.4.openmpi.numeric ScientificPython.spec - python-docutils:: rpmbuild -bb --with modules --with default_version --with prefix=/opt/python-docutils/0.5/1.${disttag}.python2.4 python-docutils.spec - pytz:: rpmbuild -bb --with modules --with default_version --with prefix=/opt/pytz/2008g/1.${disttag}.python2.4 pytz.spec - python-docutils:: rpmbuild -bb --with modules --with default_version --with prefix=/opt/python-dateutil/1.4.1/3.${disttag}.python2.4 python-dateutil.spec - python-matplotlib:: module load pytz module load python-docutils module load python-dateutil rpmbuild -bb --with modules --with default_version --with prefix=/opt/python-matplotlib/0.98.5.2/1.${disttag}.python2.4 python-matplotlib.spec - `campos-ase2 `_:: rpmbuild -bb --with modules --with default_version --with prefix=/opt/campos-ase2/2.3.13/1.${disttag}.python2.4 campos-ase2.spec - `campos-dacapo-python `_:: rpmbuild -bb --with modules --with default_version --with prefix=/opt/campos-dacapo-python/0.9.4/1.${disttag}.python2.4 campos-dacapo-python.spec - fftw2:: rpmbuild --bb --with compiler=gfortran --with default_version=1 --with modules=1 --with prefix=/opt/fftw2/2.1.5/12.gfortran.${disttag} fftw2.spec rpmbuild --bb --with compiler=gfortran43 --with default_version=1 --with modules=1 --with prefix=/opt/fftw2/2.1.5/12.gfortran43.${disttag} fftw2.spec rpmbuild --bb --with compiler=pathscale --with compiler_bindir=/opt/pathscale/bin --with compiler_libdir=/opt/pathscale/lib/3.2 \ --with default_version=1 --with modules=1 --with prefix=/opt/fftw2/2.1.5/12.pathscale.${disttag} fftw2.spec - `campos-dacapo `_:: rpmbuild --bb --with blas=acml --with blas_version=4.2.0 --with blasdir=/opt/acml/4.2.0/gfortran4364/lib \ --with compiler=gfortran43 --with compiler_bindir=/usr/bin --with compiler_libdir=/usr/lib64 \ --with fftw=fftw2 --with fftw_libdir=/opt/fftw2/2.1.5/12.gfortran43.${disttag}/lib64 \ --with lapack=acml --with lapackdir=/opt/acml/4.2.0/gfortran4364/lib --with netcdf=netcdf4 \ --with netcdf_includedir=/opt/netcdf4/4.0.1/1.gfortran43.${disttag}/include \ --with netcdf_libdir=/opt/netcdf4/4.0.1/1.gfortran43.${disttag}/lib64 \ --with modules=1 --with default_version=1 --with parallel=1 --with openmpi=openmpi \ --with ompi_bindir=/opt/openmpi/1.3.2-1.gfortran43/bin \ --with ompi_includedir=/opt/openmpi/1.3.2-1.gfortran43/include --with ompi_libdir=/opt/openmpi/1.3.2-1.gfortran43/lib64 \ --with prefix=/opt/campos-dacapo/2.7.16/1.${disttag}.gfortran43.openmpi.acml.4.2.0.acml campos-dacapo.spec rpmbuild --bb --with blas=acml --with blas_version=4.0.1 --with blasdir=/opt/acml/4.0.1/gfortran64/lib \ --with compiler=gfortran --with compiler_bindir=/usr/bin --with compiler_libdir=/usr/lib64 \ --with fftw=fftw2 --with fftw_libdir=/opt/fftw2/2.1.5/12.gfortran.${disttag}/lib64 \ --with lapack=acml --with lapackdir=/opt/acml/4.0.1/gfortran64/lib --with netcdf=netcdf4 \ --with netcdf_includedir=/opt/netcdf4/4.0.1/1.gfortran.${disttag}/include \ --with netcdf_libdir=/opt/netcdf4/4.0.1/1.gfortran.${disttag}/lib64 \ --with modules=1 --with default_version=1 --with parallel=1 --with openmpi=openmpi \ --with ompi_bindir=/opt/openmpi/1.3.2-1.gfortran/bin \ --with ompi_includedir=/opt/openmpi/1.3.2-1.gfortran/include --with ompi_libdir=/opt/openmpi/1.3.2-1.gfortran/lib64 \ --with prefix=/opt/campos-dacapo/2.7.16/1.${disttag}.gfortran.openmpi.acml.4.0.1.acml campos-dacapo.spec rpmbuild --bb --with blas=blas --with blas_version=3.0.37.el5 --with blasdir=/usr/lib64 \ --with compiler=gfortran43 --with compiler_bindir=/usr/bin --with compiler_libdir=/usr/lib64 \ --with fftw=fftw2 --with fftw_libdir=/opt/fftw2/2.1.5/12.gfortran43.${disttag}/lib64 \ --with lapack=lapack --with lapackdir=/usr/lib64 --with netcdf=netcdf4 \ --with netcdf_includedir=/opt/netcdf4/4.0.1/1.gfortran43.${disttag}/include \ --with netcdf_libdir=/opt/netcdf4/4.0.1/1.gfortran43.${disttag}/lib64 \ --with modules=1 --with default_version=1 --with parallel=1 --with openmpi=openmpi \ --with ompi_bindir=/opt/openmpi/1.3.2-1.gfortran43/bin \ --with ompi_includedir=/opt/openmpi/1.3.2-1.gfortran43/include --with ompi_libdir=/opt/openmpi/1.3.2-1.gfortran43/lib64 \ --with prefix=/opt/campos-dacapo/2.7.16/1.${disttag}.gfortran43.openmpi.blas.3.0.37.el5.lapack campos-dacapo.spec rpmbuild --bb --with blas=blas --with blas_version=3.0.37.el5 --with blasdir=/usr/lib64 \ --with compiler=gfortran --with compiler_bindir=/usr/bin --with compiler_libdir=/usr/lib64 \ --with fftw=fftw2 --with fftw_libdir=/opt/fftw2/2.1.5/12.gfortran.${disttag}/lib64 \ --with lapack=lapack --with lapackdir=/usr/lib64 --with netcdf=netcdf4 \ --with netcdf_includedir=/opt/netcdf4/4.0.1/1.gfortran.${disttag}/include \ --with netcdf_libdir=/opt/netcdf4/4.0.1/1.gfortran.${disttag}/lib64 \ --with modules=1 --with default_version=1 --with parallel=1 --with openmpi=openmpi \ --with ompi_bindir=/opt/openmpi/1.3.2-1.gfortran/bin \ --with ompi_includedir=/opt/openmpi/1.3.2-1.gfortran/include --with ompi_libdir=/opt/openmpi/1.3.2-1.gfortran/lib64 \ --with prefix=/opt/campos-dacapo/2.7.16/1.${disttag}.gfortran.openmpi.blas.3.0.37.el5.lapack campos-dacapo.spec Build following for gpaw: - fftw3:: rpmbuild --bb --with major_version=3 --with version1=2 --with version2=1 --with compiler=gfortran \ --with default_version=1 --with modules=1 --with prefix=/opt/fftw3/3.2.1/12.gfortran.${disttag} fftw2.spec rpmbuild --bb --with major_version=3 --with version1=2 --with version2=1 --with compiler=gfortran43 \ --with default_version=1 --with modules=1 --with prefix=/opt/fftw3/3.2.1/12.gfortran43.${disttag} fftw2.spec - scipy:: module unload numpy module load numpy/1.3.0-1.el5.fys.gfortran.python2.4.blas.3.0.37.el5.lapack rpmbuild --bb --with cblas_prefix=/opt/cblas/2.23.3/1.${disttag}.gfortran.blas.3.0.37.el5 \ --with blas=blas --with blas_version=3.0.37.el5 --with blasdir=/usrlib64 \ --with lapack=lapack --with lapackdir=/usr/lib64 \ --with ufsparse_libdir=/usr/lib64 --with ufsparse_includedir=/usr/include/suitesparse \ --with fftw_libdir=/opt/fftw3/3.2.1/12.gfortran.${disttag}/lib64 \ --with compiler=gfortran --with default_version=1 --with modules=1 \ --with prefix=/opt/scipy/0.7.0/1.${disttag}.gfortran.python2.4.blas.3.0.37.el5.lapack scipy.spec module unload numpy module load numpy/1.3.0-1.el5.fys.gfortran.python2.4.acml.4.0.1.acml rpmbuild --bb --with cblas_prefix=/opt/cblas/2.23.3/1.${disttag}.gfortran.acml.4.0.1 \ --with blas=acml --with blas_version=4.0.1 --with blasdir=/opt/acml/4.0.1/gfortran64/lib \ --with lapack=acml --with lapackdir=/opt/acml/4.0.1/gfortran64/lib \ --with ufsparse_libdir=/usr/lib64 --with ufsparse_includedir=/usr/include/suitesparse \ --with fftw_libdir=/opt/fftw3/3.2.1/12.gfortran.${disttag}/lib64 \ --with compiler=gfortran --with default_version=1 --with modules=1 \ --with prefix=/opt/scipy/0.7.0/1.${disttag}.gfortran.python2.4.acml.4.0.1.acml scipy.spec - `campos-gpaw-setups `_:: rpmbuild -bb --with default_version --with modules campos-gpaw-setups.spec - `campos-ase3 `_:: rpmbuild -bb --with modules --with default_version --with prefix=/opt/campos-ase3/3.1.0.846/1.${disttag}.python2.4 campos-ase3.spec - povray:: rpmbuild -bb --with modules --with default_version --with prefix=/opt/povray/3.6.1/3.${disttag} povray.spec - python-jinja2:: rpmbuild -bb --with modules --with default_version --with prefix=/opt/python-jinja2/2.1.1/1.${disttag}.python2.4 python-jinja2.spec - python-pygments:: rpmbuild -bb --with modules --with default_version --with prefix=/opt/python-pygments/1.0/1.${disttag}.python2.4 python-pygments.spec - babel:: rpmbuild -bb --with modules --with default_version --with prefix=/opt/babel/0.9.4/1.${disttag}.python2.4 babel.spec - python-sphinx:: module load python-jinja2 rpmbuild -bb --with modules --with default_version --with prefix=/opt/python-sphinx/0.6.1/1.${disttag}.python2.4 python-sphinx.spec - blacs:: rpmbuild --bb --with compiler=gfortran43 --with compiler_bindir=/usr/bin --with compiler_libdir=/usr/lib64 \ --with modules=1 --with default_version=1 --with parallel=1 --with openmpi=openmpi \ --with ompi_bindir=/opt/openmpi/1.3.2-1.gfortran43/bin \ --with ompi_includedir=/opt/openmpi/1.3.2-1.gfortran43/include --with ompi_libdir=/opt/openmpi/1.3.2-1.gfortran43/lib64 \ --with prefix=/opt/blacs/1.1/24.${disttag}.gfortran43.openmpi blacs.spec rpmbuild --bb --with compiler=gfortran --with compiler_bindir=/usr/bin --with compiler_libdir=/usr/lib64 \ --with modules=1 --with default_version=1 --with parallel=1 --with openmpi=openmpi \ --with ompi_bindir=/opt/openmpi/1.3.2-1.gfortran/bin \ --with ompi_includedir=/opt/openmpi/1.3.2-1.gfortran/include --with ompi_libdir=/opt/openmpi/1.3.2-1.gfortran/lib64 \ --with prefix=/opt/blacs/1.1/24.${disttag}.gfortran.openmpi blacs.spec - scalapack:: rpmbuild --bb --with blacs=blacs --with blacsdir=/opt/blacs/1.1/24.${disttag}.gfortran43.openmpi/lib64 \ --with blas=acml --with blas_version=4.2.0 --with blasdir=/opt/acml/4.2.0/gfortran4364/lib \ --with compiler=gfortran43 --with compiler_bindir=/usr/bin --with compiler_libdir=/usr/lib64 \ --with default_version=1 --with modules=1 --with openmpi=openmpi \ --with lapack=acml --with lapackdir=/opt/acml/4.2.0/gfortran4364/lib \ --with ompi_bindir=/opt/openmpi/1.3.2-1.gfortran43/bin \ --with ompi_includedir=/opt/openmpi/1.3.2-1.gfortran43/include --with ompi_libdir=/opt/openmpi/1.3.2-1.gfortran43/lib64 \ --with prefix=/opt/scalapack/1.8.0/1.${disttag}.gfortran43.openmpi.acml.4.2.0.acml scalapack.spec rpmbuild --bb --with blacs=blacs --with blacsdir=/opt/blacs/1.1/24.${disttag}.gfortran.openmpi/lib64 \ --with blas=acml --with blas_version=4.0.1 --with blasdir=/opt/acml/4.0.1/gfortran64/lib \ --with compiler=gfortran --with compiler_bindir=/usr/bin --with compiler_libdir=/usr/lib64 \ --with default_version=1 --with modules=1 --with openmpi=openmpi \ --with lapack=acml --with lapackdir=/opt/acml/4.0.1/gfortran64/lib \ --with ompi_bindir=/opt/openmpi/1.3.2-1.gfortran/bin \ --with ompi_includedir=/opt/openmpi/1.3.2-1.gfortran/include --with ompi_libdir=/opt/openmpi/1.3.2-1.gfortran43/lib \ --with prefix=/opt/scalapack/1.8.0/1.${disttag}.gfortran.openmpi.acml.4.0.1.acml scalapack.spec rpmbuild --bb --with blacs=blacs --with blacsdir=/opt/blacs/1.1/24.${disttag}.gfortran43.openmpi/lib64 \ --with blas=blas --with blas_version=3.0.37.el5 --with blasdir=/usr/lib64 \ --with compiler=gfortran43 --with compiler_bindir=/usr/bin --with compiler_libdir=/usr/lib64 \ --with default_version=1 --with modules=1 --with openmpi=openmpi \ --with lapack=lapack --with lapackdir=/usr/lib64 \ --with ompi_bindir=/opt/openmpi/1.3.2-1.gfortran43/bin \ --with ompi_includedir=/opt/openmpi/1.3.2-1.gfortran43/include --with ompi_libdir=/opt/openmpi/1.3.2-1.gfortran43/lib64 \ --with prefix=/opt/scalapack/1.8.0/1.${disttag}.gfortran43.openmpi.blas.3.0.37.el5.lapack scalapack.spec rpmbuild --bb --with blacs=blacs --with blacsdir=/opt/blacs/1.1/24.${disttag}.gfortran.openmpi/lib64 \ --with blas=blas --with blas_version=3.0.37.el5 --with blasdir=/usr/lib64 \ --with compiler=gfortran --with compiler_bindir=/usr/bin --with compiler_libdir=/usr/lib64 \ --with default_version=1 --with modules=1 --with openmpi=openmpi \ --with lapack=lapack --with lapackdir=/usr/lib64 \ --with ompi_bindir=/opt/openmpi/1.3.2-1.gfortran/bin \ --with ompi_includedir=/opt/openmpi/1.3.2-1.gfortran/include --with ompi_libdir=/opt/openmpi/1.3.2-1.gfortran/lib64 \ --with prefix=/opt/scalapack/1.8.0/1.${disttag}.gfortran.openmpi.blas.3.0.37.el5.lapack scalapack.spec - `campos-gpaw `_:: rpmbuild --bb --with blacs=blacs --with blacsdir=/opt/blacs/1.1/24.${disttag}.gfortran.openmpi/lib64 \ --with blas=acml --with blas_version=4.0.1 --with blasdir=/opt/acml/4.0.1/gfortran64/lib \ --with cmr_repository=/home/niflheim/repository/db \ --with compiler=gfortran --with compiler_bindir=/usr/bin --with compiler_libdir=/usr/lib64 \ --with default_version=1 --with modules=1 \ --with lapack=acml --with lapackdir=/opt/acml/4.0.1/gfortran64/lib \ --with ompi_bindir=/opt/openmpi/1.3.2-1.gfortran/bin \ --with ompi_includedir=/opt/openmpi/1.3.2-1.gfortran/include --with ompi_libdir=/opt/openmpi/1.3.2-1.gfortran/lib64 \ --with openmpi=openmpi --with parallel=1 \ --with scalapack=scalapack --with scalapackdir=/opt/scalapack/1.8.0/1.${disttag}.gfortran.openmpi.acml.4.0.1.acml/lib64 \ --with prefix=/opt/campos-gpaw/0.6.3862/1.${disttag}.gfortran.python2.4.openmpi.acml.4.0.1.acml campos-gpaw.spec rpmbuild --bb --with blacs=blacs --with blacsdir=/opt/blacs/1.1/24.${disttag}.gfortran43.openmpi/lib64 \ --with blas=acml --with blas_version=4.2.0 --with blasdir=/opt/acml/4.2.0/gfortran4364/lib \ --with cmr_repository=/home/niflheim/repository/db \ --with compiler=gfortran43 --with compiler_bindir=/usr/bin --with compiler_libdir=/usr/lib64 \ --with default_version=1 --with modules=1 \ --with lapack=acml --with lapackdir=/opt/acml/4.2.0/gfortran4364/lib \ --with ompi_bindir=/opt/openmpi/1.3.2-1.gfortran43/bin \ --with ompi_includedir=/opt/openmpi/1.3.2-1.gfortran43/include --with ompi_libdir=/opt/openmpi/1.3.2-1.gfortran43/lib64 \ --with openmpi=openmpi --with parallel=1 \ --with scalapack=scalapack --with scalapackdir=/opt/scalapack/1.8.0/1.${disttag}.gfortran43.openmpi.acml.4.2.0.acml/lib64 \ --with prefix=/opt/campos-gpaw/0.6.3862/1.${disttag}.gfortran43.python2.4.openmpi.acml.4.2.0.acml campos-gpaw.spec Testing packages ================ Test `dacapo` installation (as **normal** user!). If you use modules:: module load openmpi module load campos-dacapo-pseudopotentials module load python-numeric module load campos-dacapo-python module load ScientificPython module load gnuplot-py module load RasMol module load campos-ase2 module load campos-dacapo ulimit -s 65000 # dacapo needs a large stack Test with (make sure that `/scratch/$USER` exists):: cp -r `rpm -ql campos-dacapo-python | grep "share/campos-dacapo-python$"` /tmp cd /tmp/campos-dacapo-python/Tests python test.py 2>&1 | tee test.log It can take up to 1 day. Please consider disabling these "long" tests in `test.py`:: tests.remove('../Examples/Wannier-ethylene.py') tests.remove('../Examples/Wannier-Pt4.py') tests.remove('../Examples/Wannier-Ptwire.py') tests.remove('../Examples/Wannier-Fe-bcc.py') tests.remove('../Examples/transport_1dmodel.py') **Note** all `vtk` related tests will fail. Test `gpaw` installation (as **normal** user!): If you use modules:: module load openmpi module load campos-ase3 module load campos-gpaw-setups module load campos-gpaw Test with:: cp -r `rpm -ql campos-gpaw | grep "share/campos-gpaw/test$"` /tmp/test.gpaw.$$ cd /tmp/test.gpaw.* python test.py 2>&1 | tee test.log It takes about 20 minutes. On "Golden Client" ------------------ Login, as root, to the "Golden Client":: ssh n001 Enable nfs mount of the `server` home directory - follow 'Enable nfs mount on the "Golden Client"' from `configuring NFS `_. After this do:: cd /home/dulak-server/rpm/campos rpm -ivh campos-dacapo-2* If getting:: package example_package.el5.i386 is already installed remove these packages with:: rpm -e --nodeps example_package to allow the installation to proceed. Make sure that both python-numeric versions are installed:: rpm -q python-numeric This command will show a list of packages that need to be installed to fulfill `dacapo` dependencies. All these packages should be already under ``/home/dulak-server/rpm``. Remember to test the `dacapo` and `gpaw` installations on the "Golden Client" too. If you are installing workstation only, your setup is ready for testing - go to `benchmarking and maintenance `_. If you are building a cluster go **back** to `installing and configuring systemimager `_,