This file contains information about how I install the optional auxillary
packages.  It is intended to be notes to myself, to remind me how I
previously built the application when I upgrade to a new version, or install
on a new machine.  But you might find some of this information useful.

General approach
----------------

I use an environment variable <app name>_HOME (e.g. ARPACK_HOME) to give
the root directory of the installation of an application.  I install
applications under /local/apps.  In general, the home directory contains lib
and include subdirectories with the libraries and include files (after
installation), but the details depend on the philosophy of the application.

When the application requires a different compilation for different Fortran 90
compilers, I put a separate subdirectory of the home directory for each
compiler, and put library and module files in lib and include subdirectories
under each of them, e.g. $LAM_HOME/lahey/lib and $LAM_HOME/lahey/include.  The
name of the subdirectory is the same as the value for $PHAML_F90 in mkmkfile.sh.

When the application requires a different compilation for different MPI
libraries, I put a separate directory for each MPI library under the compiler
directory (or under the home directory if the compiler directory isn't used),
e.g. $MUMPS_HOME/lahey/mpich/lib.  The name of the subdirectory is the same
as the value for $PHAML_PARLIB in mkmkfile.sh.

Absoft f90
----------

The flags to have Absoft f90 use the single underscore for external names are
-YEXT_NAMES=LCS -YEXT_SFX=_ -YCFRL=1

ARPACK
------

The version I have tested is:
arpack96.tar.gz  Sep 24 1996
patch.tar.gz     Aug  2 2001
parpack96.tar.gz Oct  7 1997
ppatch.tar.gz    Aug  2 2001

cp ARMAKES/ARmake.MPI-SUN4 ARmake.inc

home = <top ARPACK source directory>
PLAT    = LINUX
If LAPACK and/or BLAS libraries are available, change DIRS as instructed.
ARPACKLIB  = $(home)/libarpack.a
PARPACKLIB = $(home)/libparapack.a
FC      = mpif90  (mpif77 for LAM)
FFLAGS  = -O -fno-second-underscore (g77, g95, pathscale) -ff90 (g77)
LAMHF77=ifort (if using lam and desired compiler is not the default)
OMPI_F77=ifort (if using OpenMPI and
OMPI_FC=ifort   desired compiler is not the default)
PFC     = mpif90
PFFLAGS = -O -fno-second-underscore (g77, g95, pathscale) -ff90 (g77)
MAKE = /usr/bin/make

Copy <mpi include directory>/mpif.h to
    ARPACK/PARPACK/SRC/MPI/mpif.h
    ARPACK/PARPACK/UTIL/MPI/mpif.h
    ARPACK/PARPACK/EXAMPLES/MPI/mpif.h

make lib
make plib

The -ff90 flag is because of the use of "real(double precision complex)"
which g77 says is ambiguous as to whether it should return a single or
double precision result.  -ff90 forces a double precision result.

For the NAG compiler include flags -dusty -w=all -dcfuns

ATLAS
-----

On Linux, I build the ATLAS libraries, and merge their LAPACK into the
standard LAPACK library.

make

This will ask several questions; the answers seem pretty obvious.  One of
them will assign an architecture name (e.g. Linux_P4SSE2) which I refer
to as <arch> below.

Then edit Make.<arch>
  On F2CDEFS, remove second underscore from -DAdd__
  Set F77 to the compiler
  On F77FLAGS, add -fno-second-underscore (for g77 et al.)

make install arch=<arch>

Merge the ATLAS partial lapack library into a copy of the system lapack library:
  cd lib/<arch>
  mkdir tmp
  cd tmp
  ar x ../liblapack.a
  cp /usr/lib/liblapack.a ../liblapack.a
  ar r ../liblapack.a *.o
  cd ..
  rm -rf tmp

then move lib/<arch>/lib* to where I want them located.

Because the existing LAPACK library being merged with this was compiled with
g77 (/usr/lib/liblapack.a on Linux), this will require linking in g2c, so
might as well just make one version of ATLAS with g77.

DRUM
----

DRUM 20070511

Unpack both Zoltan and DRUM in the apps directory, and set up the config
files for both of them.  Zoltan's config should have DRUM variables at the
end.  DRUM config is in drum/config.  Also check the xml file in
drum/example/xmls.  Set up DRUM with "gmake setup" and then build it
with "gmake".  Build Zoltan after DRUM.

GLUT
----

GLUT 3.7.1

From the distribution that is available on the f90gl web page.  On Linux.

mkmkfiles.imake
make

The only files that are needed are <gluthome>/lib/glut/libglut.a,
<gluthome>/include/GL/glut.h and <gluthome>/include/GL/glutf90.h.
I copy these to the Mesa lib and include directories.

The MesaGLUT distribution also works starting with Mesa version 7.0.3.
See Mesa instructions for this version.

MESA
----

Mesa Version 7.0.3

If you are also doing MesaGLUT, unpack it on top of Mesa before compiling
Mesa.

You might want to edit <mesa home>/src/mesa/main/config.h to change
DEFAULT_SOFTWARE_DEPTH_BITS to 31 to fix a transparency problem where a
hidden side of a surface comes through the front.

Edit configs/default and change INSTALL_DIR to the installation directory,
e.g. /local/apps/mesa

Edit configs/linux to remove -DHAVE_POSIX_MEMALIGN, because LAM's free() and
posix_memalign() don't play together nicely.

make linux-x86  (on 64 bit arch, use make linux-x86-64)
make install

on 64 bit machines, link lib64 to lib so I don't have to change the location
of the Mesa/glut/f90gl libraries in all my makefiles

On Cygwin, ./configure --prefix=/home/mitchell/apps/mesa --disable-egl --disable-glw --disable-gallium
make
make install
cp lib/*.dll.a to $MESA_HOME/lib
find cygGL-1.dll, cygGLU-1.dll and cygGLUT-3.dll and move them to a directory
  in your executable path, e.g. ~/bin, or make $MESA_HOME/bin in your path

ParMETIS
--------

ParMetis Version 3.1.0

On Linux, for use with LAM, in Makefile.in change mpicc to hcc in two places,
and make.

I have also had good luck with setting CC and LD to gcc and INCDIR to the
directory containing mpi.h (for building libraries, but not tests).

Then mkdir lam/lib and move the two libraries to that directory.

May need to add read and execute permission for others on ParMETIS directory.

PETSc
-----

PETSc Version 3.9.3

setenv PETSC_DIR `pwd`
setenv PETSC_ARCH linux

setenv LAMHF77 ifort (if using lam and desired compiler is not the default)
setenv OMPI_F77 ifort (if using OpenMPI and
setenv OMPI_FC ifort   desired compiler is not the default)

If you cannot execute programs in /tmp
setenv TMPDIR <directory to use as tmp>

./configure --prefix=/local/apps/petsc/intel/openmpi --with-cc=mpicc --with-fc=mpif90 --with-cxx=mpic++ --with-debugging=0 --with-shared-libraries=0 --with-x=0

follow the suggested make commands that appear at the end of each phase

clean up with:
make allclean

If compiling for use with SLEPc, omit --with-x=0
If debugging is desired, omit --with-debugging=0

To make a sequential version:
   add --with-mpi=0 to the configure flags and change the compilers to
   --with-cc=gcc --with-fc=ifort --with-cxx=g++

To make a complex version:
   add --with-scalar-type=complex --with-fortran-kernels=generic

To include support for SuperLU, add
  --download-superlu  for sequential
  --download-superlu_dist --download-parmetis --download-metis for parallel

NOTE: 8/19/16 With sequential PETSc 3.7.3 libsuperlu.a was put in
      <petsc install>/lib64 instead of lib.  I put a symbolic link to it in lib.

To include support for hypre (parallel only), add
  --download-hypre

PETSc does not support hypre in a sequential build, because of conflicts
between the mpiuni of the two packages, according to a post on petsc-users
4/27/12.
Tried again 5/22/16; says it cannot find needed MPI.

To include support for ML from Trilinos (parallel only), add
  --download-ml

Attempting to include ML in a sequential build gives "Did not find package MPI
needed by ml."

To include support for MUMPS, add

 for sequential
  --download-mumps --with-mumps-serial
  before PETSc 3.8 also use --with-mpiuni-fortran-binding=0

 for parallel
  --download-mumps --download-scalapack --download-blacs --download-parmetis --download-metis

NAG

  chokes on -pthread and -m64 in mpif90.  See what the compiler flags are in
  /usr/lib64/openmpi/share/openmpi/mpif90-wrapper-data.txt
  and set OMPI_FCFLAGS to them without -pthread -m64.
  If that file doesn't exist, use the flags shown by mpif90 --showme except for
  -pthread, -m64, -I flags, and the libraries.

  It also doesn't like -rpath in the linker flags.  I solved this with
  setenv OMPI_LDFLAGS "-L/usr/lib64/openmpi/lib -Wl,--enable-new-dtags"
  where the -L part replaced -Wl,-rpath -Wl,@{libdir} in the wrapper data.

  The test of HYPRE failed saying it was missing something from libm, so I
  also added -lm to OMPI_LDFLAGS.

  NAG gives too many obsolete warnings and needs double complex intrinsics.
  FFLAGS="-w=all -dusty -dcfuns"

  Also, to properly handle iarc and getarg,
  CFLAGS=-DPETSC_HAVE_NAGF90

  NAG does not support the #error preprocessor directive.  In PETSc 3.7.3 this
  occurs only in include/petsc/finclude/petscisdef.h.  One crude workaround is
  to delete the #error statement and the #else above it.
  email on 8/29/16 says this will be fixed in 3.7.4

  I was not able to build PETSc with any Fortran external packages (MUMPS)
  using NAG because of complications with the NAG runtime library.

Absoft
  --FFLAGS=-YNO_CDEC

g95

  sequential, --FFLAGS="-fno-second-underscore -pthread"
  parallel, --FFLAGS=-fno-second-underscore

gfortran

  sequential, with MUMPS, --FFLAGS=-pthread

Lahey

  with a library compiled with Lahey (for example, MUMPS)
  --CC_LINKER_FLAGS="-L/local/apps/lahey/lib64 -lfj90i -lfj90f -lelf"

  for parallel with MUMPS, BLACS needs mpi_init_, and SCALAPACK needs the
  runtime libraries
  --CC_LINKER_FLAGS="-lmpi_f77 -L/local/apps/lahey/lib64 -lfj90i -lfj90f -lelf"

  copy <petsc-src>/linux/*.mod to the include directory

cygwin, gfortran

  make sure cygwin python is installed

  setenv PETSC_DIR `pwd`
  setenv PETSC_ARCH cygwin
  ./configure --prefix=/home/mitchell/apps/petsc/gfortran/none --with-cc=gcc --with-cxx=g++ --with-fc=gfortran --with-windows-graphics=0 --with-mpi=0 --FFLAGS=-fno-second-underscore --download-fblaslapack --with-debugging=0 --with-x=0

before version 3.5 fblaslapack was f-blas-lapack

cygwin/MinGW for a dll

  ./configure --prefix=/home/mitchell/apps/petsc/mingw/none --with-cc=x86_64-w64-mingw32-gcc --with-cxx=x86_64-w64-mingw32-g++ --with-fc=x86_64-w64-mingw32-gfortran --with-windows-graphics=0 --with-mpi=0 --FFLAGS="-fno-second-underscore -static-libgcc -static-libgfortran" --with-ar=x86_64-w64-mingw32-ar --with-ranlib=x86_64-w64-mingw32-ranlib --download-fblaslapack --with-debugging=0 --with-shared-libraries=0 --with-x=0

before version 3.5 fblaslapack was f-blas-lapack

On raritan, I also need (but I didn't need this with 3.7.3 on raritan2)
--with-lapack-lib=/usr/lib/liblapack.a --with-blas-lib=/usr/lib/libblas.a
--CC_LINKER_FLAGS=-lpthread

SLEPc
-----

SLEPc Version 3.9.2

SLEPc picks up most of its options from the PETSc installation.
Make sure LAMHF77 or OMPI_F77 and OMPI_FC are set to the right compiler.

unsetenv PETSC_ARCH
setenv PETSC_DIR /local/apps/petsc/intel/openmpi
setenv SLEPC_DIR `pwd`

./configure --prefix=/local/apps/slepc/intel/openmpi

  for ARPACK, --with-arpack-flags=-L/local/apps/arpack/intel/openmpi/lib,-lparpack,-larpack (for scalar omit -lparpack)

  for BLOPEX, --download-blopex

follow the suggested make commands that appear at the end of each phase

With Absoft and Lahey, I needed to add a second -larpack after -lparpack.
This may have been solved by reversing the order of arpack and parpack so
parpack comes first.

I couldn't get ARPACK to work with nag, gfortran or g95 because of things it
needs in the runtime library.

TRIANGLE
--------

Triangle Version 1.6

On 32 bit Linux, no changes -- just "make".

On 64 bit Linux, might need to change X11R6/lib to X11R6/lib64

On other systems will probably need to change CSWITCHES in makefile (at
least to remove -DLINUX).

ZOLTAN
------

Zoltan Version 1.52

edit Utilities/Config/Config.nistlinux (or appropriate file) to set
the compiler, MPI library, and ParMETIS location (and other variables
as appropriate).

make YES_FORTRAN=1 ZOLTAN_ARCH=nistlinux zoltan

mv Obj_nistlinux/*.mod <compiler>/<mpi>/include
mv Obj_nistlinux/lib* <compiler>/<mpi>/lib
