Installing software in your home directory: Difference between revisions

Almost all software that is used on the new clusters is distributed centrally, using the CVMFS file system. What this means in practise is that this software is not installed under <code>/usr/bin</code>, <code>/usr/include</code>, and so on, as it would be in a Linux distribution, but instead somewhere under <code>/cvmfs/soft.computecanada.ca</code>, and is identical on all new clusters.

The core of this software stack is provided by the <code>nixpkgs/16.09</code> module, which is loaded by default. This software stack, internally managed using the Nix package manager, is located at <code>/cvmfs/soft.computecanada.ca/nix/var/nix/profiles/16.09</code>. The environment variable <code>$EBROOTNIXPKGS</code> should be used to refer to this path.

Some software, however, has been hardcoded to look under <code>/usr</code>. If that is the case, the compilation will typically fail, and needs to be explicitly told about <code>$EBROOTNIXPKGS</code>. Sometimes that means adjusting a Makefile, sometimes it needs to be specified in a certain <code>--with-</code> flag for the configure script, or a configuration file needs to be edited. If you are not sure how to do this please do not hesitate to ask for help.

Similarly, if a package depends on a library that is provided by a module other than <code>nixpkgs</code>, you may need to provide the location of the header files and libraries of that module. The EBROOT convention applies to all modules, e.g. the HDF5 library is installed in <code>$EBROOTHDF5</code>, and its <code>include</code> and <code>lib</code> subdirectories once you <code>module load hdf5</code>.

If a header file or library that would usually be provided by an RPM or other package manager in a typical Linux distributon is neither present via <code>nixpkgs</code> or via another module, please let us know. Most likely it can be easily added to the existing stack.

* all binaries under <code>/cvmfs/soft.computecanada.ca</code> use what is called a RUNPATH, which means that the directories for the runtime libraries that these binaries depend on are put inside the binary. That means it is generally *not* necessary to use <code>$LD_LIBRARY_PATH</code>. In fact, <code>$LD_LIBRARY_PATH</code> overrides this runpath and you should '''not''' set that environment variable to locations such as <code>/usr/lib64</code> or <code>$EBROOTNIXPKGS/lib</code>. Many binaries will no longer work if you attempt this.

* if you install precompiled binaries in your home directory (for example [https://conda.io/miniconda.html Miniconda]) they may fail using errors such as <code>/lib64/libc.so.6: version `GLIBC_2.18' not found</code>. Often such binaries can be patched using our <code>setrpaths.sh</code> script, using the syntax <code>setrpaths.sh --path path [--add_origin]</code> where path refers to the directory where you installed that software. This script will make sure that the binaries use the correct interpreter, and search for the libraries they are dynamically linked to in the correct folder. The option <code>--add_origin</code> will also add $ORIGIN to the RUNPATH. This is sometimes helpful if the library cannot find other libraries in the same folder as itself.