Help:Module

module is a system for managing your shell environment to make multiple conflicting software packages and versions available.

Software homepage

http://modules.sourceforge.net/

Software availability

most rocks clusters; other unix systems on request

ubuntu: see directions below to install

Other related software

rocks, cluster, rocks cluster, unix

commands to type to run

module list

module avail

module help

View online documentation

man module

Cluster specific module documentation

crcv, jay, neat, nlp

Additional notes

Most rocks clusters have the module system configured. The module system uses configuration files to make changes to the user environment to add and remove versions of multiple software packages.

As part of the install, I have fully deployed the module system which makes edits to your environment that can be temporary or permanent. There are a number of self contained software packages installed, and I have created config files for module that will make these work.

For instance:

% module list
Currently Loaded Modulefiles:
 1) rocks-openmpi

% module avail
------------------------ /usr/share/Modules/modulefiles ------------------------
dot              modules          rocks-openmpi
module-git       null             rocks-openmpi_ib
module-info      opt-python       use.own

------------------------------- /etc/modulefiles -------------------------------
cuda/7.5       openmpi-x86_64

----------------------------- /share/apps/Modules ------------------------------
localapps torch

% module load cuda
% module list
Currently Loaded Modulefiles:
 1) rocks-openmpi   2) cuda/7.5

% module help

modules available

These environments can be loaded with the module command. Note that not all modules are available on all systems (ask if you need one added)

Please consult the appropriate page for your system to see exactly what software each module includes. (see above)

Use module load XXX to enable software listed below:

cuda

nVidia cuda and cudnn

cudnn

NVIDIA CUDA Deep Neural Network library; if this is missing, it is included in the cuda module

localapps

apps installed in /share/apps/local (clusters only)

opencv

http://opencv.org/

torch

torch is a scientific computing framework with wide support for machine learning algorithms that puts GPUs first; this may include additional modules, see help:torch for details

matlab

various versions of matlab

caffe

caffe is a deep learning framework

caffe-deps

contains libraries needed to compile caffe including gflags, glog 0.3.3, boost 1.60

anaconda

prepackaged python libraries; usually the system python works better in linux; may contain a buggy version of boost

theano-force-gpu

sometimes theano will use the cpu even when the gpu is available. This tries to fix that.

Installing modules in ubuntu

If you would like to use the module system on your own ubuntu system, you can install it and configure it using these instructions.

• install modules

apt-get install environment-modules

• activate the module system in your current shell (or log out and log back in)

source /etc/profile.d/modules.sh

• check the installed location

module avail
------------------------ /usr/share/modules/modulefiles ------------------------
dot         module-git  module-info modules     null        use.own

• Create module files for software you are using. Options are:
  • add modules to the system directory (listed above: /usr/share/modules/modulefiles )
  • module load use.own and then create the listed directory (in your home directory) and place modules there

creating your own modules

As a shortcut, you can download this script to automatically create modules from installed software. Note that this is probably already installed if you are using a CECS server.

wget http://newton.i2lab.ucf.edu/~ssd/src/makemodule
chmod +x makemodule

Example use:

mkdir  /usr/share/modules/modulefiles/cuda
./makemodule /usr/local/cuda-8.0 > /usr/share/modules/modulefiles/cuda/8.0

If you are compiling software and installing it in your home directory, you should also create a privatemodules directory

 module load use.own

You can see what this does with

 module show use.own

Then save modules you make for software you compile in that directory.

If you wanted to install both cudnn 5 and cudnn 6, you could make modules for both like this:

• download cudnn from https://developer.nvidia.com/cudnn
  • get version 6 for cuda 8, instead of the deb file, get cuDNN v6.0 Library for Linux
  • get version 5.1 for cuda 8, cuDNN v5.1 Library for Linux
• Unpack the libraries in their own directories

cd /usr/local
mkdir un
cd un
tar xvzf ~/cudnn-8.0-linux-x64-v5.1.tgz
mv cuda ../cudnn-8-51
tar xzvf ~/cudnn-8.0-linux-x64-v6.0.tgz
mv cuda ../cudnn-8-60
cd ..
rmdir un

• make the modules

mkdir /usr/share/modules/modulefiles/cudnn
makemodule /usr/local/cudnn-8-51 | sudo tee  /usr/share/modules/modulefiles/cudnn/8-5.1
makemodule /usr/local/cudnn-8-60 | sudo tee /usr/share/modules/modulefiles/cudnn/8-6.0

• You may also want to edit both of these files and add the following line since these depend on cuda 8:

module load cuda/8.0