Utiliser des modules/en: Difference between revisions

Jump to navigation Jump to search
Updating to match new version of source page
No edit summary
(Updating to match new version of source page)
Line 1: Line 1:
<languages />
<languages />
Compute Canada servers can execute all software that runs under Linux. In the simplest case, the software you need will already be installed on one of the compute servers. It will then be accessible in the form of a "module". If this is not the case, you can either ask our staff to install it for you, or do it yourself.  
<div class="mw-translate-fuzzy">
Compute Canada servers can execute all software that runs under Linux. In the simplest case, the software you need will already be installed on one of the compute servers. It will then be accessible in the form of a "module". If this is not the case, you can either ask our staff to install it for you, or do it yourself.
</div>


Modules are configuration files that contain instructions for modifying your software environment. This modular architecture allows multiple versions of the same application to be installed without conflict. For new Compute Canada servers, modules are managed with the [https://www.tacc.utexas.edu/research-development/tacc-projects/lmod Lmod] tool developed at [https://www.tacc.utexas.edu/ TACC]. This tool replaces [http://modules.sourceforge.net ''Environment Modules''], which is used on most legacy servers. If you are familiar with this system you should not be too disoriented since "Lmod" was designed to be very similar to "Environment Modules". Refer to the [[#Lmod vs Environment Modules]] section for the main differences between the two systems.  
<div class="mw-translate-fuzzy">
Modules are configuration files that contain instructions for modifying your software environment. This modular architecture allows multiple versions of the same application to be installed without conflict. For new Compute Canada servers, modules are managed with the [https://www.tacc.utexas.edu/research-development/tacc-projects/lmod Lmod] tool developed at [https://www.tacc.utexas.edu/ TACC]. This tool replaces [http://modules.sourceforge.net ''Environment Modules''], which is used on most legacy servers. If you are familiar with this system you should not be too disoriented since "Lmod" was designed to be very similar to "Environment Modules". Refer to the [[#Lmod vs Environment Modules]] section for the main differences between the two systems.
</div>


<div class="mw-translate-fuzzy">
A "modulefile" contains the information needed to make an application or library available in the user's login session. Typically a module file contains instructions that modify or initialize environment variables such as <code>PATH</code> and <code>LD_LIBRARY_PATH</code> in order to use different installed programs. Note that the simple fact of loading a module doesn't execute the software in question. To learn the name of the program binary or the syntax for its use, you should read the documentation for this software. By using the <tt>module</tt> command, you shouldn't normally need to know the exact location or path of the software or library but you can nevertheless see such details about the module by means of the command <tt>module show <module-name></tt>.
A "modulefile" contains the information needed to make an application or library available in the user's login session. Typically a module file contains instructions that modify or initialize environment variables such as <code>PATH</code> and <code>LD_LIBRARY_PATH</code> in order to use different installed programs. Note that the simple fact of loading a module doesn't execute the software in question. To learn the name of the program binary or the syntax for its use, you should read the documentation for this software. By using the <tt>module</tt> command, you shouldn't normally need to know the exact location or path of the software or library but you can nevertheless see such details about the module by means of the command <tt>module show <module-name></tt>.
</div>


= Important <code>module</code> commands =
= Important <code>module</code> commands =
Line 10: Line 16:
{{Command | module command [other options]}}
{{Command | module command [other options]}}


<div class="mw-translate-fuzzy">
To see a list of available sub-commands use
To see a list of available sub-commands use
{{Command|module help}}
{{Command|module help}}
</div>


<div class="mw-translate-fuzzy">
== Sub-command <code>avail</code> ==
== Sub-command <code>avail</code> ==
To list the modules available on a given system, use
To list the modules available on a given system, use
{{Command|module avail}}
{{Command|module avail}}
</div>


<div class="mw-translate-fuzzy">
You can obtain a list of modules available for a particular library or tool, for example modules related to <code>openmpi</code>:
You can obtain a list of modules available for a particular library or tool, for example modules related to <code>openmpi</code>:
{{Command|module avail openmpi}}
{{Command|module avail openmpi}}
</div>


Note that the <code>module avail</code> command may not list some modules that are incompatible with the modules you have loaded. To see the complete list of all modules use the <code>spider</code> sub-command documented below.  
<div class="mw-translate-fuzzy">
Note that the <code>module avail</code> command may not list some modules that are incompatible with the modules you have loaded. To see the complete list of all modules use the <code>spider</code> sub-command documented below.
</div>


<div class="mw-translate-fuzzy">
== Sub-command <code>spider</code> (Lmod only) ==
== Sub-command <code>spider</code> (Lmod only) ==
The <code>spider</code> sub-command searches the complete tree of all modules and displays it.
The <code>spider</code> sub-command searches the complete tree of all modules and displays it.
{{Command|module spider}}
{{Command|module spider}}
</div>


<div class="mw-translate-fuzzy">
If you specify the name of an application, for example with
If you specify the name of an application, for example with
{{Command|module spider openmpi}}
{{Command|module spider openmpi}}
this will show you a list of all available versions of the application.  
this will show you a list of all available versions of the application.
</div>


<div class="mw-translate-fuzzy">
If you specify the name of the application along with a version number, for example with
If you specify the name of the application along with a version number, for example with
{{Command|module spider openmpi/1.8.4}}
{{Command|module spider openmpi/1.8.4}}
this will display a list of the modules you must load in order to access this version.
this will display a list of the modules you must load in order to access this version.
</div>


== Sub-command <code>list</code> ==
== Sub-command <code>list</code> ==
Line 49: Line 70:
If you load a module that is incompatible with one you already have loaded, Lmod will tell you that it has replaced the old module with a new one. This can occur especially for compilers and MPI implementations.  
If you load a module that is incompatible with one you already have loaded, Lmod will tell you that it has replaced the old module with a new one. This can occur especially for compilers and MPI implementations.  


<div class="mw-translate-fuzzy">
== Sub-command <code>unload</code> ==
== Sub-command <code>unload</code> ==
After the <code>load</code> sub-command, <code>unload</code> removes a module from your environment. For example,
After the <code>load</code> sub-command, <code>unload</code> removes a module from your environment. For example,
{{Command|module unload gcc/4.8}}
{{Command|module unload gcc/4.8}}
would remove the GCC 4.8 compilers from your environment.  
would remove the GCC 4.8 compilers from your environment.
</div>


If you have other modules loaded that depend on this compiler, Lmod will tell you that they have been disabled.
If you have other modules loaded that depend on this compiler, Lmod will tell you that they have been disabled.
Line 59: Line 82:
The sub-command <code>purge</code> allows you to remove all the modules you have loaded in one go.
The sub-command <code>purge</code> allows you to remove all the modules you have loaded in one go.


<div class="mw-translate-fuzzy">
Some modules may be marked "sticky" (permanent) by system administrators. These will not be unloaded.
Some modules may be marked "sticky" (permanent) by system administrators. These will not be unloaded.
</div>


== Sub-commands <code>show, help</code> and <code>whatis</code> ==
== Sub-commands <code>show, help</code> and <code>whatis</code> ==
The sub-commands <code>show, help</code> and <code>whatis</code> provide additional information about a given module. The <code>show</code> sub-command displays the entire module, <code>help</code> displays a help message, and <code>whatis</code> shows a description of the module.
The sub-commands <code>show, help</code> and <code>whatis</code> provide additional information about a given module. The <code>show</code> sub-command displays the entire module, <code>help</code> displays a help message, and <code>whatis</code> shows a description of the module.


<div class="mw-translate-fuzzy">
== Sub-command <code>apropos</code> or <code>keyword</code> ==
== Sub-command <code>apropos</code> or <code>keyword</code> ==
The sub-commands <code>apropos</code> or <code>keyword</code> allow you to search for a keyword in all modules. If you don't know which module is appropriate for your calculation, you can search the description.
The sub-commands <code>apropos</code> or <code>keyword</code> allow you to search for a keyword in all modules. If you don't know which module is appropriate for your calculation, you can search the description.
</div>


<div class="mw-translate-fuzzy">
= Loading modules automatically =
= Loading modules automatically =
'''We advise against loading modules automatically in your .bashrc.''' Instead we recommend that you load modules only when required, for example in your job scripts. To facilitate the repeated loading of a large number of modules we recommend you use a module collection.  
'''We advise against loading modules automatically in your .bashrc.''' Instead we recommend that you load modules only when required, for example in your job scripts. To facilitate the repeated loading of a large number of modules we recommend you use a module collection.
</div>


<div class="mw-translate-fuzzy">
= Module collections (Lmod only) =
= Module collections (Lmod only) =
Lmod allows you to create a collection of modules. To do so, first load the desired modules. For example:
Lmod allows you to create a collection of modules. To do so, first load the desired modules. For example:
{{Command|module load gcc/4.8 openmpi/1.8 mkl}}
{{Command|module load gcc/4.8 openmpi/1.8 mkl}}
</div>


<div class="mw-translate-fuzzy">
Then use the <code>save</code> sub-command to save this collection:
Then use the <code>save</code> sub-command to save this collection:
{{Command|module save my_modules}}
{{Command|module save my_modules}}
The <code>my_modules</code> argument is a name you give the collection.  
The <code>my_modules</code> argument is a name you give the collection.
</div>


Then in a later session or in a job you can restore the collection with the command
Then in a later session or in a job you can restore the collection with the command
Line 84: Line 117:
Some modules are hidden. You may ignore them. They are typically modules that you don't have to load manually. They are loaded automatically based on other modules.  
Some modules are hidden. You may ignore them. They are typically modules that you don't have to load manually. They are loaded automatically based on other modules.  


<div class="mw-translate-fuzzy">
= Lmod vs Environment Modules =
= Lmod vs Environment Modules =
The main differences between the environment available to you on the new Compute Canada servers and the servers you have used in the past are as follows.  
The main differences between the environment available to you on the new Compute Canada servers and the servers you have used in the past are as follows.
</div>


<div class="mw-translate-fuzzy">
== Module hierarchy ==
== Module hierarchy ==
Most systems use a flat module structure: All modules are at the same level. This becomes problematic when many combinations of versions of different modules are available on a system. For example, if you need to use the [[FFTW]] library and the module <code>fftw</code> is available in several versions, including a version compiled with GCC 4.8 and Open MPI 1.6, you might see modules named <code>openmpi/1.6_gcc4.8</code> and <code>fftw/3.3_gcc4.8_openmpi1.6</code>. This is neither elegant nor practical. To solve this problem we use a hierarchy of modules. Rather than using the command
Most systems use a flat module structure: All modules are at the same level. This becomes problematic when many combinations of versions of different modules are available on a system. For example, if you need to use the [[FFTW]] library and the module <code>fftw</code> is available in several versions, including a version compiled with GCC 4.8 and Open MPI 1.6, you might see modules named <code>openmpi/1.6_gcc4.8</code> and <code>fftw/3.3_gcc4.8_openmpi1.6</code>. This is neither elegant nor practical. To solve this problem we use a hierarchy of modules. Rather than using the command
Line 92: Line 128:
you instead use
you instead use
{{Command|module load gcc/4.8 openmpi/1.6 fftw/3.3}}
{{Command|module load gcc/4.8 openmpi/1.6 fftw/3.3}}
This is made possible by using a module hierarchy. The <code>fftw/3.3</code> module that is loaded will not be the same one that would be loaded if you had previously loaded the Intel compilers instead of GCC.  
This is made possible by using a module hierarchy. The <code>fftw/3.3</code> module that is loaded will not be the same one that would be loaded if you had previously loaded the Intel compilers instead of GCC.
</div>


<div class="mw-translate-fuzzy">
The inconvenience of using a module hierarchy is that, since modules can have the same name, only the modules that are compatible with the "parent" modules are displayed by the <code>module avail</code> command. Loading a parent module is therefore a prerequisite to loading some modules. To get complete information, Lmod provides the <code>module spider</code> command. It scans the entire hierarchy and displays all the modules. By specifying a module and a particular version, it is then possible to see which paths in the hierarchy enable the desired module to be loaded.
The inconvenience of using a module hierarchy is that, since modules can have the same name, only the modules that are compatible with the "parent" modules are displayed by the <code>module avail</code> command. Loading a parent module is therefore a prerequisite to loading some modules. To get complete information, Lmod provides the <code>module spider</code> command. It scans the entire hierarchy and displays all the modules. By specifying a module and a particular version, it is then possible to see which paths in the hierarchy enable the desired module to be loaded.
</div>


<div class="mw-translate-fuzzy">
== Module collections ==
== Module collections ==
Module collections are an additional functionality provided by Lmod. See [[#Module collections (Lmod only)|this section]] for more details.
Module collections are an additional functionality provided by Lmod. See [[#Module collections (Lmod only)|this section]] for more details.
</div>


== Only one version loaded at a time ==
== Only one version loaded at a time ==
Lmod will refuse to load two versions of the same module at the same time. For example, you cannot have versions 4.8 and 5.4 of the GCC compilers loaded at once.
Lmod will refuse to load two versions of the same module at the same time. For example, you cannot have versions 4.8 and 5.4 of the GCC compilers loaded at once.


<div class="mw-translate-fuzzy">
== Only one module in the same family loaded at a time ==
== Only one module in the same family loaded at a time ==
It is possible for administrators to specify that two modules with different names are of the same family. Lmod will refuse to load two modules of the same family. Typical examples are compiler modules (gcc, intel), MPI modules (openmpi, mvapich2), or BLAS library modules (mkl, openblas).  
It is possible for administrators to specify that two modules with different names are of the same family. Lmod will refuse to load two modules of the same family. Typical examples are compiler modules (gcc, intel), MPI modules (openmpi, mvapich2), or BLAS library modules (mkl, openblas).
</div>


<div class="mw-translate-fuzzy">
== Automatic replacement of modules ==
== Automatic replacement of modules ==
When Lmod detects two modules of the same family, or two version of the same module, the command <code>module load</code> will automatically replace the original module with the one to be loaded. In the cases where the replaced module is a node in the module hierarchy, dependent modules will be reloaded if there are compatible versions, or deactivated otherwise.  
When Lmod detects two modules of the same family, or two version of the same module, the command <code>module load</code> will automatically replace the original module with the one to be loaded. In the cases where the replaced module is a node in the module hierarchy, dependent modules will be reloaded if there are compatible versions, or deactivated otherwise.
</div>


<div class="mw-translate-fuzzy">
== Permanent modules ==
== Permanent modules ==
Lmod allows administrators to define a module as permanent or "sticky". Such a module will not be removed with the <code>module purge</code> command.  
Lmod allows administrators to define a module as permanent or "sticky". Such a module will not be removed with the <code>module purge</code> command.
</div>


= Creating modules =  
= Creating modules =  
For instructions about creating modules, please refer to the [http://lmod.readthedocs.io/en/latest/015_writing_modules.html official documentation].  
For instructions about creating modules, please refer to the [http://lmod.readthedocs.io/en/latest/015_writing_modules.html official documentation].  


<div class="mw-translate-fuzzy">
= Using modules with ZSH or KSH =
= Using modules with ZSH or KSH =
If you wish to use modules with the ZSH or KSH shell, execute the appropriate following command:
If you wish to use modules with the ZSH or KSH shell, execute the appropriate following command:
{{Command|source $LMOD_PKG/init/zsh}}
{{Command|source $LMOD_PKG/init/zsh}}
{{Command|source $LMOD_PKG/init/ksh}}
{{Command|source $LMOD_PKG/init/ksh}}
</div>
35,879

edits

Navigation menu