https://docs.alliancecan.ca/mediawiki/api.php?action=feedcontributions&user=Kerrache&feedformat=atomAlliance Doc - User contributions [en]2024-03-28T10:36:39ZUser contributionsMediaWiki 1.39.6https://docs.alliancecan.ca/mediawiki/index.php?title=MRCC&diff=151094MRCC2024-03-18T16:54:13Z<p>Kerrache: </p>
<hr />
<div>{{draft}}<br />
<br />
==Introduction==<br />
<br />
==License limitations==<br />
<br />
==Module==<br />
<br />
==Examples and job scripts==</div>Kerrachehttps://docs.alliancecan.ca/mediawiki/index.php?title=MRCC&diff=151093MRCC2024-03-18T16:53:49Z<p>Kerrache: </p>
<hr />
<div>==Introduction==<br />
<br />
==License limitations==<br />
<br />
==Module==<br />
<br />
==Examples and job scripts==</div>Kerrachehttps://docs.alliancecan.ca/mediawiki/index.php?title=MRCC&diff=151092MRCC2024-03-18T16:51:40Z<p>Kerrache: MRCC page</p>
<hr />
<div>MRCC<br />
License<br />
Example</div>Kerrachehttps://docs.alliancecan.ca/mediawiki/index.php?title=DL_POLY&diff=150956DL POLY2024-03-07T21:29:41Z<p>Kerrache: </p>
<hr />
<div><languages /><br />
<translate><br />
<br />
= General = <!--T:1--><br />
<br />
<!--T:2--><br />
* Project web site: [https://www.scd.stfc.ac.uk/Pages/DL_POLY.aspx DL_POLY4]<br />
* Documentation: [ftp://ftp.dl.ac.uk/ccp5/DL_POLY/DL_POLY_4.0/DOCUMENTS/USRMAN4.pdf PDF manual].<br />
* [https://www.jiscmail.ac.uk/cgi-bin/webadmin?A0=DLPOLY Mailing list.]<br />
<br />
<!--T:3--><br />
DL_POLY is a general purpose classical molecular dynamics (MD) simulation software. It provides scalable performance from a single processor workstation to a high performance parallel computer. DL_POLY_4 offers fully parallel I/O as well as a NetCDF alternative to the default ASCII trajectory file.<br />
<br />
<!--T:4--><br />
For more on the capabilities, design, and history of DL_POLY, please see the [https://www.scd.stfc.ac.uk/Pages/DL_POLY.aspx project's web site].<br />
<br />
= License limitations = <!--T:5--><br />
<br />
<!--T:6--><br />
'''DL_POLY''' is now [https://gitlab.com/DL%20POLY%20Classic/dl%20poly open source] and it does not require registration. A new module '''dl_poly4/5.1.0''' is already installed under '''StdEnv/2023''' and it is accessible for all users. However, if you would like to use the previous versions ('''dl_poly4/4.10.0''' and/or '''dl_poly4/4.08'''), you should contact [[Technical_support | support]] and ask to be added to the POSIX group that controls access to DL_POLY4. There is no need to register on DL_POLY website.<br />
<br />
= Modules = <!--T:8--><br />
To see which versions of DL_POLY are installed on our systems, run <code>module spider dl_poly4</code>. See [[Using modules]] for more about <code>module</code> subcommands.<br />
<br />
<!--T:9--><br />
To load the version '''5.x''', use:<br />
<br />
<code>module load StdEnv/2023 intel/2023.2.1 openmpi/4.1.5 dl_poly4/5.1.0</code><br />
<br />
<!--T:10--><br />
To load the previous version 4.10.0, use: <br />
<br />
<code>module load StdEnv/2023 intel/2020.1.217 openmpi/4.0.3 dl_poly4/4.10.0</code><br />
<br />
Note that this version requires to be added to a POSIX group as explained in the section [[License limitations]] <br />
<br />
<!--T:11--><br />
We do not currently provide a module for the Java GUI interface.<br />
<br />
= Getting started = <!--T:12--><br />
<br />
<!--T:13--><br />
* [ftp://ftp.dl.ac.uk/ccp5/DL_POLY/DL_POLY_4.0/TUTORIAL/ DL_POLY tutorial and exercises]<br />
* [ftp://ftp.dl.ac.uk/ccp5/DL_POLY/DL_POLY_4.0/DATA/ DL_POLY4 examples]<br />
* [https://www.scd.stfc.ac.uk/Pages/DL_POLY-FAQs.aspx DL_POLY FAQs]<br />
* [https://www.scd.stfc.ac.uk/Pages/DL_Software-Code-of-Conduct.aspx DL_Software User Community]<br />
* [https://www.scd.stfc.ac.uk/Pages/DL_POLY-Useful-Resources.aspx Useful Resources for DL_POLY Users]<br />
<br />
= Scripts and examples = <!--T:14--><br />
<br />
<!--T:15--><br />
The input files shown below (CONTROL and FIELD) were taken from example TEST01 that can be downloaded from the page of [ftp://ftp.dl.ac.uk/ccp5/DL_POLY/DL_POLY_4.0/DATA/ DL_POLY examples].<br />
<br />
<!--T:16--><br />
To start a simulation, one must have at least three files:<br />
<br />
<!--T:17--><br />
* '''CONFIG''': simulation box (atomic coordinates)<br />
* '''FIELD''': force field parameters<br />
* '''CONTROL''': simulation parameters (time step, number of MD steps, simulation ensemble, ...etc.)<br />
<br />
<br />
<!--T:20--><br />
<tabs><br />
<tab name="CONTROL"><br />
{{File<br />
|name=CONTROL<br />
|lang="txt"<br />
|contents=<br />
SODIUM CHLORIDE WITH (27000 IONS)<br />
<br />
<!--T:21--><br />
restart scale<br />
temperature 500.0<br />
equilibration steps 20<br />
steps 20<br />
timestep 0.001<br />
<br />
<!--T:22--><br />
cutoff 12.0<br />
rvdw 12.0<br />
ewald precision 1d-6 <br />
<br />
<!--T:23--><br />
ensemble nvt berendsen 0.01<br />
<br />
<!--T:24--><br />
print every 2<br />
stats every 2<br />
collect<br />
job time 100<br />
close time 10<br />
<br />
<!--T:25--><br />
finish<br />
}}<br />
</tab><br />
<tab name="FIELD"><br />
{{File<br />
|name=FIELD<br />
|lang="txt"<br />
|contents=<br />
SODIUM CHLORIDE WITH EWALD SUM (27000 IONS)<br />
units internal<br />
molecular types 1<br />
SODIUM CHLORIDE<br />
nummols 27<br />
atoms 1000<br />
Na+ 22.9898 1.0 500<br />
Cl- 35.453 -1.0 500<br />
finish<br />
vdw 3 <br />
Na+ Na+ bhm 2544.35 3.1545 2.3400 1.0117e+4 4.8177e+3<br />
Na+ Cl- bhm 2035.48 3.1545 2.7550 6.7448e+4 8.3708e+4<br />
Cl- Cl- bhm 1526.61 3.1545 3.1700 6.9857e+5 1.4032e+6<br />
close<br />
}}<br />
</tab><br />
<tab name="Serial job"><br />
{{File<br />
|name=run_serial_dlp.sh<br />
|lang="bash"<br />
|contents=<br />
#!/bin/bash<br />
<br />
<!--T:26--><br />
#SBATCH --account=def-someuser<br />
#SBATCH --ntasks=1<br />
#SBATCH --mem-per-cpu=2500M # memory; default unit is megabytes.<br />
#SBATCH --time=0-00:30 # time (DD-HH:MM).<br />
<br />
<!--T:27--><br />
# Load the module:<br />
<br />
<!--T:28--><br />
module load intel/2020.1.217 openmpi/4.0.3 dl_poly4/4.10.0<br />
<br />
<!--T:29--><br />
echo "Starting run at: `date`"<br />
<br />
<!--T:30--><br />
dlp_exec=DLPOLY.Z<br />
<br />
<!--T:31--><br />
${dlp_exec}<br />
<br />
<!--T:32--><br />
echo "Program finished with exit code $? at: `date`"<br />
}}<br />
</tab><br />
<tab name="MPI job"><br />
{{File<br />
|name=run_mpi_dlp.sh<br />
|lang="bash"<br />
|contents=<br />
#!/bin/bash<br />
<br />
<!--T:33--><br />
#SBATCH --account=def-someuser<br />
#SBATCH --nodes=1<br />
#SBATCH --ntasks-per-node=4<br />
#SBATCH --mem-per-cpu=2500M # memory; default unit is megabytes.<br />
#SBATCH --time=0-00:30 # time (DD-HH:MM).<br />
<br />
<!--T:34--><br />
# Load the module:<br />
<br />
<!--T:35--><br />
module load intel/2020.1.217 openmpi/4.0.3 dl_poly4/4.10.0<br />
<br />
<!--T:36--><br />
echo "Starting run at: `date`"<br />
<br />
<!--T:37--><br />
dlp_exec=DLPOLY.Z<br />
<br />
<!--T:38--><br />
srun ${dlp_exec}<br />
<br />
<!--T:39--><br />
echo "Program finished with exit code $? at: `date`"<br />
}}<br />
</tab><br />
</tabs><br />
<br />
<br />
= Related software = <!--T:18--><br />
<br />
<!--T:19--><br />
* [[VMD]]<br />
* [[LAMMPS]]<br />
<br />
</translate></div>Kerrachehttps://docs.alliancecan.ca/mediawiki/index.php?title=DL_POLY&diff=150955DL POLY2024-03-07T21:29:00Z<p>Kerrache: </p>
<hr />
<div><languages /><br />
<translate><br />
<br />
= General = <!--T:1--><br />
<br />
<!--T:2--><br />
* Project web site: [https://www.scd.stfc.ac.uk/Pages/DL_POLY.aspx DL_POLY4]<br />
* Documentation: [ftp://ftp.dl.ac.uk/ccp5/DL_POLY/DL_POLY_4.0/DOCUMENTS/USRMAN4.pdf PDF manual].<br />
* [https://www.jiscmail.ac.uk/cgi-bin/webadmin?A0=DLPOLY Mailing list.]<br />
<br />
<!--T:3--><br />
DL_POLY is a general purpose classical molecular dynamics (MD) simulation software. It provides scalable performance from a single processor workstation to a high performance parallel computer. DL_POLY_4 offers fully parallel I/O as well as a NetCDF alternative to the default ASCII trajectory file.<br />
<br />
<!--T:4--><br />
For more on the capabilities, design, and history of DL_POLY, please see the [https://www.scd.stfc.ac.uk/Pages/DL_POLY.aspx project's web site].<br />
<br />
= License limitations = <!--T:5--><br />
<br />
<!--T:6--><br />
'''DL_POLY''' is now [https://gitlab.com/DL%20POLY%20Classic/dl%20poly open source] and it does not require registration. A new module '''dl_poly4/5.1.0''' is already installed under '''StdEnv/2023''' and it is accessible for all users. However, if you would like to use the previous versions ('''dl_poly4/4.10.0''' and/or '''dl_poly4/4.08'''), you should contact [[Technical_support | support]] and ask to be added to the POSIX group that controls access to DL_POLY4. There is no need to register on DL_POLY website.<br />
<br />
= Modules = <!--T:8--><br />
To see which versions of DL_POLY are installed on our systems, run <code>module spider dl_poly4</code>. See [[Using modules]] for more about <code>module</code> subcommands.<br />
<br />
<!--T:9--><br />
To load the version '''5.x''', use:<br />
<br />
<code>module load StdEnv/2023 intel/2023.2.1 openmpi/4.1.5 dl_poly4/5.1.0</code><br />
<br />
<!--T:10--><br />
To load the previous version 4.10.0, use: <br />
<br />
<code>module load intel/2020.1.217 openmpi/4.0.3 dl_poly4/4.10.0</code><br />
<br />
Note that this version requires to be added to a POSIX group as explained in the section [[License limitations]] <br />
<br />
<!--T:11--><br />
We do not currently provide a module for the Java GUI interface.<br />
<br />
= Getting started = <!--T:12--><br />
<br />
<!--T:13--><br />
* [ftp://ftp.dl.ac.uk/ccp5/DL_POLY/DL_POLY_4.0/TUTORIAL/ DL_POLY tutorial and exercises]<br />
* [ftp://ftp.dl.ac.uk/ccp5/DL_POLY/DL_POLY_4.0/DATA/ DL_POLY4 examples]<br />
* [https://www.scd.stfc.ac.uk/Pages/DL_POLY-FAQs.aspx DL_POLY FAQs]<br />
* [https://www.scd.stfc.ac.uk/Pages/DL_Software-Code-of-Conduct.aspx DL_Software User Community]<br />
* [https://www.scd.stfc.ac.uk/Pages/DL_POLY-Useful-Resources.aspx Useful Resources for DL_POLY Users]<br />
<br />
= Scripts and examples = <!--T:14--><br />
<br />
<!--T:15--><br />
The input files shown below (CONTROL and FIELD) were taken from example TEST01 that can be downloaded from the page of [ftp://ftp.dl.ac.uk/ccp5/DL_POLY/DL_POLY_4.0/DATA/ DL_POLY examples].<br />
<br />
<!--T:16--><br />
To start a simulation, one must have at least three files:<br />
<br />
<!--T:17--><br />
* '''CONFIG''': simulation box (atomic coordinates)<br />
* '''FIELD''': force field parameters<br />
* '''CONTROL''': simulation parameters (time step, number of MD steps, simulation ensemble, ...etc.)<br />
<br />
<br />
<!--T:20--><br />
<tabs><br />
<tab name="CONTROL"><br />
{{File<br />
|name=CONTROL<br />
|lang="txt"<br />
|contents=<br />
SODIUM CHLORIDE WITH (27000 IONS)<br />
<br />
<!--T:21--><br />
restart scale<br />
temperature 500.0<br />
equilibration steps 20<br />
steps 20<br />
timestep 0.001<br />
<br />
<!--T:22--><br />
cutoff 12.0<br />
rvdw 12.0<br />
ewald precision 1d-6 <br />
<br />
<!--T:23--><br />
ensemble nvt berendsen 0.01<br />
<br />
<!--T:24--><br />
print every 2<br />
stats every 2<br />
collect<br />
job time 100<br />
close time 10<br />
<br />
<!--T:25--><br />
finish<br />
}}<br />
</tab><br />
<tab name="FIELD"><br />
{{File<br />
|name=FIELD<br />
|lang="txt"<br />
|contents=<br />
SODIUM CHLORIDE WITH EWALD SUM (27000 IONS)<br />
units internal<br />
molecular types 1<br />
SODIUM CHLORIDE<br />
nummols 27<br />
atoms 1000<br />
Na+ 22.9898 1.0 500<br />
Cl- 35.453 -1.0 500<br />
finish<br />
vdw 3 <br />
Na+ Na+ bhm 2544.35 3.1545 2.3400 1.0117e+4 4.8177e+3<br />
Na+ Cl- bhm 2035.48 3.1545 2.7550 6.7448e+4 8.3708e+4<br />
Cl- Cl- bhm 1526.61 3.1545 3.1700 6.9857e+5 1.4032e+6<br />
close<br />
}}<br />
</tab><br />
<tab name="Serial job"><br />
{{File<br />
|name=run_serial_dlp.sh<br />
|lang="bash"<br />
|contents=<br />
#!/bin/bash<br />
<br />
<!--T:26--><br />
#SBATCH --account=def-someuser<br />
#SBATCH --ntasks=1<br />
#SBATCH --mem-per-cpu=2500M # memory; default unit is megabytes.<br />
#SBATCH --time=0-00:30 # time (DD-HH:MM).<br />
<br />
<!--T:27--><br />
# Load the module:<br />
<br />
<!--T:28--><br />
module load intel/2020.1.217 openmpi/4.0.3 dl_poly4/4.10.0<br />
<br />
<!--T:29--><br />
echo "Starting run at: `date`"<br />
<br />
<!--T:30--><br />
dlp_exec=DLPOLY.Z<br />
<br />
<!--T:31--><br />
${dlp_exec}<br />
<br />
<!--T:32--><br />
echo "Program finished with exit code $? at: `date`"<br />
}}<br />
</tab><br />
<tab name="MPI job"><br />
{{File<br />
|name=run_mpi_dlp.sh<br />
|lang="bash"<br />
|contents=<br />
#!/bin/bash<br />
<br />
<!--T:33--><br />
#SBATCH --account=def-someuser<br />
#SBATCH --nodes=1<br />
#SBATCH --ntasks-per-node=4<br />
#SBATCH --mem-per-cpu=2500M # memory; default unit is megabytes.<br />
#SBATCH --time=0-00:30 # time (DD-HH:MM).<br />
<br />
<!--T:34--><br />
# Load the module:<br />
<br />
<!--T:35--><br />
module load intel/2020.1.217 openmpi/4.0.3 dl_poly4/4.10.0<br />
<br />
<!--T:36--><br />
echo "Starting run at: `date`"<br />
<br />
<!--T:37--><br />
dlp_exec=DLPOLY.Z<br />
<br />
<!--T:38--><br />
srun ${dlp_exec}<br />
<br />
<!--T:39--><br />
echo "Program finished with exit code $? at: `date`"<br />
}}<br />
</tab><br />
</tabs><br />
<br />
<br />
= Related software = <!--T:18--><br />
<br />
<!--T:19--><br />
* [[VMD]]<br />
* [[LAMMPS]]<br />
<br />
</translate></div>Kerrachehttps://docs.alliancecan.ca/mediawiki/index.php?title=DL_POLY&diff=150954DL POLY2024-03-07T21:28:39Z<p>Kerrache: </p>
<hr />
<div><languages /><br />
<translate><br />
<br />
= General = <!--T:1--><br />
<br />
<!--T:2--><br />
* Project web site: [https://www.scd.stfc.ac.uk/Pages/DL_POLY.aspx DL_POLY4]<br />
* Documentation: [ftp://ftp.dl.ac.uk/ccp5/DL_POLY/DL_POLY_4.0/DOCUMENTS/USRMAN4.pdf PDF manual].<br />
* [https://www.jiscmail.ac.uk/cgi-bin/webadmin?A0=DLPOLY Mailing list.]<br />
<br />
<!--T:3--><br />
DL_POLY is a general purpose classical molecular dynamics (MD) simulation software. It provides scalable performance from a single processor workstation to a high performance parallel computer. DL_POLY_4 offers fully parallel I/O as well as a NetCDF alternative to the default ASCII trajectory file.<br />
<br />
<!--T:4--><br />
For more on the capabilities, design, and history of DL_POLY, please see the [https://www.scd.stfc.ac.uk/Pages/DL_POLY.aspx project's web site].<br />
<br />
= License limitations = <!--T:5--><br />
<br />
<!--T:6--><br />
'''DL_POLY''' is now [https://gitlab.com/DL%20POLY%20Classic/dl%20poly open source] and it does not require registration. A new module '''dl_poly4/5.1.0''' is already installed under '''StdEnv/2023''' and it is accessible for all users. However, if you would like to use the previous versions ('''dl_poly4/4.10.0''' and/or '''dl_poly4/4.08'''), you should contact [[Technical_support | support]] and ask to be added to the POSIX group that controls access to DL_POLY4. There is no need to register on DL_POLY website.<br />
<br />
= Modules = <!--T:8--><br />
To see which versions of DL_POLY are installed on our systems, run <code>module spider dl_poly4</code>. See [[Using modules]] for more about <code>module</code> subcommands.<br />
<br />
<!--T:9--><br />
To load the version '''5.x''', use:<br />
<br />
<code>module load StdEnv/2023 intel/2023.2.1 openmpi/4.1.5 dl_poly4/5.1.0</code><br />
<br />
<!--T:10--><br />
To load the previous version 4.10.0, use: <br />
<br />
<code>module load intel/2020.1.217 openmpi/4.0.3 dl_poly4/4.10.0</code><br />
<br />
Note that this version requires to be added to a POSIX group as explained in the section [[License limitations]] <br />
<!--T:11--><br />
We do not currently provide a module for the Java GUI interface.<br />
<br />
= Getting started = <!--T:12--><br />
<br />
<!--T:13--><br />
* [ftp://ftp.dl.ac.uk/ccp5/DL_POLY/DL_POLY_4.0/TUTORIAL/ DL_POLY tutorial and exercises]<br />
* [ftp://ftp.dl.ac.uk/ccp5/DL_POLY/DL_POLY_4.0/DATA/ DL_POLY4 examples]<br />
* [https://www.scd.stfc.ac.uk/Pages/DL_POLY-FAQs.aspx DL_POLY FAQs]<br />
* [https://www.scd.stfc.ac.uk/Pages/DL_Software-Code-of-Conduct.aspx DL_Software User Community]<br />
* [https://www.scd.stfc.ac.uk/Pages/DL_POLY-Useful-Resources.aspx Useful Resources for DL_POLY Users]<br />
<br />
= Scripts and examples = <!--T:14--><br />
<br />
<!--T:15--><br />
The input files shown below (CONTROL and FIELD) were taken from example TEST01 that can be downloaded from the page of [ftp://ftp.dl.ac.uk/ccp5/DL_POLY/DL_POLY_4.0/DATA/ DL_POLY examples].<br />
<br />
<!--T:16--><br />
To start a simulation, one must have at least three files:<br />
<br />
<!--T:17--><br />
* '''CONFIG''': simulation box (atomic coordinates)<br />
* '''FIELD''': force field parameters<br />
* '''CONTROL''': simulation parameters (time step, number of MD steps, simulation ensemble, ...etc.)<br />
<br />
<br />
<!--T:20--><br />
<tabs><br />
<tab name="CONTROL"><br />
{{File<br />
|name=CONTROL<br />
|lang="txt"<br />
|contents=<br />
SODIUM CHLORIDE WITH (27000 IONS)<br />
<br />
<!--T:21--><br />
restart scale<br />
temperature 500.0<br />
equilibration steps 20<br />
steps 20<br />
timestep 0.001<br />
<br />
<!--T:22--><br />
cutoff 12.0<br />
rvdw 12.0<br />
ewald precision 1d-6 <br />
<br />
<!--T:23--><br />
ensemble nvt berendsen 0.01<br />
<br />
<!--T:24--><br />
print every 2<br />
stats every 2<br />
collect<br />
job time 100<br />
close time 10<br />
<br />
<!--T:25--><br />
finish<br />
}}<br />
</tab><br />
<tab name="FIELD"><br />
{{File<br />
|name=FIELD<br />
|lang="txt"<br />
|contents=<br />
SODIUM CHLORIDE WITH EWALD SUM (27000 IONS)<br />
units internal<br />
molecular types 1<br />
SODIUM CHLORIDE<br />
nummols 27<br />
atoms 1000<br />
Na+ 22.9898 1.0 500<br />
Cl- 35.453 -1.0 500<br />
finish<br />
vdw 3 <br />
Na+ Na+ bhm 2544.35 3.1545 2.3400 1.0117e+4 4.8177e+3<br />
Na+ Cl- bhm 2035.48 3.1545 2.7550 6.7448e+4 8.3708e+4<br />
Cl- Cl- bhm 1526.61 3.1545 3.1700 6.9857e+5 1.4032e+6<br />
close<br />
}}<br />
</tab><br />
<tab name="Serial job"><br />
{{File<br />
|name=run_serial_dlp.sh<br />
|lang="bash"<br />
|contents=<br />
#!/bin/bash<br />
<br />
<!--T:26--><br />
#SBATCH --account=def-someuser<br />
#SBATCH --ntasks=1<br />
#SBATCH --mem-per-cpu=2500M # memory; default unit is megabytes.<br />
#SBATCH --time=0-00:30 # time (DD-HH:MM).<br />
<br />
<!--T:27--><br />
# Load the module:<br />
<br />
<!--T:28--><br />
module load intel/2020.1.217 openmpi/4.0.3 dl_poly4/4.10.0<br />
<br />
<!--T:29--><br />
echo "Starting run at: `date`"<br />
<br />
<!--T:30--><br />
dlp_exec=DLPOLY.Z<br />
<br />
<!--T:31--><br />
${dlp_exec}<br />
<br />
<!--T:32--><br />
echo "Program finished with exit code $? at: `date`"<br />
}}<br />
</tab><br />
<tab name="MPI job"><br />
{{File<br />
|name=run_mpi_dlp.sh<br />
|lang="bash"<br />
|contents=<br />
#!/bin/bash<br />
<br />
<!--T:33--><br />
#SBATCH --account=def-someuser<br />
#SBATCH --nodes=1<br />
#SBATCH --ntasks-per-node=4<br />
#SBATCH --mem-per-cpu=2500M # memory; default unit is megabytes.<br />
#SBATCH --time=0-00:30 # time (DD-HH:MM).<br />
<br />
<!--T:34--><br />
# Load the module:<br />
<br />
<!--T:35--><br />
module load intel/2020.1.217 openmpi/4.0.3 dl_poly4/4.10.0<br />
<br />
<!--T:36--><br />
echo "Starting run at: `date`"<br />
<br />
<!--T:37--><br />
dlp_exec=DLPOLY.Z<br />
<br />
<!--T:38--><br />
srun ${dlp_exec}<br />
<br />
<!--T:39--><br />
echo "Program finished with exit code $? at: `date`"<br />
}}<br />
</tab><br />
</tabs><br />
<br />
<br />
= Related software = <!--T:18--><br />
<br />
<!--T:19--><br />
* [[VMD]]<br />
* [[LAMMPS]]<br />
<br />
</translate></div>Kerrachehttps://docs.alliancecan.ca/mediawiki/index.php?title=DL_POLY&diff=150950DL POLY2024-03-07T21:27:28Z<p>Kerrache: </p>
<hr />
<div><languages /><br />
<translate><br />
<br />
= General = <!--T:1--><br />
<br />
<!--T:2--><br />
* Project web site: [https://www.scd.stfc.ac.uk/Pages/DL_POLY.aspx DL_POLY4]<br />
* Documentation: [ftp://ftp.dl.ac.uk/ccp5/DL_POLY/DL_POLY_4.0/DOCUMENTS/USRMAN4.pdf PDF manual].<br />
* [https://www.jiscmail.ac.uk/cgi-bin/webadmin?A0=DLPOLY Mailing list.]<br />
<br />
<!--T:3--><br />
DL_POLY is a general purpose classical molecular dynamics (MD) simulation software. It provides scalable performance from a single processor workstation to a high performance parallel computer. DL_POLY_4 offers fully parallel I/O as well as a NetCDF alternative to the default ASCII trajectory file.<br />
<br />
<!--T:4--><br />
For more on the capabilities, design, and history of DL_POLY, please see the [https://www.scd.stfc.ac.uk/Pages/DL_POLY.aspx project's web site].<br />
<br />
= License limitations = <!--T:5--><br />
<br />
<!--T:6--><br />
'''DL_POLY''' is now [https://gitlab.com/DL%20POLY%20Classic/dl%20poly open source] and it does not require registration. A new module '''dl_poly4/5.1.0''' is already installed under '''StdEnv/2023''' and it is accessible for all users. However, if you would like to use the previous versions ('''dl_poly4/4.10.0''' and/or '''dl_poly4/4.08'''), you should contact [[Technical_support | support]] and ask to be added to the POSIX group that controls access to DL_POLY4. There is no need to register on DL_POLY website.<br />
<br />
= Modules = <!--T:8--><br />
To see which versions of DL_POLY are installed on our systems, run <code>module spider dl_poly4</code>. See [[Using modules]] for more about <code>module</code> subcommands.<br />
<br />
<!--T:9--><br />
To load the version '''5.x''', use:<br />
<code>module load StdEnv/2023 intel/2023.2.1 openmpi/4.1.5 dl_poly4/5.1.0</code><br />
<br />
<!--T:10--><br />
To load the previous version 4.10.0, use: <br />
<br />
<code>module load intel/2020.1.217 openmpi/4.0.3 dl_poly4/4.10.0</code><br />
<br />
Note that this version requires to be added to a POSIX group as explained in the section [[License limitations]] <br />
<!--T:11--><br />
We do not currently provide a module for the Java GUI interface.<br />
<br />
= Getting started = <!--T:12--><br />
<br />
<!--T:13--><br />
* [ftp://ftp.dl.ac.uk/ccp5/DL_POLY/DL_POLY_4.0/TUTORIAL/ DL_POLY tutorial and exercises]<br />
* [ftp://ftp.dl.ac.uk/ccp5/DL_POLY/DL_POLY_4.0/DATA/ DL_POLY4 examples]<br />
* [https://www.scd.stfc.ac.uk/Pages/DL_POLY-FAQs.aspx DL_POLY FAQs]<br />
* [https://www.scd.stfc.ac.uk/Pages/DL_Software-Code-of-Conduct.aspx DL_Software User Community]<br />
* [https://www.scd.stfc.ac.uk/Pages/DL_POLY-Useful-Resources.aspx Useful Resources for DL_POLY Users]<br />
<br />
= Scripts and examples = <!--T:14--><br />
<br />
<!--T:15--><br />
The input files shown below (CONTROL and FIELD) were taken from example TEST01 that can be downloaded from the page of [ftp://ftp.dl.ac.uk/ccp5/DL_POLY/DL_POLY_4.0/DATA/ DL_POLY examples].<br />
<br />
<!--T:16--><br />
To start a simulation, one must have at least three files:<br />
<br />
<!--T:17--><br />
* '''CONFIG''': simulation box (atomic coordinates)<br />
* '''FIELD''': force field parameters<br />
* '''CONTROL''': simulation parameters (time step, number of MD steps, simulation ensemble, ...etc.)<br />
<br />
<br />
<!--T:20--><br />
<tabs><br />
<tab name="CONTROL"><br />
{{File<br />
|name=CONTROL<br />
|lang="txt"<br />
|contents=<br />
SODIUM CHLORIDE WITH (27000 IONS)<br />
<br />
<!--T:21--><br />
restart scale<br />
temperature 500.0<br />
equilibration steps 20<br />
steps 20<br />
timestep 0.001<br />
<br />
<!--T:22--><br />
cutoff 12.0<br />
rvdw 12.0<br />
ewald precision 1d-6 <br />
<br />
<!--T:23--><br />
ensemble nvt berendsen 0.01<br />
<br />
<!--T:24--><br />
print every 2<br />
stats every 2<br />
collect<br />
job time 100<br />
close time 10<br />
<br />
<!--T:25--><br />
finish<br />
}}<br />
</tab><br />
<tab name="FIELD"><br />
{{File<br />
|name=FIELD<br />
|lang="txt"<br />
|contents=<br />
SODIUM CHLORIDE WITH EWALD SUM (27000 IONS)<br />
units internal<br />
molecular types 1<br />
SODIUM CHLORIDE<br />
nummols 27<br />
atoms 1000<br />
Na+ 22.9898 1.0 500<br />
Cl- 35.453 -1.0 500<br />
finish<br />
vdw 3 <br />
Na+ Na+ bhm 2544.35 3.1545 2.3400 1.0117e+4 4.8177e+3<br />
Na+ Cl- bhm 2035.48 3.1545 2.7550 6.7448e+4 8.3708e+4<br />
Cl- Cl- bhm 1526.61 3.1545 3.1700 6.9857e+5 1.4032e+6<br />
close<br />
}}<br />
</tab><br />
<tab name="Serial job"><br />
{{File<br />
|name=run_serial_dlp.sh<br />
|lang="bash"<br />
|contents=<br />
#!/bin/bash<br />
<br />
<!--T:26--><br />
#SBATCH --account=def-someuser<br />
#SBATCH --ntasks=1<br />
#SBATCH --mem-per-cpu=2500M # memory; default unit is megabytes.<br />
#SBATCH --time=0-00:30 # time (DD-HH:MM).<br />
<br />
<!--T:27--><br />
# Load the module:<br />
<br />
<!--T:28--><br />
module load intel/2020.1.217 openmpi/4.0.3 dl_poly4/4.10.0<br />
<br />
<!--T:29--><br />
echo "Starting run at: `date`"<br />
<br />
<!--T:30--><br />
dlp_exec=DLPOLY.Z<br />
<br />
<!--T:31--><br />
${dlp_exec}<br />
<br />
<!--T:32--><br />
echo "Program finished with exit code $? at: `date`"<br />
}}<br />
</tab><br />
<tab name="MPI job"><br />
{{File<br />
|name=run_mpi_dlp.sh<br />
|lang="bash"<br />
|contents=<br />
#!/bin/bash<br />
<br />
<!--T:33--><br />
#SBATCH --account=def-someuser<br />
#SBATCH --nodes=1<br />
#SBATCH --ntasks-per-node=4<br />
#SBATCH --mem-per-cpu=2500M # memory; default unit is megabytes.<br />
#SBATCH --time=0-00:30 # time (DD-HH:MM).<br />
<br />
<!--T:34--><br />
# Load the module:<br />
<br />
<!--T:35--><br />
module load intel/2020.1.217 openmpi/4.0.3 dl_poly4/4.10.0<br />
<br />
<!--T:36--><br />
echo "Starting run at: `date`"<br />
<br />
<!--T:37--><br />
dlp_exec=DLPOLY.Z<br />
<br />
<!--T:38--><br />
srun ${dlp_exec}<br />
<br />
<!--T:39--><br />
echo "Program finished with exit code $? at: `date`"<br />
}}<br />
</tab><br />
</tabs><br />
<br />
<br />
= Related software = <!--T:18--><br />
<br />
<!--T:19--><br />
* [[VMD]]<br />
* [[LAMMPS]]<br />
<br />
</translate></div>Kerrachehttps://docs.alliancecan.ca/mediawiki/index.php?title=DL_POLY&diff=150949DL POLY2024-03-07T21:27:00Z<p>Kerrache: </p>
<hr />
<div><languages /><br />
<translate><br />
<br />
= General = <!--T:1--><br />
<br />
<!--T:2--><br />
* Project web site: [https://www.scd.stfc.ac.uk/Pages/DL_POLY.aspx DL_POLY4]<br />
* Documentation: [ftp://ftp.dl.ac.uk/ccp5/DL_POLY/DL_POLY_4.0/DOCUMENTS/USRMAN4.pdf PDF manual].<br />
* [https://www.jiscmail.ac.uk/cgi-bin/webadmin?A0=DLPOLY Mailing list.]<br />
<br />
<!--T:3--><br />
DL_POLY is a general purpose classical molecular dynamics (MD) simulation software. It provides scalable performance from a single processor workstation to a high performance parallel computer. DL_POLY_4 offers fully parallel I/O as well as a NetCDF alternative to the default ASCII trajectory file.<br />
<br />
<!--T:4--><br />
For more on the capabilities, design, and history of DL_POLY, please see the [https://www.scd.stfc.ac.uk/Pages/DL_POLY.aspx project's web site].<br />
<br />
= License limitations = <!--T:5--><br />
<br />
<!--T:6--><br />
'''DL_POLY''' is now [https://gitlab.com/DL%20POLY%20Classic/dl%20poly open source] and it does not require registration. A new module '''dl_poly4/5.1.0''' is already installed under '''StdEnv/2023''' and it is accessible for all users. However, if you would like to use the previous versions ('''dl_poly4/4.10.0''' and/or '''dl_poly4/4.08'''), you should contact [[Technical_support | support]] and ask to be added to the POSIX group that controls access to DL_POLY4. There is no need to register on DL_POLY website.s:<br />
<br />
= Modules = <!--T:8--><br />
To see which versions of DL_POLY are installed on our systems, run <code>module spider dl_poly4</code>. See [[Using modules]] for more about <code>module</code> subcommands.<br />
<br />
<!--T:9--><br />
To load the version '''5.x''', use:<br />
<code>module load StdEnv/2023 intel/2023.2.1 openmpi/4.1.5 dl_poly4/5.1.0</code><br />
<br />
<!--T:10--><br />
To load the previous version 4.10.0, use: <br />
<br />
<code>module load intel/2020.1.217 openmpi/4.0.3 dl_poly4/4.10.0</code><br />
<br />
Note that this version requires to be added to a POSIX group as explained in the section [[License limitations]] <br />
<!--T:11--><br />
We do not currently provide a module for the Java GUI interface.<br />
<br />
= Getting started = <!--T:12--><br />
<br />
<!--T:13--><br />
* [ftp://ftp.dl.ac.uk/ccp5/DL_POLY/DL_POLY_4.0/TUTORIAL/ DL_POLY tutorial and exercises]<br />
* [ftp://ftp.dl.ac.uk/ccp5/DL_POLY/DL_POLY_4.0/DATA/ DL_POLY4 examples]<br />
* [https://www.scd.stfc.ac.uk/Pages/DL_POLY-FAQs.aspx DL_POLY FAQs]<br />
* [https://www.scd.stfc.ac.uk/Pages/DL_Software-Code-of-Conduct.aspx DL_Software User Community]<br />
* [https://www.scd.stfc.ac.uk/Pages/DL_POLY-Useful-Resources.aspx Useful Resources for DL_POLY Users]<br />
<br />
= Scripts and examples = <!--T:14--><br />
<br />
<!--T:15--><br />
The input files shown below (CONTROL and FIELD) were taken from example TEST01 that can be downloaded from the page of [ftp://ftp.dl.ac.uk/ccp5/DL_POLY/DL_POLY_4.0/DATA/ DL_POLY examples].<br />
<br />
<!--T:16--><br />
To start a simulation, one must have at least three files:<br />
<br />
<!--T:17--><br />
* '''CONFIG''': simulation box (atomic coordinates)<br />
* '''FIELD''': force field parameters<br />
* '''CONTROL''': simulation parameters (time step, number of MD steps, simulation ensemble, ...etc.)<br />
<br />
<br />
<!--T:20--><br />
<tabs><br />
<tab name="CONTROL"><br />
{{File<br />
|name=CONTROL<br />
|lang="txt"<br />
|contents=<br />
SODIUM CHLORIDE WITH (27000 IONS)<br />
<br />
<!--T:21--><br />
restart scale<br />
temperature 500.0<br />
equilibration steps 20<br />
steps 20<br />
timestep 0.001<br />
<br />
<!--T:22--><br />
cutoff 12.0<br />
rvdw 12.0<br />
ewald precision 1d-6 <br />
<br />
<!--T:23--><br />
ensemble nvt berendsen 0.01<br />
<br />
<!--T:24--><br />
print every 2<br />
stats every 2<br />
collect<br />
job time 100<br />
close time 10<br />
<br />
<!--T:25--><br />
finish<br />
}}<br />
</tab><br />
<tab name="FIELD"><br />
{{File<br />
|name=FIELD<br />
|lang="txt"<br />
|contents=<br />
SODIUM CHLORIDE WITH EWALD SUM (27000 IONS)<br />
units internal<br />
molecular types 1<br />
SODIUM CHLORIDE<br />
nummols 27<br />
atoms 1000<br />
Na+ 22.9898 1.0 500<br />
Cl- 35.453 -1.0 500<br />
finish<br />
vdw 3 <br />
Na+ Na+ bhm 2544.35 3.1545 2.3400 1.0117e+4 4.8177e+3<br />
Na+ Cl- bhm 2035.48 3.1545 2.7550 6.7448e+4 8.3708e+4<br />
Cl- Cl- bhm 1526.61 3.1545 3.1700 6.9857e+5 1.4032e+6<br />
close<br />
}}<br />
</tab><br />
<tab name="Serial job"><br />
{{File<br />
|name=run_serial_dlp.sh<br />
|lang="bash"<br />
|contents=<br />
#!/bin/bash<br />
<br />
<!--T:26--><br />
#SBATCH --account=def-someuser<br />
#SBATCH --ntasks=1<br />
#SBATCH --mem-per-cpu=2500M # memory; default unit is megabytes.<br />
#SBATCH --time=0-00:30 # time (DD-HH:MM).<br />
<br />
<!--T:27--><br />
# Load the module:<br />
<br />
<!--T:28--><br />
module load intel/2020.1.217 openmpi/4.0.3 dl_poly4/4.10.0<br />
<br />
<!--T:29--><br />
echo "Starting run at: `date`"<br />
<br />
<!--T:30--><br />
dlp_exec=DLPOLY.Z<br />
<br />
<!--T:31--><br />
${dlp_exec}<br />
<br />
<!--T:32--><br />
echo "Program finished with exit code $? at: `date`"<br />
}}<br />
</tab><br />
<tab name="MPI job"><br />
{{File<br />
|name=run_mpi_dlp.sh<br />
|lang="bash"<br />
|contents=<br />
#!/bin/bash<br />
<br />
<!--T:33--><br />
#SBATCH --account=def-someuser<br />
#SBATCH --nodes=1<br />
#SBATCH --ntasks-per-node=4<br />
#SBATCH --mem-per-cpu=2500M # memory; default unit is megabytes.<br />
#SBATCH --time=0-00:30 # time (DD-HH:MM).<br />
<br />
<!--T:34--><br />
# Load the module:<br />
<br />
<!--T:35--><br />
module load intel/2020.1.217 openmpi/4.0.3 dl_poly4/4.10.0<br />
<br />
<!--T:36--><br />
echo "Starting run at: `date`"<br />
<br />
<!--T:37--><br />
dlp_exec=DLPOLY.Z<br />
<br />
<!--T:38--><br />
srun ${dlp_exec}<br />
<br />
<!--T:39--><br />
echo "Program finished with exit code $? at: `date`"<br />
}}<br />
</tab><br />
</tabs><br />
<br />
<br />
= Related software = <!--T:18--><br />
<br />
<!--T:19--><br />
* [[VMD]]<br />
* [[LAMMPS]]<br />
<br />
</translate></div>Kerrachehttps://docs.alliancecan.ca/mediawiki/index.php?title=DL_POLY&diff=150947DL POLY2024-03-07T21:19:34Z<p>Kerrache: </p>
<hr />
<div><languages /><br />
<translate><br />
<br />
= General = <!--T:1--><br />
<br />
<!--T:2--><br />
* Project web site: [https://www.scd.stfc.ac.uk/Pages/DL_POLY.aspx DL_POLY4]<br />
* Documentation: [ftp://ftp.dl.ac.uk/ccp5/DL_POLY/DL_POLY_4.0/DOCUMENTS/USRMAN4.pdf PDF manual].<br />
* [https://www.jiscmail.ac.uk/cgi-bin/webadmin?A0=DLPOLY Mailing list.]<br />
<br />
<!--T:3--><br />
DL_POLY is a general purpose classical molecular dynamics (MD) simulation software. It provides scalable performance from a single processor workstation to a high performance parallel computer. DL_POLY_4 offers fully parallel I/O as well as a NetCDF alternative to the default ASCII trajectory file.<br />
<br />
<!--T:4--><br />
For more on the capabilities, design, and history of DL_POLY, please see the [https://www.scd.stfc.ac.uk/Pages/DL_POLY.aspx project's web site].<br />
<br />
= License limitations = <!--T:5--><br />
<br />
<!--T:6--><br />
'''DL_POLY''' is now [https://gitlab.com/DL%20POLY%20Classic/dl%20poly open source] and it does not require registration. A new module '''dl_poly4/5.1.0''' is already installed under '''StdEnv/2023''' and it is accessible for all users. However, if you would like to use the previous versions ('''dl_poly4/4.10.0''' and/or '''dl_poly4/4.08'''), you should contact [[Technical_support | support]] and ask to be added to the POSIX group that controls access to DL_POLY4. There is no need to register on DL_POLY website.s:<br />
<br />
= Modules = <!--T:8--><br />
To see which versions of DL_POLY are installed on our systems, run <code>module spider dl_poly</code>. See [[Using modules]] for more about <code>module</code> subcommands.<br />
<br />
<!--T:9--><br />
One of the options to load the module is: <br />
<br />
<!--T:10--><br />
<code>module load intel/2020.1.217 openmpi/4.0.3 dl_poly4/4.10.0</code><br />
<br />
<!--T:11--><br />
We do not currently provide a module for the Java GUI interface.<br />
<br />
= Getting started = <!--T:12--><br />
<br />
<!--T:13--><br />
* [ftp://ftp.dl.ac.uk/ccp5/DL_POLY/DL_POLY_4.0/TUTORIAL/ DL_POLY tutorial and exercises]<br />
* [ftp://ftp.dl.ac.uk/ccp5/DL_POLY/DL_POLY_4.0/DATA/ DL_POLY4 examples]<br />
* [https://www.scd.stfc.ac.uk/Pages/DL_POLY-FAQs.aspx DL_POLY FAQs]<br />
* [https://www.scd.stfc.ac.uk/Pages/DL_Software-Code-of-Conduct.aspx DL_Software User Community]<br />
* [https://www.scd.stfc.ac.uk/Pages/DL_POLY-Useful-Resources.aspx Useful Resources for DL_POLY Users]<br />
<br />
= Scripts and examples = <!--T:14--><br />
<br />
<!--T:15--><br />
The input files shown below (CONTROL and FIELD) were taken from example TEST01 that can be downloaded from the page of [ftp://ftp.dl.ac.uk/ccp5/DL_POLY/DL_POLY_4.0/DATA/ DL_POLY examples].<br />
<br />
<!--T:16--><br />
To start a simulation, one must have at least three files:<br />
<br />
<!--T:17--><br />
* '''CONFIG''': simulation box (atomic coordinates)<br />
* '''FIELD''': force field parameters<br />
* '''CONTROL''': simulation parameters (time step, number of MD steps, simulation ensemble, ...etc.)<br />
<br />
<br />
<!--T:20--><br />
<tabs><br />
<tab name="CONTROL"><br />
{{File<br />
|name=CONTROL<br />
|lang="txt"<br />
|contents=<br />
SODIUM CHLORIDE WITH (27000 IONS)<br />
<br />
<!--T:21--><br />
restart scale<br />
temperature 500.0<br />
equilibration steps 20<br />
steps 20<br />
timestep 0.001<br />
<br />
<!--T:22--><br />
cutoff 12.0<br />
rvdw 12.0<br />
ewald precision 1d-6 <br />
<br />
<!--T:23--><br />
ensemble nvt berendsen 0.01<br />
<br />
<!--T:24--><br />
print every 2<br />
stats every 2<br />
collect<br />
job time 100<br />
close time 10<br />
<br />
<!--T:25--><br />
finish<br />
}}<br />
</tab><br />
<tab name="FIELD"><br />
{{File<br />
|name=FIELD<br />
|lang="txt"<br />
|contents=<br />
SODIUM CHLORIDE WITH EWALD SUM (27000 IONS)<br />
units internal<br />
molecular types 1<br />
SODIUM CHLORIDE<br />
nummols 27<br />
atoms 1000<br />
Na+ 22.9898 1.0 500<br />
Cl- 35.453 -1.0 500<br />
finish<br />
vdw 3 <br />
Na+ Na+ bhm 2544.35 3.1545 2.3400 1.0117e+4 4.8177e+3<br />
Na+ Cl- bhm 2035.48 3.1545 2.7550 6.7448e+4 8.3708e+4<br />
Cl- Cl- bhm 1526.61 3.1545 3.1700 6.9857e+5 1.4032e+6<br />
close<br />
}}<br />
</tab><br />
<tab name="Serial job"><br />
{{File<br />
|name=run_serial_dlp.sh<br />
|lang="bash"<br />
|contents=<br />
#!/bin/bash<br />
<br />
<!--T:26--><br />
#SBATCH --account=def-someuser<br />
#SBATCH --ntasks=1<br />
#SBATCH --mem-per-cpu=2500M # memory; default unit is megabytes.<br />
#SBATCH --time=0-00:30 # time (DD-HH:MM).<br />
<br />
<!--T:27--><br />
# Load the module:<br />
<br />
<!--T:28--><br />
module load intel/2020.1.217 openmpi/4.0.3 dl_poly4/4.10.0<br />
<br />
<!--T:29--><br />
echo "Starting run at: `date`"<br />
<br />
<!--T:30--><br />
dlp_exec=DLPOLY.Z<br />
<br />
<!--T:31--><br />
${dlp_exec}<br />
<br />
<!--T:32--><br />
echo "Program finished with exit code $? at: `date`"<br />
}}<br />
</tab><br />
<tab name="MPI job"><br />
{{File<br />
|name=run_mpi_dlp.sh<br />
|lang="bash"<br />
|contents=<br />
#!/bin/bash<br />
<br />
<!--T:33--><br />
#SBATCH --account=def-someuser<br />
#SBATCH --nodes=1<br />
#SBATCH --ntasks-per-node=4<br />
#SBATCH --mem-per-cpu=2500M # memory; default unit is megabytes.<br />
#SBATCH --time=0-00:30 # time (DD-HH:MM).<br />
<br />
<!--T:34--><br />
# Load the module:<br />
<br />
<!--T:35--><br />
module load intel/2020.1.217 openmpi/4.0.3 dl_poly4/4.10.0<br />
<br />
<!--T:36--><br />
echo "Starting run at: `date`"<br />
<br />
<!--T:37--><br />
dlp_exec=DLPOLY.Z<br />
<br />
<!--T:38--><br />
srun ${dlp_exec}<br />
<br />
<!--T:39--><br />
echo "Program finished with exit code $? at: `date`"<br />
}}<br />
</tab><br />
</tabs><br />
<br />
<br />
= Related software = <!--T:18--><br />
<br />
<!--T:19--><br />
* [[VMD]]<br />
* [[LAMMPS]]<br />
<br />
</translate></div>Kerrachehttps://docs.alliancecan.ca/mediawiki/index.php?title=DL_POLY&diff=150946DL POLY2024-03-07T21:19:06Z<p>Kerrache: </p>
<hr />
<div><languages /><br />
<translate><br />
<br />
= General = <!--T:1--><br />
<br />
<!--T:2--><br />
* Project web site: [https://www.scd.stfc.ac.uk/Pages/DL_POLY.aspx DL_POLY4]<br />
* Documentation: [ftp://ftp.dl.ac.uk/ccp5/DL_POLY/DL_POLY_4.0/DOCUMENTS/USRMAN4.pdf PDF manual].<br />
* [https://www.jiscmail.ac.uk/cgi-bin/webadmin?A0=DLPOLY Mailing list.]<br />
<br />
<!--T:3--><br />
DL_POLY is a general purpose classical molecular dynamics (MD) simulation software. It provides scalable performance from a single processor workstation to a high performance parallel computer. DL_POLY_4 offers fully parallel I/O as well as a NetCDF alternative to the default ASCII trajectory file.<br />
<br />
<!--T:4--><br />
For more on the capabilities, design, and history of DL_POLY, please see the [https://www.scd.stfc.ac.uk/Pages/DL_POLY.aspx project's web site].<br />
<br />
= License limitations = <!--T:5--><br />
<br />
<!--T:6--><br />
'''DL_POLY''' is now [https://gitlab.com/DL%20POLY%20Classic/dl%20poly open source] and it does not require registration. A new module '''dl_poly4/5.1.0''' is already installed under '''StdEnv/2023''' and it is accessible for all users. However, if you would like to use the previous versions ('''dl_poly4/4.10.0''' and/or '''dl_poly4/4.08'''), you should contact [[Technical_support | support]] and ask to be added to the POSIX group that controls access to DL_POLY4. There is no need to register on DL_POLY website.s:<br />
<br />
<!--T:7--><br />
::Dear xxxx,<br /><br />
::You have registered for a free copy of the DL_POLY_4 program and thus<br /><br />
::accepted the terms and conditions of the DL_POLY_4 ACADEMIC LICENCE! <br /><br />
::...<br /><br />
<br />
= Modules = <!--T:8--><br />
To see which versions of DL_POLY are installed on our systems, run <code>module spider dl_poly</code>. See [[Using modules]] for more about <code>module</code> subcommands.<br />
<br />
<!--T:9--><br />
One of the options to load the module is: <br />
<br />
<!--T:10--><br />
<code>module load intel/2020.1.217 openmpi/4.0.3 dl_poly4/4.10.0</code><br />
<br />
<!--T:11--><br />
We do not currently provide a module for the Java GUI interface.<br />
<br />
= Getting started = <!--T:12--><br />
<br />
<!--T:13--><br />
* [ftp://ftp.dl.ac.uk/ccp5/DL_POLY/DL_POLY_4.0/TUTORIAL/ DL_POLY tutorial and exercises]<br />
* [ftp://ftp.dl.ac.uk/ccp5/DL_POLY/DL_POLY_4.0/DATA/ DL_POLY4 examples]<br />
* [https://www.scd.stfc.ac.uk/Pages/DL_POLY-FAQs.aspx DL_POLY FAQs]<br />
* [https://www.scd.stfc.ac.uk/Pages/DL_Software-Code-of-Conduct.aspx DL_Software User Community]<br />
* [https://www.scd.stfc.ac.uk/Pages/DL_POLY-Useful-Resources.aspx Useful Resources for DL_POLY Users]<br />
<br />
= Scripts and examples = <!--T:14--><br />
<br />
<!--T:15--><br />
The input files shown below (CONTROL and FIELD) were taken from example TEST01 that can be downloaded from the page of [ftp://ftp.dl.ac.uk/ccp5/DL_POLY/DL_POLY_4.0/DATA/ DL_POLY examples].<br />
<br />
<!--T:16--><br />
To start a simulation, one must have at least three files:<br />
<br />
<!--T:17--><br />
* '''CONFIG''': simulation box (atomic coordinates)<br />
* '''FIELD''': force field parameters<br />
* '''CONTROL''': simulation parameters (time step, number of MD steps, simulation ensemble, ...etc.)<br />
<br />
<br />
<!--T:20--><br />
<tabs><br />
<tab name="CONTROL"><br />
{{File<br />
|name=CONTROL<br />
|lang="txt"<br />
|contents=<br />
SODIUM CHLORIDE WITH (27000 IONS)<br />
<br />
<!--T:21--><br />
restart scale<br />
temperature 500.0<br />
equilibration steps 20<br />
steps 20<br />
timestep 0.001<br />
<br />
<!--T:22--><br />
cutoff 12.0<br />
rvdw 12.0<br />
ewald precision 1d-6 <br />
<br />
<!--T:23--><br />
ensemble nvt berendsen 0.01<br />
<br />
<!--T:24--><br />
print every 2<br />
stats every 2<br />
collect<br />
job time 100<br />
close time 10<br />
<br />
<!--T:25--><br />
finish<br />
}}<br />
</tab><br />
<tab name="FIELD"><br />
{{File<br />
|name=FIELD<br />
|lang="txt"<br />
|contents=<br />
SODIUM CHLORIDE WITH EWALD SUM (27000 IONS)<br />
units internal<br />
molecular types 1<br />
SODIUM CHLORIDE<br />
nummols 27<br />
atoms 1000<br />
Na+ 22.9898 1.0 500<br />
Cl- 35.453 -1.0 500<br />
finish<br />
vdw 3 <br />
Na+ Na+ bhm 2544.35 3.1545 2.3400 1.0117e+4 4.8177e+3<br />
Na+ Cl- bhm 2035.48 3.1545 2.7550 6.7448e+4 8.3708e+4<br />
Cl- Cl- bhm 1526.61 3.1545 3.1700 6.9857e+5 1.4032e+6<br />
close<br />
}}<br />
</tab><br />
<tab name="Serial job"><br />
{{File<br />
|name=run_serial_dlp.sh<br />
|lang="bash"<br />
|contents=<br />
#!/bin/bash<br />
<br />
<!--T:26--><br />
#SBATCH --account=def-someuser<br />
#SBATCH --ntasks=1<br />
#SBATCH --mem-per-cpu=2500M # memory; default unit is megabytes.<br />
#SBATCH --time=0-00:30 # time (DD-HH:MM).<br />
<br />
<!--T:27--><br />
# Load the module:<br />
<br />
<!--T:28--><br />
module load intel/2020.1.217 openmpi/4.0.3 dl_poly4/4.10.0<br />
<br />
<!--T:29--><br />
echo "Starting run at: `date`"<br />
<br />
<!--T:30--><br />
dlp_exec=DLPOLY.Z<br />
<br />
<!--T:31--><br />
${dlp_exec}<br />
<br />
<!--T:32--><br />
echo "Program finished with exit code $? at: `date`"<br />
}}<br />
</tab><br />
<tab name="MPI job"><br />
{{File<br />
|name=run_mpi_dlp.sh<br />
|lang="bash"<br />
|contents=<br />
#!/bin/bash<br />
<br />
<!--T:33--><br />
#SBATCH --account=def-someuser<br />
#SBATCH --nodes=1<br />
#SBATCH --ntasks-per-node=4<br />
#SBATCH --mem-per-cpu=2500M # memory; default unit is megabytes.<br />
#SBATCH --time=0-00:30 # time (DD-HH:MM).<br />
<br />
<!--T:34--><br />
# Load the module:<br />
<br />
<!--T:35--><br />
module load intel/2020.1.217 openmpi/4.0.3 dl_poly4/4.10.0<br />
<br />
<!--T:36--><br />
echo "Starting run at: `date`"<br />
<br />
<!--T:37--><br />
dlp_exec=DLPOLY.Z<br />
<br />
<!--T:38--><br />
srun ${dlp_exec}<br />
<br />
<!--T:39--><br />
echo "Program finished with exit code $? at: `date`"<br />
}}<br />
</tab><br />
</tabs><br />
<br />
<br />
= Related software = <!--T:18--><br />
<br />
<!--T:19--><br />
* [[VMD]]<br />
* [[LAMMPS]]<br />
<br />
</translate></div>Kerrachehttps://docs.alliancecan.ca/mediawiki/index.php?title=GLOST&diff=150067GLOST2024-02-12T20:05:43Z<p>Kerrache: removing nix modules from the page</p>
<hr />
<div><languages /><br />
<translate><br />
= Introduction = <!--T:1--><br />
<br />
<!--T:2--><br />
[https://github.com/cea-hpc/glost GLOST], the Greedy Launcher Of Small Tasks, is a tool for running many sequential jobs of short or variable duration, or for doing parameter sweeps. It works like [[GNU Parallel|GNU parallel]] or [[Job arrays|job arrays]] but with a simpler syntax. <br />
GLOST uses a wrapper called <code>glost_launch</code> and [[MPI]] commands <code>srun</code>, <code>mpiexec</code> and <code>mpirun</code>. Jobs are grouped into one text file, <b>list_glost_tasks.txt</b>, which is used as an argument for <code>glost_launch</code>.<br />
<br />
<!--T:3--><br />
<b>GLOST</b> can be used in the following situations:<br />
<br />
<!--T:4--><br />
* large number of serial jobs with comparative runtime,<br />
* large number of short serial jobs,<br />
* serial jobs with different parameters (parameter sweep).<br />
<br />
<!--T:5--><br />
The idea behind using GLOST consists on bundling serial jobs and run them as an MPI job. It can use multiple cores (one or more nodes). This will reduce considerably the number of the jobs on the queue, and therefore, reduce the stress on the [[Running jobs | scheduler]].<br />
<br />
<!--T:53--><br />
As an alternative, you may also want to consider the [[META: A package for job farming | META]] software package developed by our staff, which has some important advantages over GLOST. In particular, with META the queue wait time may be significantly shorter than with GLOST, and META overheads are smaller (fewer wasted CPU cycles). In addition, META has a convenient mechanism for re-submitting all the computations that never ran or failed. Finally, unlike GLOST, META can be used for all kinds of jobs; serial, multi-threaded, MPI, GPU, or hybrid.<br />
<br />
<!--T:7--><br />
<b>Note:</b> please read this document until the end and if you think that your workflow can fit within this framework, contact [[Technical support]] to help you change your workflow.<br />
<br />
= Advantage of using GLOST = <!--T:52--> <br />
<br />
<!--T:51--><br />
GLOST is used to bundle a set of serial jobs into one single or more MPI jobs depending on the duration of the jobs and their number. <br />
<br />
<!--T:9--><br />
Submitting a large number of serial jobs at once can slow down the scheduler leading in most cases to a slow response and frequent time out from <code>sbatch</code> or <code>squeue</code> requests. The idea is to put all the serial tasks into one single file, for example <b>list_glost_tasks.txt</b>, and submit an MPI job using the <code>glost_launch</code> wrapper. This will reduce considerably the number of the jobs on the queue leading to less requests to the scheduler compared to the situation if the jobs are submitted separately. Using GLOST to submit serial jobs reduces the stress experienced by the Slurm scheduler when a large number of jobs are submitted at the same time without any delay.<br />
<br />
<!--T:10--><br />
Using GLOST, the user will submit and handle few MPI jobs rather than hundreds or thousands serial jobs.<br />
<br />
= Modules = <!--T:11--><br />
<br />
<!--T:12--><br />
GLOST uses OpenMPI to run a set of serial tasks as an MPI job. For each OpenMPI version, a corresponding module of Glost is installed. To use it, make sure to load OpenMPI and Glost modules. For more information, please refer to the page [[using modules]]. To see the current installed modules on our systems, use <code>module spider glost</code>. Before submitting a job, make sure that you can load GLOST along with the other modules that are required to run your application.<br />
</translate><br />
<source lang="bash"><br />
$ module spider glost/0.3.1<br />
<br />
--------------------------------------------------------------------------------------------------------------------------------------<br />
glost: glost/0.3.1<br />
--------------------------------------------------------------------------------------------------------------------------------------<br />
Description:<br />
This is GLOST, the Greedy Launcher Of Small Tasks. <br />
<br />
Properties:<br />
Tools for development / Outils de développement<br />
<br />
You will need to load all module(s) on any one of the lines below before the "glost/0.3.1" module is available to load.<br />
<br />
StdEnv/2020 gcc/9.3.0 openmpi/4.0.3<br />
StdEnv/2020 intel/2020.1.217 openmpi/4.0.3<br />
StdEnv/2023 gcc/12.3 openmpi/4.1.5<br />
StdEnv/2023 intel/2023.2.1 openmpi/4.1.5<br />
<br />
Help:<br />
<br />
Description<br />
===========<br />
This is GLOST, the Greedy Launcher Of Small Tasks.<br />
<br />
<br />
More information<br />
================<br />
- Homepage: https://github.com/cea-hpc/glost<br />
<br />
</source><br />
<translate><br />
<!--T:13--><br />
If there is already an OpenMPI module in your environment, like the default environment, adding <code>module load glost</code> to your list of the modules needed for your application, is sufficient to activate GLOST. Use <code>module list</code> to make sure that GLOST module is loaded along with other modules before submitting your job.<br />
<br />
= How to use GLOST? = <!--T:14--><br />
<br />
== GLOST syntax == <!--T:15--><br />
<br />
<!--T:16--><br />
The general syntax of GLOST can take one of the following forms:<br />
<br />
<!--T:17--><br />
<source lang="bash"><br />
srun glost_launch list_glost_tasks.txt<br />
<br />
<!--T:18--><br />
mpiexec glost_launch list_glost_tasks.txt <br />
<br />
<!--T:19--><br />
mpirun glost_launch list_glost_tasks.txt<br />
</source><br />
<br />
== Number of cores versus number of jobs == <!--T:20--><br />
<br />
<!--T:21--><br />
GLOST uses a cyclic distribution to distribute the serial jobs among the available cores for the job. The GLOST wrapper picks the first lines from the list of jobs and assign one processor to each job (or line from the list) and when one or more processors are done with the first tasks, GLOST will assign them the following lines on the list until the end of the list or until the job runs out of time. Therefore, the number of cores may not necessarily match the number of requested jobs in the list. However, in order to optimize the use of resources, one may need to make sure that the serial jobs have similar runtime and they can be distributed evenly among the cores asked for. Different situations can be treated:<br />
<br />
<!--T:22--><br />
* If you have a large number of very short serial jobs to run (hundreds or thousands of jobs with a very short time, few minutes for example), you submit one or more GLOST jobs that will run a set of serial jobs using few cores. The jobs can be scheduled for short time and by node to take advantage of the back-filling and the scheduler. <br />
* If you have tens to hundreds of relatively short runtime jobs (an hour or so), you can bundle them into one or more GLOST jobs.<br />
* If you have many long serial jobs with similar runtimes, they can also be used as a GLOST job. <br />
<br />
== Estimation of the wall time for GLOST job == <!--T:23--><br />
<br />
<!--T:24--><br />
Before running a GLOST job, try to estimate the runtime for your serial jobs. It can be used to estimate the wall time for your GLOST job.<br />
<br />
Let us suppose you want to run a GLOST job where you have a list of <b>Njobs</b> of similar jobs where each job take <b>t0</b> as a runtime using 1 processor. The total runtime for all these jobs will be: <b>t0*Njobs</b> <br />
<br />
<!--T:25--><br />
Now, if you are going to use <b>Ncores</b> to run your GLOST job, the time required for this job will be: <b>wt = t0*Njobs/Ncores</b>.<br />
<br />
<!--T:26--><br />
<b>Note:</b> An MPI job is often designed so that MPI processes need to exchange information. Designs like this can spend a large fraction of time on communication, and so wind up doing less computation. Many, small, dependent communications can reduce the efficiency of the code. In contrast, GLOST uses MPI but only to start entirely serial jobs, which means that communication overhead is relatively infrequent. You could write the same program yourself, using MPI directly, but GLOST provides nearly the same efficiency, without the effort of writing MPI.<br />
<br />
== Choosing the memory == <!--T:27--><br />
<br />
<!--T:28--><br />
GLOST uses MPI to run serial jobs and the memory per core should be the same as the memory required for the serial job if it runs separately. Use <code>--mem-per-cpu</code> instead of <code>--mem</code> in your Slurm script.<br />
<br />
== Create the list of tasks == <!--T:29--><br />
<br />
<!--T:30--><br />
Before submitting a job using GLOST, create a text file,<b>list_glost_tasks.txt</b>, that contains all the commands needed to run the serial jobs: one job per line. Ideally, one has to choose jobs with similar runtime in order to optimize the use of resources asked for. The GLOST job can run all the tasks in one or multiple directories. If you run all the jobs in one directory, make sure that the output from the different jobs do not overlap or use the same temporary or output files. To do so, standard output may be redirected to a file with a variable indicating the argument or the option used to run the corresponding jobs. In case of the jobs use similar temporary or output files, you may need to create a directory for each task: one directory for each argument or option that correspond to a particular job. <br />
<br />
<!--T:31--><br />
<b>Note:</b> one job may contain one command or multiple commands executed one after another. The commands should be separated by <code>&&</code>.<br />
<br />
<!--T:32--><br />
Here is an example of the file <b>list_glost_example.txt</b> with 8 jobs:<br />
<tabs><br />
<tab name="Script"><br />
{{File<br />
|name=run_glost_test.sh<br />
|lang="bash"<br />
|contents=<br />
#!/bin/bash<br />
<br />
<!--T:55--><br />
#SBATCH --nodes=1<br />
#SBATCH --ntasks-per-node=2<br />
#SBATCH --time=00-02:00<br />
#SBATCH --mem-per-cpu=4000M<br />
<br />
<!--T:56--><br />
# Load GLOST module:<br />
<br />
<!--T:57--><br />
module load intel/2020.1.217 openmpi/4.0.3 glost/0.3.1<br />
<br />
<!--T:58--><br />
echo "Starting run at: `date`"<br />
<br />
<!--T:59--><br />
# Run GLOST with the argument: list_glost_example.txt<br />
<br />
<!--T:60--><br />
srun glost_launch list_glost_example.txt<br />
<br />
<!--T:61--><br />
echo "Program glost_launch finished with exit code $? at: `date`"<br />
}}<br />
</tab><br />
<br />
<!--T:62--><br />
<tab name="List of tasks"><br />
{{File<br />
|name=list_glost_example.txt<br />
|lang="txt"<br />
|contents=<br />
job01 and/or other commands related to job01 <br />
job02 and/or other commands related to job02<br />
job03 and/or other commands related to job03<br />
job04 and/or other commands related to job04<br />
job05 and/or other commands related to job05<br />
job06 and/or other commands related to job06<br />
job07 and/or other commands related to job07<br />
job08 and/or other commands related to job08<br />
}}<br />
</tab><br />
<br />
<!--T:33--><br />
</tabs><br />
<b>Note:</b> the above example cannot be executed. The commands are not defined. It shows only:<br />
<br />
<!--T:34--><br />
* a simple syntax for a list of jobs, <b>list_glost_tasks.txt</b> that will serve as an argument for the <code>glost_launch</code> wrapper;<br />
* a typical script to submit the job.<br />
<br />
<!--T:35--><br />
Both the list of jobs and the script should be adapted to your workflow.<br />
<br />
== List of jobs to run in one directory == <!--T:36--><br />
<br />
<!--T:37--><br />
GLOST can be used to run a set or a list of serial jobs in one directory. To avoid the overlap of the results, one has to make sure that the different jobs will not use the same temporary or output file. This can be achieved by adding arguments to differentiate the different jobs. In the following example, we have a list of 10 tasks. Each task may contain one or more commands. In this example, each job runs three commands one after another:<br />
<br />
<!--T:38--><br />
* <b>First command:</b> Fix a variable <b>nargument</b>. This could be a parameter or a variable to pass to the program for example.<br />
* <b>Second command:</b> run the program. For testing, we have used the command <code>sleep 360</code>. This should be replaced by the command line to run your application. For example: <code>./my_first_prog < first_input_file.txt > first_output_file.txt</code><br />
* <b>Third command:</b> If needed, add one or more commands that will be executed just after the previous ones. All the commands should be separated by <code>&&</code>. For testing, we have used the command: <code>echo ${nargument}.`hostname` > log_${nargument}.txt</code>. For this command, we print out the argument and the <code>hostname</code> to a file log_${nargument}.txt. Similarly to the second command, this line should be replaced by another command line to run an application just after the previous one if needed. For example: <code>./my_second_prog < second_input_file.txt > second_output_file.txt</code>. <br />
<tabs><br />
<tab name="Script"><br />
{{File<br />
|name=run_glost_test.sh<br />
|lang="bash"<br />
|contents=<br />
#!/bin/bash<br />
<br />
<!--T:63--><br />
#SBATCH --nodes=1<br />
#SBATCH --ntasks-per-node=2<br />
#SBATCH --time=00-02:00<br />
#SBATCH --mem-per-cpu=4000M<br />
<br />
<!--T:64--><br />
# Load GLOST module along with the modules required to run your application:<br />
<br />
<!--T:65--><br />
module load intel/2020.1.217 openmpi/4.0.3 glost/0.3.1<br />
<br />
<!--T:66--><br />
echo "Starting run at: `date`"<br />
<br />
<!--T:67--><br />
# Run GLOST with the argument: list_glost_tasks.txt<br />
<br />
<!--T:68--><br />
srun glost_launch list_glost_tasks.txt<br />
<br />
<!--T:69--><br />
echo "Program glost_launch finished with exit code $? at: `date`"<br />
}}<br />
</tab><br />
<br />
<!--T:70--><br />
<tab name="List of tasks"><br />
{{File<br />
|name=list_glost_tasks.txt<br />
|lang="txt"<br />
|contents=<br />
nargument=20 && sleep 360 && echo ${nargument}.`hostname` > log_${nargument}.txt<br />
nargument=21 && sleep 360 && echo ${nargument}.`hostname` > log_${nargument}.txt<br />
nargument=22 && sleep 360 && echo ${nargument}.`hostname` > log_${nargument}.txt<br />
nargument=23 && sleep 360 && echo ${nargument}.`hostname` > log_${nargument}.txt<br />
nargument=24 && sleep 360 && echo ${nargument}.`hostname` > log_${nargument}.txt<br />
nargument=25 && sleep 360 && echo ${nargument}.`hostname` > log_${nargument}.txt<br />
nargument=26 && sleep 360 && echo ${nargument}.`hostname` > log_${nargument}.txt<br />
nargument=27 && sleep 360 && echo ${nargument}.`hostname` > log_${nargument}.txt<br />
nargument=28 && sleep 360 && echo ${nargument}.`hostname` > log_${nargument}.txt<br />
nargument=29 && sleep 360 && echo ${nargument}.`hostname` > log_${nargument}.txt<br />
}}<br />
</tab><br />
<br />
<!--T:71--><br />
</tabs><br />
<br />
<!--T:39--><br />
<b>Note:</b> In the above example, we have used 2 cores and a list of 10 jobs. GLOST will assign the first two jobs (two first lines) to the available processors, and whenever one and/or both of them are done with the first set of jobs, they will continue with the following jobs until the end of the list.<br />
<br />
== List of jobs to run in separate directories == <!--T:40--><br />
<br />
<!--T:41--><br />
Similarly to the previous case, GLOST can be used to run multiple serial jobs where each one is executed in a dedicated directory. This could be useful to run a program that uses files (temporary, input and/or output) with the same names in order to avoid the crash of the jobs or an overlap of the results from the different jobs. To do so, one has to make sure to create the input files and a directory for each job before running GLOST. It can be also achieved if included within the line commands as shown in the following example: <br />
<tabs><br />
<tab name="Script"><br />
{{File<br />
|name=run_glost_test.sh<br />
|lang="bash"<br />
|contents=<br />
#!/bin/bash<br />
<br />
<!--T:72--><br />
#SBATCH --nodes=1<br />
#SBATCH --ntasks-per-node=3<br />
#SBATCH --time=00-03:00<br />
#SBATCH --mem-per-cpu=4000M<br />
<br />
<!--T:73--><br />
# Load GLOST module along with the modules required to run your application:<br />
<br />
<!--T:74--><br />
module load intel/2020.1.217 openmpi/4.0.3 glost/0.3.1<br />
<br />
<!--T:75--><br />
echo "Starting run at: `date`"<br />
<br />
<!--T:76--><br />
# Run GLOST with the argument: list_glost_tasks.txt<br />
<br />
<!--T:77--><br />
srun glost_launch list_glost_tasks.txt<br />
<br />
<!--T:78--><br />
echo "Program glost_launch finished with exit code $? at: `date`"<br />
}}<br />
</tab><br />
<br />
<!--T:79--><br />
<tab name="List of tasks"><br />
{{File<br />
|name=list_glost_tasks.txt<br />
|lang="txt"<br />
|contents=<br />
nargument=20 && mkdir -p RUN_${nargument} && cd RUN_${nargument} && sleep 360 && echo ${nargument}.`hostname` > log_run.txt<br />
nargument=21 && mkdir -p RUN_${nargument} && cd RUN_${nargument} && sleep 360 && echo ${nargument}.`hostname` > log_run.txt<br />
nargument=22 && mkdir -p RUN_${nargument} && cd RUN_${nargument} && sleep 360 && echo ${nargument}.`hostname` > log_run.txt<br />
nargument=23 && mkdir -p RUN_${nargument} && cd RUN_${nargument} && sleep 360 && echo ${nargument}.`hostname` > log_run.txt<br />
nargument=24 && mkdir -p RUN_${nargument} && cd RUN_${nargument} && sleep 360 && echo ${nargument}.`hostname` > log_run.txt<br />
nargument=25 && mkdir -p RUN_${nargument} && cd RUN_${nargument} && sleep 360 && echo ${nargument}.`hostname` > log_run.txt<br />
nargument=26 && mkdir -p RUN_${nargument} && cd RUN_${nargument} && sleep 360 && echo ${nargument}.`hostname` > log_run.txt<br />
nargument=27 && mkdir -p RUN_${nargument} && cd RUN_${nargument} && sleep 360 && echo ${nargument}.`hostname` > log_run.txt<br />
nargument=28 && mkdir -p RUN_${nargument} && cd RUN_${nargument} && sleep 360 && echo ${nargument}.`hostname` > log_run.txt<br />
nargument=29 && mkdir -p RUN_${nargument} && cd RUN_${nargument} && sleep 360 && echo ${nargument}.`hostname` > log_run.txt<br />
nargument=30 && mkdir -p RUN_${nargument} && cd RUN_${nargument} && sleep 360 && echo ${nargument}.`hostname` > log_run.txt<br />
nargument=31 && mkdir -p RUN_${nargument} && cd RUN_${nargument} && sleep 360 && echo ${nargument}.`hostname` > log_run.txt<br />
}}<br />
</tab><br />
<br />
</tabs><br />
== Restarting a GLOST job == <!--T:42--><br />
<br />
<!--T:43--><br />
If you underestimated the wall time for your GLOST job, it may require to be restarted to complete the list of the jobs that were inserted in the list of glost tasks. In this case, make sure to identify the jobs that are already done in order to not run them again. Once identified, remove the corresponding lines from the list of the tasks or create a new list of the jobs that contain the remaining jobs from the previous GLOST job and resubmit your script using the new list as an argument for the <code>glost_launch</code> wrapper.<br />
<br />
== More examples == <!--T:44--><br />
<br />
<!--T:45--><br />
If you are an advanced user and familiar with scripting, you may have a look at the examples by making a copy of the original scripts and adapting them to your workflow.<br />
<br />
<!--T:46--><br />
After loading GLOST module, the examples can be copied to your local directory by running the command:<br />
<br />
<!--T:47--><br />
<source lang="bash"><br />
cp -r $EBROOTGLOST/examples Glost_Examples<br />
</source><br />
<br />
<!--T:48--><br />
The copy of the examples will be saved under the directory: Glost_Examples<br />
<br />
= Related links = <!--T:49--><br />
<br />
<!--T:50--><br />
* [[GNU Parallel]]<br />
* [[Job arrays]]<br />
* [[MPI|MPI jobs]]<br />
* [[Running jobs]]<br />
</translate></div>Kerrachehttps://docs.alliancecan.ca/mediawiki/index.php?title=CPMD&diff=148329CPMD2023-12-13T19:49:49Z<p>Kerrache: </p>
<hr />
<div><languages /><br />
<translate><br />
<!--T:1--><br />
[[Category:Software]][[Category:ComputationalChemistry]][[Category:BiomolecularSimulation]]<br />
<br />
<!--T:13--><br />
[https://www.cpmd.org/wordpress/ CPMD] is a plane wave/pseudo-potential DFT code for ab initio molecular dynamics simulations.<br />
<br />
= License limitations = <!--T:2--><br />
<br />
<!--T:3--><br />
In the past, access to CPMD requires a registration and a confirmation with the developers. Now, it becomes open source and the registration on their website is no longer needed. However, the modules installed on our clusters still protected by a POSIX group. <br />
<br />
Before you can start using [http://cpmd.org CPMD] on our clusters, you have to contact [[Technical_support | send us a support request]] and ask to be added to a POSIX group that will allow you to access the software.<br />
<br />
= Module = <!--T:14--><br />
<br />
<!--T:15--><br />
You can access CPMD by loading a [[Utiliser des modules/en|module]]. <br />
<br />
<!--T:16--><br />
<source lang="bash"><br />
module load intel/2020.1.217 openmpi/4.0.3 cpmd/4.3<br />
</source><br />
<br />
= Local installation of CPMD = <!--T:17--><br />
<br />
<!--T:18--><br />
Recently it has been our experience that a response from CPMD admins can unfortunately take weeks or even months. If you are a registered CPMD user, you have access to the CPMD source files and can therefore build the software yourself in your home directory using our software build environment called EasyBuild, using the exact same recipe that we would use for a central installation.<br />
<br />
<!--T:4--><br />
Below are instructions on how to build CPMD 4.3 under your account on the cluster of your choice:<br />
<br />
<!--T:5--><br />
Create a local directory, first, like so<br />
$ mkdir -p ~/.local/easybuild/sources/c/CPMD<br />
<br />
<!--T:6--><br />
Place all the CPMD source tarballs and patches into that directory<br />
<pre><br />
$ ls -al ~/.local/easybuild/sources/c/CPMD<br />
cpmd2cube.tar.gz<br />
cpmd2xyz-scripts.tar.gz<br />
cpmd-v4.3.tar.gz<br />
fourier.tar.gz<br />
patch.to.4612<br />
patch.to.4615<br />
patch.to.4616<br />
patch.to.4621<br />
patch.to.4624<br />
patch.to.4627<br />
</pre><br />
<br />
<!--T:7--><br />
Then run the EasyBuild command<br />
$ eb CPMD-4.3-iomkl-2020a.eb --rebuild<br />
<br />
<!--T:8--><br />
The <code>--rebuild</code> option forces EasyBuild to ignore CPMD 4.3 installed in a central location and proceed instead with the installation in your home directory.<br />
<br />
<!--T:9--><br />
Once the software is installed, log out and log back in.<br />
<br />
<!--T:10--><br />
Now, when you type <code>module load cpmd</code>, the software installed in your home directory will get picked up.<br />
<br />
<!--T:11--><br />
<pre><br />
$ module load cpmd<br />
$ which cpmd.x<br />
~/.local/easybuild/software/2020/avx2/MPI/intel2020/openmpi4/cpmd/4.3/bin/cpmd.x<br />
</pre><br />
<br />
<!--T:12--><br />
You can use it now as usual in your submission script.<br />
<br />
=Example of job script = <!--T:19--><br />
<br />
<!--T:20--><br />
To run a job, you will need to set an input file and access to the [https://www.cpmd.org/wordpress/index.php/documentation/pseudo-potentials/ pseudo-potentials] available for download from the CPMD website after authentication (only for registered users).<br />
<br />
<!--T:21--><br />
If the input file and the pseudo-potentials are in the same directory, the command to run the program in parallel is:<br />
<br />
<!--T:22--><br />
<code>srun cpmd.x <input files> > <output file></code> (as in the script 1)<br />
<br />
<!--T:23--><br />
It is also possible to put the pseudo-potentials in another directory and run the code as follows:<br />
<br />
<!--T:24--><br />
<code>srun cpmd.x <input files> <path to pseudo potentials location> > <output file></code> (as in the script 2)<br />
<br />
<!--T:25--><br />
<tabs><br />
<tab name="INPUT"><br />
{{File<br />
|name=1-h2-wave.inp<br />
|lang="txt"<br />
|contents=<br />
&INFO<br />
isolated hydrogen molecule.<br />
single point calculation.<br />
&END<br />
<br />
<!--T:26--><br />
&CPMD<br />
OPTIMIZE WAVEFUNCTION<br />
CONVERGENCE ORBITALS<br />
1.0d-7<br />
CENTER MOLECULE ON<br />
PRINT FORCES ON<br />
&END<br />
<br />
&SYSTEM<br />
SYMMETRY<br />
1<br />
ANGSTROM<br />
CELL<br />
8.00 1.0 1.0 0.0 0.0 0.0<br />
CUTOFF<br />
70.0<br />
&END <br />
<br />
<!--T:27--><br />
&DFT<br />
FUNCTIONAL LDA<br />
&END <br />
<br />
<!--T:28--><br />
&ATOMS<br />
*H_MT_LDA.psp<br />
LMAX=S<br />
2<br />
4.371 4.000 4.000<br />
3.629 4.000 4.000<br />
&END <br />
}}<br />
</tab><br />
<br />
<!--T:29--><br />
<tab name="Script 1"><br />
{{File<br />
|name=run-cpmd.sh<br />
|lang="bash"<br />
|contents=<br />
#!/bin/bash<br />
<br />
<!--T:30--><br />
#SBATCH --account=def-someacct<br />
#SBATCH --nodes=1<br />
#SBATCH --ntasks-per-node=4<br />
#SBATCH --mem-per-cpu=2500M<br />
#SBATCH --time=0-1:00<br />
<br />
<!--T:31--><br />
# Load the modules:<br />
<br />
<!--T:32--><br />
module load intel/2020.1.217 openmpi/4.0.3 cpmd/4.3<br />
<br />
<!--T:33--><br />
echo "Starting run at: `date`"<br />
<br />
<!--T:34--><br />
CPMD_INPUT="1-h2-wave.inp"<br />
CPMD_OUTPUT="1-h2-wave_output.txt"<br />
<br />
<!--T:35--><br />
srun cpmd.x ${CPMD_INPUT} > ${CPMD_OUTPUT}<br />
<br />
<!--T:36--><br />
echo "Program finished with exit code $? at: `date`"<br />
}}<br />
</tab><br />
<br />
<!--T:37--><br />
<tab name="Script 2"><br />
{{File<br />
|name=run-cpmd.sh<br />
|lang="bash"<br />
|contents=<br />
#!/bin/bash<br />
<br />
<!--T:38--><br />
#SBATCH --account=def-someacct<br />
#SBATCH --nodes=1<br />
#SBATCH --ntasks-per-node=4<br />
#SBATCH --mem-per-cpu=2500M<br />
#SBATCH --time=0-1:00<br />
<br />
<!--T:39--><br />
# Load the modules:<br />
<br />
<!--T:40--><br />
module load intel/2020.1.217 openmpi/4.0.3 cpmd/4.3<br />
<br />
<!--T:41--><br />
echo "Starting run at: `date`"<br />
<br />
<!--T:42--><br />
CPMD_INPUT="1-h2-wave.inp"<br />
CPMD_OUTPUT="1-h2-wave_output.txt"<br />
PP_PATH=<path to the location of pseudo-potentials><br />
<br />
<!--T:43--><br />
srun cpmd.x ${CPMD_INPUT} ${PP_PATH} > ${CPMD_OUTPUT}<br />
<br />
<!--T:44--><br />
echo "Program finished with exit code $? at: `date`"<br />
}}<br />
</tab><br />
</tabs><br />
<br />
=Related links = <!--T:45--><br />
<br />
<!--T:46--><br />
* CPMD [https://www.cpmd.org/wordpress/ home page].<br />
* CPMD [https://www.cpmd.org/wordpress/index.php/documentation/ manual].<br />
* CPMD [https://www.cpmd.org/wordpress/index.php/documentation/pseudo-potentials/ pseudo-potentials].<br />
* CPMD [https://www.cpmd.org/wordpress/index.php/documentation/faqs/ FAQs].<br />
</translate></div>Kerrachehttps://docs.alliancecan.ca/mediawiki/index.php?title=ORCA&diff=148322ORCA2023-12-13T19:13:09Z<p>Kerrache: NBO with ORCA</p>
<hr />
<div><languages /><br />
[[Category:Software]][[Category:ComputationalChemistry]]<br />
<br />
<translate><br />
<br />
==Introduction== <!--T:1--><br />
ORCA is a flexible, efficient and easy-to-use general-purpose tool for quantum chemistry with specific emphasis on spectroscopic properties of open-shell molecules. It features a wide variety of standard quantum chemical methods ranging from semiempirical methods to DFT to single- and multireference correlated <i>ab initio</i> methods. It can also treat environmental and relativistic effects.<br />
<br />
== Licensing == <!--T:2--><br />
If you wish to use prebuilt ORCA executables:<br />
# You have to register at https://orcaforum.kofo.mpg.de/.<br />
# You will receive a first email to verify the email address and activate the account. Follow the instructions in that email.<br />
# Once the registration is complete, you will get a <b>second email</b> stating that the "<i>registration for ORCA download and usage has been completed</i>".<br />
# [[Technical support | Contact us]] requesting access to ORCA with a copy of the <b>second email</b>.<br />
<br />
== ORCA versions == <!--T:21--><br />
<br />
<!--T:22--><br />
On July 2021, a first version 5 of ORCA was released. This is a major upgrade of ORCA 4.<br />
<br />
=== ORCA 5 === <!--T:23--><br />
<br />
<!--T:24--><br />
Versions 5.0 through 5.0.3 have some bugs that were fixed in version 5.0.4, most notably a [https://orcaforum.kofo.mpg.de/viewtopic.php?f=56&t=9985 bug involving D4 dispersion gradients]. <br />
We therefore recommend that you use the latest version, which is version 5.0.4 as of this writing. <br />
Versions 5.0.1, 5.0.2 and 5.0.3 are in our software stack but might be removed in the future.<br />
<br />
<!--T:25--><br />
To load version 5.0.4, use<br />
<br />
<!--T:26--><br />
<code><br />
module load StdEnv/2020 gcc/10.3.0 openmpi/4.1.1 orca/5.0.4<br />
</code><br />
<br />
=== ORCA 4 === <!--T:28--><br />
<br />
<!--T:29--><br />
The latest released version of ORCA 4 is 4.2.1. Other versions prior to this one are also available in our software stack.<br />
<br />
<!--T:30--><br />
To load version 4.2.1, use<br />
<br />
<!--T:31--><br />
<code><br />
module load StdEnv/2020 gcc/9.3.0 openmpi/4.0.3 orca/4.2.1<br />
</code><br />
<br />
<!--T:32--><br />
or<br />
<br />
<!--T:33--><br />
<code><br />
module load nixpkgs/16.09 gcc/7.3.0 openmpi/3.1.4 orca/4.2.1<br />
</code><br />
<br />
== Setting ORCA input files == <!--T:34--><br />
<br />
<!--T:35--><br />
In addition to the different keywords required to run a given simulation, you should make sure to set two additional parameters:<br />
<br />
<!--T:36--><br />
* number of CPUs<br />
<br />
<!--T:37--><br />
* maxcore<br />
<br />
==Using the software== <!--T:3--><br />
To see which versions of ORCA are currently available, type <code>module spider orca</code>. For detailed information about a specific version, including the other modules that must be loaded first, use the module's full name. For example, <code>module spider orca/4.0.1.2</code>.<br />
<br />
<!--T:11--><br />
See [[Using modules]] for general guidance.<br />
<br />
===Job submission=== <!--T:4--><br />
For a general discussion about submitting jobs, see [[Running jobs]].<br />
<br />
<!--T:12--><br />
'''NOTE''': If you run into MPI errors with some of the ORCA executables, you can try to define the following variables:<br />
<br />
<!--T:13--><br />
export OMPI_MCA_mtl='^mxm'<br />
export OMPI_MCA_pml='^yalla'<br />
<br />
<!--T:5--><br />
The following is a job script to run ORCA using [[MPI]]:<br />
<br />
</translate><br />
{{File<br />
|name=run_orca.sh<br />
|lang="bash"<br />
|contents=<br />
#!/bin/bash<br />
#SBATCH --account=def-youPIs<br />
#SBATCH --ntasks=8 # cpus, the nprocs defined in the input file<br />
#SBATCH --mem-per-cpu=3G # memory per cpu<br />
#SBATCH --time=00-03:00 # time (DD-HH:MM)<br />
#SBATCH --output=benzene.log # output .log file<br />
<br />
module load StdEnv/2020 gcc/9.3.0 openmpi/4.0.3<br />
module load orca/4.2.1<br />
$EBROOTORCA/orca benzene.inp<br />
}}<br />
<translate><br />
<!--T:10--><br />
Example of the input file, benzene.inp:<br />
</translate><br />
{{File<br />
|name=benzene.inp<br />
|lang="text"<br />
|contents=<br />
# Benzene RHF Opt Calculation<br />
%pal nprocs 8 end<br />
! RHF TightSCF PModel<br />
! opt<br />
<br />
* xyz 0 1<br />
C 0.000000000000 1.398696930758 0.000000000000<br />
C 0.000000000000 -1.398696930758 0.000000000000<br />
C 1.211265339156 0.699329968382 0.000000000000<br />
C 1.211265339156 -0.699329968382 0.000000000000<br />
C -1.211265339156 0.699329968382 0.000000000000<br />
C -1.211265339156 -0.699329968382 0.000000000000<br />
H 0.000000000000 2.491406946734 0.000000000000<br />
H 0.000000000000 -2.491406946734 0.000000000000<br />
H 2.157597486829 1.245660462400 0.000000000000<br />
H 2.157597486829 -1.245660462400 0.000000000000<br />
H -2.157597486829 1.245660462400 0.000000000000<br />
H -2.157597486829 -1.245660462400 0.000000000000<br />
*<br />
}}<br />
<translate><br />
<br />
===Notes=== <!--T:15--><br />
* To make sure that the program runs efficiently and makes use of all the resources or the cores asked for in your job script, please add this line <code>%pal nprocs <ncores> end</code> to your input file as shown in the above example. Replace <code><ncores></code> by the number of cores you used in your script.<br />
<br />
===(Sep. 6 2019) Temporary fix to OpenMPI version inconsistency issue=== <!--T:16--><br />
For some type of calculations (DLPNO-STEOM-CCSD in particular), you could receive unknown openmpi related fatal errors. This could be due to using an older version of openmpi (''i.e.'' 3.1.2 as suggested by 'module' for both orca/4.1.0 and 4.2.0) than recommended officially (3.1.3 for orca/4.1.0 and 3.1.4 for orca/4.2.0). To temporarily fix this issue, one can build a custom version of openmpi.<br />
<br />
<!--T:17--><br />
The following two commands prepare a custom openmpi/3.1.4 for orca/4.2.0:<br />
module load gcc/7.3.0<br />
eb OpenMPI-3.1.2-GCC-7.3.0.eb --try-software-version=3.1.4<br />
When the building is finished, you can load the custom openmpi using module<br />
module load openmpi/3.1.4<br />
At this step, one can manually install orca/4.2.0 binaries from the official forum under the home directory after finishing the registration on the official orca forum and being granted access to the orca program on our clusters.<br />
<br />
<br />
<!--T:18--><br />
Additional notes from the contributor:<br />
<br />
<!--T:19--><br />
This is a '''temporary''' fix prior to the official upgrade of openmpi on our clusters. Please remember to delete the manually installed orca binaries once the official openmpi version is up to date.<br />
<br />
<!--T:20--><br />
The compiling command does not seem to apply to openmpi/2.1.x.<br />
<br />
== Using NBO with ORCA ==<br />
<br />
To run NBO with ORCA, one need to have access to NBO. On our clusters, NBO is not available as a separate module. However, it is possible to access it via Gaussian modules that are installed on cedar and graham. Users interested to use NBO with ORCA should have access to ORCA and Gaussian. To get access to Gaussian, please have a look to the conditions for Gaussian [[https://docs.alliancecan.ca/wiki/Gaussian#License_agreement|usage]]. <br />
<br />
=== Script example ===<br />
<br />
The input file should have the keyword '''NBO'''.<br />
<br />
{{File<br />
|name=run_orca-nbo.sh<br />
|lang="bash"<br />
|contents=<br />
#!/bin/bash<br />
#SBATCH --account=def-youPIs<br />
#SBATCH --nodes=1<br />
#SBATCH --ntasks=16<br />
#SBATCH --mem-per-cpu=4000<br />
#SBATCH --time=0-3:00:00<br />
<br />
# Load the modules:<br />
<br />
module load StdEnv/2020 gcc/10.3.0 openmpi/4.1.1 orca/5.0.4<br />
module load gaussian/g16.c01<br />
<br />
export GENEXE=`which gennbo.i4.exe`<br />
export NBOEXE=`which nbo7.i4.exe`<br />
<br />
${EBROOTORCA}/orca orca_input.inp > orca_output.out<br />
<br />
}}<br />
== Related links == <!--T:38--><br />
<br />
<!--T:39--><br />
* [https://www.orcasoftware.de/tutorials_orca/ ORCA tutorials]<br />
* [https://orcaforum.kofo.mpg.de/app.php/portal ORCA Forum]<br />
<br />
</translate></div>Kerrachehttps://docs.alliancecan.ca/mediawiki/index.php?title=GATK&diff=141100GATK2023-07-24T14:36:53Z<p>Kerrache: older version requires to load nixpkgs/16.09</p>
<hr />
<div><languages /><br />
<br />
<translate><br />
<br />
<!--T:1--><br />
The [https://bio.tools/gatk Genome Analysis Toolkit (GATK)] is a set of bioinformatic tools for<br />
analyzing high-throughput sequencing (HTS) and variant call format (VCF)<br />
data. The toolkit is well established for germline short variant<br />
discovery from whole genome and exome sequencing data.<br />
It is a leading tool in variant discovery and [https://gatk.broadinstitute.org/hc/en-us/sections/360007226651-Best-Practices-Workflows best practices]<br />
for genomics research.<br />
<br />
==Availability and module loading == <!--T:2--><br />
We provide several versions of GATK. To access the version information, use<br />
the [https://docs.computecanada.ca/wiki/Utiliser_des_modules/en <code>module</code> command]<br />
<br />
<!--T:3--><br />
{{Commands<br />
|module spider gatk<br />
}}<br />
<br />
<!--T:4--><br />
which gives you some information about GATK and versions<br />
<pre><br />
gatk/3.7<br />
gatk/3.8<br />
gatk/4.0.0.0<br />
gatk/4.0.8.1<br />
gatk/4.0.12.0<br />
gatk/4.1.0.0<br />
gatk/4.1.2.0<br />
gatk/4.1.7.0<br />
gatk/4.1.8.0<br />
gatk/4.1.8.1<br />
gatk/4.2.2.0<br />
gatk/4.2.4.0<br />
gatk/4.2.5.0<br />
</pre><br />
<br />
<!--T:5--><br />
More specific information on any given version can be accessed with<br />
<br />
<!--T:6--><br />
{{Commands<br />
|module spider gatk/4.1.8.1<br />
}}<br />
<br />
<!--T:7--><br />
As you can see, this module only has the <code>StdEnv/2020</code> module as prerequisite<br />
so it can be loaded with<br />
<br />
<!--T:8--><br />
{{Commands<br />
|module load StdEnv/2020 gatk/4.1.8.1<br />
}}<br />
<br />
<!--T:9--><br />
or, given that <code>StdEnv/2020</code> is loaded by default, simply with<br />
<br />
<!--T:10--><br />
{{Commands<br />
|module load gatk/4.1.8.1<br />
}}<br />
<br />
==General usage == <!--T:11--><br />
The later versions of GATK (>=4.0.0.0) provide a wrapper over the Java executables (.jar). Loading the GATK modules will automatically set most of the environmental variables you will need to successfully run GATK.<br />
<br />
<!--T:12--><br />
The <code>module spider</code> command also provides information on usage and some examples of the wrapper:<br />
<pre><br />
Usage<br />
=====<br />
gatk [--java-options "-Xmx4G"] ToolName [GATK args]<br />
<br />
<br />
Examples<br />
========<br />
gatk --java-options "-Xmx8G" HaplotypeCaller -R reference.fasta -I input.bam -O output.vcf<br />
</pre><br />
<br />
<!--T:13--><br />
As you probably notice, there are some arguments to be passed directly to Java through the <code>--java-options</code> such as the maximum heap memory (<code>-Xmx8G</code> in the example, reserving 8 Gb of memory for the virtual machine). We recommend that you <b>always</b> use <code>-DGATK_STACKTRACE_ON_USER_EXCEPTION=true</code> since it will give you more information in case the program fails. This information can help you or us (if you need support) to solve the issue.<br />
Note that all options passed to <code>--java-options</code> have to be within quotation marks.<br />
<br />
=== Considerations regarding our systems === <!--T:50--><br />
<br />
To use GATK on our systems, we recommend that you use the <code>--tmp-dir</code> option and set it to <code>${SLURM_TMPDIR}</code> when in an <code>sbatch</code> job, so that the temporary files are redirected to the local storage.<br />
<br />
<!--T:51--><br />
Also, when using <code>GenomicsDBImport</code>, make sure to have the option <code>--genomicsdb-shared-posixfs-optimizations</code> enabled as it will [https://gatk.broadinstitute.org/hc/en-us/articles/4414594350619-SelectVariants#--genomicsdb-shared-posixfs-optimizations Allow for optimizations to improve the usability and performance for shared Posix Filesystems(e.g. NFS, Lustre)]. If not possible or if you are using GNU parallel to run multiple intervals at the same time, please copy your database to <code>${SLURM_TMPDIR}</code> and run it from there as your IO operations might disrupt the filesystem. <code>${SLURM_TMPDIR}</code> is a local storage and therefore is not only faster, but the IO operations would not affect other users.<br />
<br />
===Earlier versions than GATK 4 === <!--T:14--><br />
Earlier versions of GATK do not have the <code>gatk</code> command. Instead, you have to call the jar file:<br />
<br />
<!--T:15--><br />
<pre><br />
java -jar GenomeAnalysisTK.jar PROGRAM OPTIONS<br />
</pre><br />
<br />
<!--T:16--><br />
However, GenomeAnalysisTK.jar must be in PATH. On our systems, the environmental variables <code>$EBROOTPICARD</code> for Picard (included in GATK >= 4) and <code>$EBROOTGATK</code> for GATK contain the path to the jar file, so the appropriate way to call GATK <= 3 is<br />
<br />
<!--T:17--><br />
<pre><br />
module load nixpkgs/16.09 gatk/3.8<br />
java -jar "${EBROOTGATK}"/GenomeAnalysisTK.jar PROGRAM OPTIONS<br />
</pre><br />
<br />
<!--T:18--><br />
You can find the specific usage of GATK <= 3 in the [https://github.com/broadinstitute/gatk-docs/tree/master/gatk3-tooldocs GATK3 guide].<br />
<br />
===Multicore usage === <!--T:19--><br />
Most GATK (>=4) tools are not multicore by default. This means that you should request only one core when calling them. Some tools use threads in some of the computations (e.g. <code>Mutect2</code> has the <code>--native-pair-hmm-threads</code>) and therefore you can require more CPUs (most of them with up to 4 threads) for these computations. GATK4, however, does provide <b>some</b> Spark commands<ref> https://gatk.broadinstitute.org/hc/en-us/articles/360035890591-</ref><br />
<br />
<!--T:46--><br />
<blockquote><br />
<b>Not all GATK tools use Spark</b><br />
<br />
<!--T:47--><br />
Tools that can use Spark generally have a note to that effect in their respective Tool Doc.<br />
<br />
<!--T:48--><br />
* Some GATK tools exist in distinct Spark-capable and non-Spark-capable versions. The "sparkified" versions have the suffix "Spark" at the end of their names. Many of these are still experimental; down the road we plan to consolidate them so that there will be only one version per tool.<br />
<br />
<!--T:49--><br />
* Some GATK tools only exist in a Spark-capable version. Those tools don't have the "Spark" suffix.<br />
</blockquote><br />
<br />
<!--T:22--><br />
For the commands that do use Spark, you can request multiple CPUs. <b>NOTE:</b> Please provide the exact number of CPUs to the <code>spark</code> command. For example if you requested 10 CPUs, use <code>--spark-master local[10]</code> instead of <code>--spark-master local[*]</code>. If you want to use multiple nodes to scale the Spark cluster, you have to first [[Apache_Spark|deploy a SPARK cluster]] and then set the appropriate variables in the GATK command.<br />
<br />
==Running GATK via Apptainer== <!--T:36--><br />
<br />
<!--T:37--><br />
If you encounter errors like [https://gatk.broadinstitute.org/hc/en-us/community/posts/360067054832-GATK-4-1-7-0-error-java-lang-IllegalArgumentException-malformed-input-off-17635906-length-1 IllegalArgumentException] while using the installed modules on our clusters, we recommend that you try another workflow by using the program via [[Apptainer]].<br />
<br />
<!--T:38--><br />
A Docker image of GATK can be found [https://hub.docker.com/r/broadinstitute/gatk here] and other versions are available on this [https://hub.docker.com/r/broadinstitute/gatk/tags page]. You will need first to build an Apptainer image from the Docker image; <br />
to get the latest version for example, you can run the following commands on the cluster<br />
<br />
<!--T:40--><br />
<pre><br />
module load apptainer<br />
apptainer build gatk.sif docker://broadinstitute/gatk<br />
</pre><br />
<br />
<!--T:41--><br />
or to get a particular [https://hub.docker.com/r/broadinstitute/gatk/tags version]:<br />
<br />
<!--T:42--><br />
<pre><br />
module load apptainer<br />
apptainer build gatk_VERSION.sif docker://broadinstitute/gatk:VERSION<br />
</pre><br />
<br />
<!--T:43--><br />
In your [[Running jobs|SBATCH]] script, you should use something like this:<br />
<br />
<!--T:44--><br />
<pre><br />
module load apptainer<br />
apptainer exec -B /home -B /project -B /scratch -B /localscratch \<br />
<path to the image>/gatk.sif gatk [--java-options "-Xmx4G"] ToolName [GATK args]</pre><br />
<br />
<!--T:45--><br />
For more information about Apptainer, watch the recorded [https://www.youtube.com/watch?v=bpmrfVqBowY Apptainer webinar].<br />
<br />
==Frequently asked questions == <!--T:23--><br />
===How do I add a read group (RG) tag in my bam file? ===<br />
Assuming that you want to add a read group called <i>tag</i> to the file called <i>input.bam</i>, you can use the GATK/PICARD command [https://gatk.broadinstitute.org/hc/en-us/articles/360037226472-AddOrReplaceReadGroups-Picard- AddOrReplaceReadGroups]:<br />
<pre><br />
gatk AddOrReplaceReadGroups \<br />
-I input.bam \<br />
-O output.bam \<br />
--RGLB tag \<br />
--RGPL ILLUMINA <br />
--RGPU tag \<br />
--RGSM tag \<br />
--SORT_ORDER 'coordinate' \<br />
--CREATE_INDEX true<br />
</pre><br />
This assumes that your input file is sorted by coordinates and will generate an index along with the annotated output (<code>--CREATE_INDEX true</code>)<br />
<br />
===How do I deal with <code>java.lang.OutOfMemoryError: Java heap space</code> === <!--T:24--><br />
Subprograms of GATK often require more memory to process your files. If you were not using the <code>-Xms</code> command, add it to the <code>--java-options</code>. For example, let's imagine that you run the following command:<br />
<pre><br />
gatk MarkDuplicates \<br />
-I input.bam \<br />
-O marked_duplicates.bam \<br />
-M marked_dup_metrics.txt <br />
</pre><br />
<br />
<!--T:25--><br />
but it gives you the <code>java.lang.OutOfMemoryError: Java heap space</code> error. Try:<br />
<br />
<!--T:26--><br />
<pre><br />
gatk MarkDuplicates \<br />
--java-options "-Xmx8G DGATK_STACKTRACE_ON_USER_EXCEPTION=true"<br />
-I input.bam \<br />
-O marked_duplicates.bam \<br />
-M marked_dup_metrics.txt <br />
</pre><br />
<br />
<!--T:27--><br />
If it fails again, keep increasing the memory until you find the required memory for your particular dataset. If you are using any of our systems, <b>remember to request enough memory for this</b>.<br />
<br />
<!--T:28--><br />
If you are interested in knowing more about java heap space, you can start [https://plumbr.io/outofmemoryerror/java-heap-space here].<br />
<br />
===Increasing the heap memory does not fix <code>java.lang.OutOfMemoryError: Java heap space</code> === <!--T:29--><br />
There are cases in which the memory issue cannot be fixed with increasing the heap memory. This often happens with non-model organisms, and you are using too many scaffolds in your reference. In this case it is recommended to remove small scaffolds and create subsets of your reference. This implies that you have to map multiple times and run the pipelines in each of the subsets. <b>This approach does not work for all pipelines</b> so review your results carefully. GATK is designed with the human genome in mind, and therefore other organisms will require adjustments in many parameters and pipelines.<br />
<br />
===Using more resources than asked for === <!--T:30--><br />
Sometimes GATK/JAVA applications will use more memory or CPUs/threads than the numbers requested. This is often generated by the JAVA garbage collection. To control this, add <code>-XX:ConcGCThreads=1</code> to the <code>--java-options</code> argument. <br />
<br />
===FAQ on GATK === <!--T:31--><br />
You can find the [https://gatk.broadinstitute.org/hc/en-us/sections/360007226791-Troubleshooting-GATK4-Issues GATK FAQs on their website].<br />
<br />
=References = <!--T:32--><br />
[https://gatk.broadinstitute.org/hc/en-us GATK Home]<br />
<br />
<!--T:33--><br />
[https://gatk.broadinstitute.org/hc/en-us/articles/360035532012-Parallelism-Multithreading-Scatter-Gather GATK SPARK]<br />
<br />
<!--T:34--><br />
[https://gatk.broadinstitute.org/hc/en-us/articles/360035889611-How-can-I-make-GATK-tools-run-faster- Make GATK tools run faster]<br />
<br />
<!--T:35--><br />
[[Category:Bioinformatics]]<br />
[[Category:Software]]<br />
<br />
</translate></div>Kerrachehttps://docs.alliancecan.ca/mediawiki/index.php?title=GATK&diff=141099GATK2023-07-24T14:28:12Z<p>Kerrache: Replacing nix by StdEnv/2020</p>
<hr />
<div><languages /><br />
<br />
<translate><br />
<br />
<!--T:1--><br />
The [https://bio.tools/gatk Genome Analysis Toolkit (GATK)] is a set of bioinformatic tools for<br />
analyzing high-throughput sequencing (HTS) and variant call format (VCF)<br />
data. The toolkit is well established for germline short variant<br />
discovery from whole genome and exome sequencing data.<br />
It is a leading tool in variant discovery and [https://gatk.broadinstitute.org/hc/en-us/sections/360007226651-Best-Practices-Workflows best practices]<br />
for genomics research.<br />
<br />
==Availability and module loading == <!--T:2--><br />
We provide several versions of GATK. To access the version information, use<br />
the [https://docs.computecanada.ca/wiki/Utiliser_des_modules/en <code>module</code> command]<br />
<br />
<!--T:3--><br />
{{Commands<br />
|module spider gatk<br />
}}<br />
<br />
<!--T:4--><br />
which gives you some information about GATK and versions<br />
<pre><br />
gatk/3.7<br />
gatk/3.8<br />
gatk/4.0.0.0<br />
gatk/4.0.8.1<br />
gatk/4.0.12.0<br />
gatk/4.1.0.0<br />
gatk/4.1.2.0<br />
gatk/4.1.7.0<br />
gatk/4.1.8.0<br />
gatk/4.1.8.1<br />
gatk/4.2.2.0<br />
gatk/4.2.4.0<br />
gatk/4.2.5.0<br />
</pre><br />
<br />
<!--T:5--><br />
More specific information on any given version can be accessed with<br />
<br />
<!--T:6--><br />
{{Commands<br />
|module spider gatk/4.1.8.1<br />
}}<br />
<br />
<!--T:7--><br />
As you can see, this module only has the <code>StdEnv/2020</code> module as prerequisite<br />
so it can be loaded with<br />
<br />
<!--T:8--><br />
{{Commands<br />
|module load StdEnv/2020 gatk/4.1.8.1<br />
}}<br />
<br />
<!--T:9--><br />
or, given that <code>StdEnv/2020</code> is loaded by default, simply with<br />
<br />
<!--T:10--><br />
{{Commands<br />
|module load gatk/4.1.8.1<br />
}}<br />
<br />
==General usage == <!--T:11--><br />
The later versions of GATK (>=4.0.0.0) provide a wrapper over the Java executables (.jar). Loading the GATK modules will automatically set most of the environmental variables you will need to successfully run GATK.<br />
<br />
<!--T:12--><br />
The <code>module spider</code> command also provides information on usage and some examples of the wrapper:<br />
<pre><br />
Usage<br />
=====<br />
gatk [--java-options "-Xmx4G"] ToolName [GATK args]<br />
<br />
<br />
Examples<br />
========<br />
gatk --java-options "-Xmx8G" HaplotypeCaller -R reference.fasta -I input.bam -O output.vcf<br />
</pre><br />
<br />
<!--T:13--><br />
As you probably notice, there are some arguments to be passed directly to Java through the <code>--java-options</code> such as the maximum heap memory (<code>-Xmx8G</code> in the example, reserving 8 Gb of memory for the virtual machine). We recommend that you <b>always</b> use <code>-DGATK_STACKTRACE_ON_USER_EXCEPTION=true</code> since it will give you more information in case the program fails. This information can help you or us (if you need support) to solve the issue.<br />
Note that all options passed to <code>--java-options</code> have to be within quotation marks.<br />
<br />
=== Considerations regarding our systems === <!--T:50--><br />
<br />
To use GATK on our systems, we recommend that you use the <code>--tmp-dir</code> option and set it to <code>${SLURM_TMPDIR}</code> when in an <code>sbatch</code> job, so that the temporary files are redirected to the local storage.<br />
<br />
<!--T:51--><br />
Also, when using <code>GenomicsDBImport</code>, make sure to have the option <code>--genomicsdb-shared-posixfs-optimizations</code> enabled as it will [https://gatk.broadinstitute.org/hc/en-us/articles/4414594350619-SelectVariants#--genomicsdb-shared-posixfs-optimizations Allow for optimizations to improve the usability and performance for shared Posix Filesystems(e.g. NFS, Lustre)]. If not possible or if you are using GNU parallel to run multiple intervals at the same time, please copy your database to <code>${SLURM_TMPDIR}</code> and run it from there as your IO operations might disrupt the filesystem. <code>${SLURM_TMPDIR}</code> is a local storage and therefore is not only faster, but the IO operations would not affect other users.<br />
<br />
===Earlier versions than GATK 4 === <!--T:14--><br />
Earlier versions of GATK do not have the <code>gatk</code> command. Instead, you have to call the jar file:<br />
<br />
<!--T:15--><br />
<pre><br />
java -jar GenomeAnalysisTK.jar PROGRAM OPTIONS<br />
</pre><br />
<br />
<!--T:16--><br />
However, GenomeAnalysisTK.jar must be in PATH. On our systems, the environmental variables <code>$EBROOTPICARD</code> for Picard (included in GATK >= 4) and <code>$EBROOTGATK</code> for GATK contain the path to the jar file, so the appropriate way to call GATK <= 3 is<br />
<br />
<!--T:17--><br />
<pre><br />
module load gatk/3.8<br />
java -jar "${EBROOTGATK}"/GenomeAnalysisTK.jar PROGRAM OPTIONS<br />
</pre><br />
<br />
<!--T:18--><br />
You can find the specific usage of GATK <= 3 in the [https://github.com/broadinstitute/gatk-docs/tree/master/gatk3-tooldocs GATK3 guide].<br />
<br />
===Multicore usage === <!--T:19--><br />
Most GATK (>=4) tools are not multicore by default. This means that you should request only one core when calling them. Some tools use threads in some of the computations (e.g. <code>Mutect2</code> has the <code>--native-pair-hmm-threads</code>) and therefore you can require more CPUs (most of them with up to 4 threads) for these computations. GATK4, however, does provide <b>some</b> Spark commands<ref> https://gatk.broadinstitute.org/hc/en-us/articles/360035890591-</ref><br />
<br />
<!--T:46--><br />
<blockquote><br />
<b>Not all GATK tools use Spark</b><br />
<br />
<!--T:47--><br />
Tools that can use Spark generally have a note to that effect in their respective Tool Doc.<br />
<br />
<!--T:48--><br />
* Some GATK tools exist in distinct Spark-capable and non-Spark-capable versions. The "sparkified" versions have the suffix "Spark" at the end of their names. Many of these are still experimental; down the road we plan to consolidate them so that there will be only one version per tool.<br />
<br />
<!--T:49--><br />
* Some GATK tools only exist in a Spark-capable version. Those tools don't have the "Spark" suffix.<br />
</blockquote><br />
<br />
<!--T:22--><br />
For the commands that do use Spark, you can request multiple CPUs. <b>NOTE:</b> Please provide the exact number of CPUs to the <code>spark</code> command. For example if you requested 10 CPUs, use <code>--spark-master local[10]</code> instead of <code>--spark-master local[*]</code>. If you want to use multiple nodes to scale the Spark cluster, you have to first [[Apache_Spark|deploy a SPARK cluster]] and then set the appropriate variables in the GATK command.<br />
<br />
==Running GATK via Apptainer== <!--T:36--><br />
<br />
<!--T:37--><br />
If you encounter errors like [https://gatk.broadinstitute.org/hc/en-us/community/posts/360067054832-GATK-4-1-7-0-error-java-lang-IllegalArgumentException-malformed-input-off-17635906-length-1 IllegalArgumentException] while using the installed modules on our clusters, we recommend that you try another workflow by using the program via [[Apptainer]].<br />
<br />
<!--T:38--><br />
A Docker image of GATK can be found [https://hub.docker.com/r/broadinstitute/gatk here] and other versions are available on this [https://hub.docker.com/r/broadinstitute/gatk/tags page]. You will need first to build an Apptainer image from the Docker image; <br />
to get the latest version for example, you can run the following commands on the cluster<br />
<br />
<!--T:40--><br />
<pre><br />
module load apptainer<br />
apptainer build gatk.sif docker://broadinstitute/gatk<br />
</pre><br />
<br />
<!--T:41--><br />
or to get a particular [https://hub.docker.com/r/broadinstitute/gatk/tags version]:<br />
<br />
<!--T:42--><br />
<pre><br />
module load apptainer<br />
apptainer build gatk_VERSION.sif docker://broadinstitute/gatk:VERSION<br />
</pre><br />
<br />
<!--T:43--><br />
In your [[Running jobs|SBATCH]] script, you should use something like this:<br />
<br />
<!--T:44--><br />
<pre><br />
module load apptainer<br />
apptainer exec -B /home -B /project -B /scratch -B /localscratch \<br />
<path to the image>/gatk.sif gatk [--java-options "-Xmx4G"] ToolName [GATK args]</pre><br />
<br />
<!--T:45--><br />
For more information about Apptainer, watch the recorded [https://www.youtube.com/watch?v=bpmrfVqBowY Apptainer webinar].<br />
<br />
==Frequently asked questions == <!--T:23--><br />
===How do I add a read group (RG) tag in my bam file? ===<br />
Assuming that you want to add a read group called <i>tag</i> to the file called <i>input.bam</i>, you can use the GATK/PICARD command [https://gatk.broadinstitute.org/hc/en-us/articles/360037226472-AddOrReplaceReadGroups-Picard- AddOrReplaceReadGroups]:<br />
<pre><br />
gatk AddOrReplaceReadGroups \<br />
-I input.bam \<br />
-O output.bam \<br />
--RGLB tag \<br />
--RGPL ILLUMINA <br />
--RGPU tag \<br />
--RGSM tag \<br />
--SORT_ORDER 'coordinate' \<br />
--CREATE_INDEX true<br />
</pre><br />
This assumes that your input file is sorted by coordinates and will generate an index along with the annotated output (<code>--CREATE_INDEX true</code>)<br />
<br />
===How do I deal with <code>java.lang.OutOfMemoryError: Java heap space</code> === <!--T:24--><br />
Subprograms of GATK often require more memory to process your files. If you were not using the <code>-Xms</code> command, add it to the <code>--java-options</code>. For example, let's imagine that you run the following command:<br />
<pre><br />
gatk MarkDuplicates \<br />
-I input.bam \<br />
-O marked_duplicates.bam \<br />
-M marked_dup_metrics.txt <br />
</pre><br />
<br />
<!--T:25--><br />
but it gives you the <code>java.lang.OutOfMemoryError: Java heap space</code> error. Try:<br />
<br />
<!--T:26--><br />
<pre><br />
gatk MarkDuplicates \<br />
--java-options "-Xmx8G DGATK_STACKTRACE_ON_USER_EXCEPTION=true"<br />
-I input.bam \<br />
-O marked_duplicates.bam \<br />
-M marked_dup_metrics.txt <br />
</pre><br />
<br />
<!--T:27--><br />
If it fails again, keep increasing the memory until you find the required memory for your particular dataset. If you are using any of our systems, <b>remember to request enough memory for this</b>.<br />
<br />
<!--T:28--><br />
If you are interested in knowing more about java heap space, you can start [https://plumbr.io/outofmemoryerror/java-heap-space here].<br />
<br />
===Increasing the heap memory does not fix <code>java.lang.OutOfMemoryError: Java heap space</code> === <!--T:29--><br />
There are cases in which the memory issue cannot be fixed with increasing the heap memory. This often happens with non-model organisms, and you are using too many scaffolds in your reference. In this case it is recommended to remove small scaffolds and create subsets of your reference. This implies that you have to map multiple times and run the pipelines in each of the subsets. <b>This approach does not work for all pipelines</b> so review your results carefully. GATK is designed with the human genome in mind, and therefore other organisms will require adjustments in many parameters and pipelines.<br />
<br />
===Using more resources than asked for === <!--T:30--><br />
Sometimes GATK/JAVA applications will use more memory or CPUs/threads than the numbers requested. This is often generated by the JAVA garbage collection. To control this, add <code>-XX:ConcGCThreads=1</code> to the <code>--java-options</code> argument. <br />
<br />
===FAQ on GATK === <!--T:31--><br />
You can find the [https://gatk.broadinstitute.org/hc/en-us/sections/360007226791-Troubleshooting-GATK4-Issues GATK FAQs on their website].<br />
<br />
=References = <!--T:32--><br />
[https://gatk.broadinstitute.org/hc/en-us GATK Home]<br />
<br />
<!--T:33--><br />
[https://gatk.broadinstitute.org/hc/en-us/articles/360035532012-Parallelism-Multithreading-Scatter-Gather GATK SPARK]<br />
<br />
<!--T:34--><br />
[https://gatk.broadinstitute.org/hc/en-us/articles/360035889611-How-can-I-make-GATK-tools-run-faster- Make GATK tools run faster]<br />
<br />
<!--T:35--><br />
[[Category:Bioinformatics]]<br />
[[Category:Software]]<br />
<br />
</translate></div>Kerrachehttps://docs.alliancecan.ca/mediawiki/index.php?title=Multifactor_authentication&diff=140425Multifactor authentication2023-07-11T15:41:01Z<p>Kerrache: </p>
<hr />
<div><languages /><br />
<br />
<translate><br />
<br />
<!--T:26--><br />
{{Panel<br />
|title=This topic is in testing phase<br />
|panelstyle=draft<br />
|content=<b>This article currently applies only to staff members</b>: Multifactor authentication is still being tested by staff members. It will be made available to all users as an option at some later date. <br />
[[Category:Draft]]<br />
}}<br />
<br />
<!--T:1--><br />
Multifactor authentication (MFA) allows you to protect your account with more than a password. Once your account is configured to use this feature, you will need to enter your username and password as usual, and then perform a second action (the <i>second factor</i>) to access most of our services. <br><br />
<br />
<!--T:21--><br />
You can choose any of these factors for this second authentication step:<br />
*Approving a notification on a smart device through the Duo Mobile application.<br />
*Entering a code generated on demand.<br />
*Pushing a button on a hardware key (YubiKey).<br />
<br />
<!--T:22--><br />
This feature will be progressively deployed, that is, it will not be immediately available for all our services.<br />
<br />
= Registering factors = <!--T:2--><br />
== Registering multiple factors ==<br />
When you enable multifactor authentication for your account, we <b>strongly recommend</b> that you configure at least two options for your second factor. For example, you can use a phone and single-use codes; a phone and a hardware key; or two hardware keys. This will ensure that if you lose one factor, you can still use your other one to access your account.<br />
<br />
== To use a smartphone or tablet == <!--T:3--><br />
#Install the Duo Mobile authentication application from the [https://itunes.apple.com/us/app/duo-mobile/id422663827 Apple Store] or on [https://play.google.com/store/apps/details?id=com.duosecurity.duomobile Google Play]<br />
#Go to the [https://ccdb.alliancecan.ca CCDB], connect to your account and select <i>My account → [https://ccdb.alliancecan.ca/multi_factor_authentications Multifactor authentication management]</i>.<br />
#Under <i>Register a device</i>, click on <i>Duo Mobile</i>.<br />
#Enter a name for your device.<br />
#In the Duo Mobile application, click the "+" sign to add a new account, and scan the QR code that is shown to you.<br />
<br />
== To use a YubiKey == <!--T:4--><br />
A YubiKey is a hardware token made by the [https://www.yubico.com/ Yubico] company. If you do not have a smartphone or tablet, do not wish to use your phone or tablet for multifactor authentication, or are often in a situation when using your phone or tablet is not possible, then a YubiKey is your best option.<br />
<br />
<!--T:23--><br />
A YubiKey is the size of a small USB stick and costs between $50 and $100. Different models can fit in USB-A, USB-C, or Lightning ports, and some also support near-field communication (NFC) for use with a phone or tablet.<br />
<br />
<!--T:5--><br />
Among the many protocols supported by YubiKeys, the one which works with SSH connections to our clusters is the Yubico One-Time Password (OTP). After you have registered a YubiKey for multifactor authentication, when you log on to one of our clusters you will be prompted for a one-time password (OTP). You respond by touching a button on your YubiKey, which generates and transmits a string of 32 characters to complete your authentication.<br />
<br />
<!--T:6--><br />
To register your YubiKey you will need its Public ID, Private ID, and Secret Key. If you have this information, go to the [https://ccdb.computecanada.ca/multi_factor_authentications Multifactor authentication management page]. If you do not have this information, configure your key using the steps below.<br />
<br />
=== Configuring your YubiKey for Yubico OTP === <!--T:7--><br />
<br />
<!--T:8--><br />
# Download and install the YubiKey Manager software from the [https://www.yubico.com/support/download/yubikey-manager/ Yubico website].<br />
# Insert your YubiKey and launch the YubiKey Manager software.<br />
# In the YubiKey Manager software, select <i>Applications</i>, then <i>OTP</i>. (Images below illustrate this and the next few steps.)<br />
# Select <i>Configure</i> for either slot 1 or slot 2. Slot 1 corresponds to a short touch (pressing for 1s to 2.5), while slot 2 is a long touch on the key (pressing for 3s to 5s). Slot 1 is typically pre-registered for Yubico cloud mode. If you are already using this slot for other services, either use slot 2, or click on <i>Swap</i> to transfer the configuration to slot 2 before configuring slot 1. <br />
# Select <i>Yubico OTP</i>.<br />
# Select <i>Use serial</i>, then generate a private ID and a secret key. <b>Securely save a copy of the data in the Public ID, Private ID, and Secret Key fields before you click on <i>Finish</i>, as you will need the data for the next step.</b><br />
# <b>IMPORTANT: Make sure you clicked on "Finish" in the previous step.</b><br />
# Log into the CCDB to register your YubiKey in the <i>[https://ccdb.alliancecan.ca/multi_factor_authentications Multifactor authentication management page]</i>.<br />
<gallery widths=300px heights=300px><br />
File:Yubico Manager OTP.png|Step 3<br />
File:Yubico Manager OTP configuration.png|Step 4<br />
File:Select Yubico OTP.png|Step 5<br />
File:Generate Yubikey IDs.png|Step 6, Step 7<br />
CCDB Yubikeys.png|Step 8<br />
</gallery><br />
<br />
= Using your second factor = <!--T:9--><br />
== When connecting via SSH == <br />
If your account has multifactor authentication enabled, when you connect via SSH to a cluster which supports MFA, you will be prompted to use your second factor after you first use either your password or your [[SSH Keys|SSH key]]. This prompt will look like this:<br />
{{Command|ssh cluster.computecanada.ca<br />
|result= Duo two-factor login for name<br />
<br />
<!--T:10--><br />
Enter a passcode or select one of the following options:<br />
<br />
<!--T:11--><br />
1. Duo Push to My phone (iOS)<br />
<br />
<!--T:12--><br />
Passcode or option (1-1):}}<br />
At this point, you can select which phone or tablet you want Duo to send a notification to. If you have multiple devices enrolled, you will be shown a list. You will then get a notification on your device, which you accept to complete the authentication.<br />
<br />
<!--T:13--><br />
If you are using a YubiKey, a backup code, or if you prefer to enter the time-based one-time password that the Duo Mobile application shows, you would write these instead of selecting an option. For example:<br />
{{Command|ssh cluster.computecanada.ca<br />
|result= Duo two-factor login for name<br />
<br />
<!--T:14--><br />
Enter a passcode or select one of the following options:<br />
<br />
<!--T:15--><br />
1. Duo Push to My phone (iOS)<br />
<br />
<!--T:16--><br />
Passcode or option (1-1):vvcccbhbllnuuebegkkbcfdftndjijlneejilrgiguki<br />
Success. Logging you in...}}<br />
<br />
=== Configuring your SSH client to only ask every so often === <!--T:17--><br />
If you use OpenSSH to connect, you can reduce the frequency with which you are asked for a second factor. To do so, edit your <code>.ssh/config</code> to add the lines:<br />
<br />
<!--T:24--><br />
<pre><br />
Host HOSTNAME<br />
ControlPath ~/.ssh/cm-%r@%h:%p<br />
ControlMaster auto<br />
ControlPersist 10m<br />
</pre><br />
where you would replace <code>HOSTNAME</code> with the host name of the server for which you want this configuration.<br />
<br />
== When authenticating to our account portal == <!--T:18--><br />
Once multifactor authentication is enabled on your account, you will be required to use it when connecting to our account portal. After entering your username and password, you will see a prompt similar to this, where you click on the option you want to use. <br><br />
(Note: <i>This screen will be updated</i>.)<br />
<gallery widths=300px heights=300px><br />
File:CCDB MFA prompt.png<br />
</gallery><br />
<br />
= Frequently asked questions = <!--T:19--><br />
== I have an Android phone which is older than Android 9. I do not find the Duo Mobile application. Can I still use Duo ? ==<br />
Yes. However, you have to download the application from the Duo website. See [https://help.duo.com/s/article/2211?language=en_US this page] for more details. <br />
<br />
== I do not have a smartphone or tablet, or they are too old. Can I still use multifactor authentication? == <!--T:25--><br />
Yes. In this case, you need [[#To use a YubiKey|to use a YubiKey]].<br />
<br />
== I have lost my second factor device. What can I do? == <!--T:20--><br />
* If you have backup codes, or if you have more than one device, use that other mechanism to connect to your account on our [https://ccdb.alliancecan.ca/multi_factor_authentications account portal], and then delete your lost device from the list. Then, register a new device. <br />
* If you do not have backup codes or have lost all of your devices, copy the following list providing answers to as many questions as you can. Email this information to support@tech.alliancecan.ca. <br />
<br />
<!--T:30--><br />
What is the primary email address registered in your account?<br />
For how long have you had an active account with us?<br />
What is your research area?<br />
What is your IP address? (to see your IP address, point your browser to this [https://whatismyipaddress.com/ link]).<br />
Who is the principal investigator sponsoring your account?<br />
Who are your group members?<br />
Who can we contact to validate your request?<br />
Which clusters do you use the most?<br />
Which modules do you load most often?<br />
When did you run your last job?<br />
Provide a few of your latest job IDs.<br />
Provide ticket topics and ticket IDs from your recent requests for technical support.<br />
<br />
== Which SSH clients can be used when multifactor authentication is configured? == <!--T:29--><br />
* MobaXTerm version 23.1 or more recent<br />
*<br />
<br />
= Advanced usage = <!--T:27--><br />
== Configuring your YubiKey for Yubico OTP using the Command Line (<code>ykman</code>)==<br />
# Install the command line YubiKey Manager software (<code>ykman</code>) following instructions for your OS from Yubico's [https://docs.yubico.com/software/yubikey/tools/ykman/Install_ykman.html#download-ykman ykman guide].<br />
# Insert your YubiKey and read key information with the command <code>ykman info</code>.<br />
# Read OTP information with the command <code>ykman otp info</code>.<br />
# Select the slot you wish to program and use the command <code>ykman otp yubiotp</code> to program it.<br />
# <b>Securely save a copy of the data in the Public ID, Private ID, and Secret Key fields. You will need the data for the next step.</b><br />
# Log into the CCDB to register your YubiKey in the <i>[https://ccdb.alliancecan.ca/multi_factor_authentications Multifactor authentication management page]</i>.<br />
<br />
<!--T:28--><br />
:<source lang="console"><br />
[name@yourLaptop]$ ykman otp yubiotp -uGgP vvcccctffclk 2<br />
Using a randomly generated private ID: bc3dd98eaa12<br />
Using a randomly generated secret key: ae012f11bc5a00d3cac00f1d57aa0b12<br />
Upload credential to YubiCloud? [y/N]: y<br />
Upload to YubiCloud initiated successfully.<br />
Program an OTP credential in slot 2? [y/N]: y<br />
Opening upload form in browser: https://upload.yubico.com/proceed/4567ad02-c3a2-1234-a1c3-abe3f4d21c69<br />
</source><br />
<br />
</translate></div>Kerrachehttps://docs.alliancecan.ca/mediawiki/index.php?title=Multifactor_authentication&diff=140424Multifactor authentication2023-07-11T15:39:41Z<p>Kerrache: </p>
<hr />
<div><languages /><br />
<br />
<translate><br />
<br />
<!--T:26--><br />
{{Panel<br />
|title=This topic is in testing phase<br />
|panelstyle=draft<br />
|content=<b>This article currently applies only to staff members</b>: Multifactor authentication is still being tested by staff members. It will be made available to all users as an option at some later date. <br />
[[Category:Draft]]<br />
}}<br />
<br />
<!--T:1--><br />
Multifactor authentication (MFA) allows you to protect your account with more than a password. Once your account is configured to use this feature, you will need to enter your username and password as usual, and then perform a second action (the <i>second factor</i>) to access most of our services. <br><br />
<br />
<!--T:21--><br />
You can choose any of these factors for this second authentication step:<br />
*Approving a notification on a smart device through the Duo Mobile application.<br />
*Entering a code generated on demand.<br />
*Pushing a button on a hardware key (YubiKey).<br />
<br />
<!--T:22--><br />
This feature will be progressively deployed, that is, it will not be immediately available for all our services.<br />
<br />
= Registering factors = <!--T:2--><br />
== Registering multiple factors ==<br />
When you enable multifactor authentication for your account, we <b>strongly recommend</b> that you configure at least two options for your second factor. For example, you can use a phone and single-use codes; a phone and a hardware key; or two hardware keys. This will ensure that if you lose one factor, you can still use your other one to access your account.<br />
<br />
== To use a smartphone or tablet == <!--T:3--><br />
#Install the Duo Mobile authentication application from the [https://itunes.apple.com/us/app/duo-mobile/id422663827 Apple Store] or on [https://play.google.com/store/apps/details?id=com.duosecurity.duomobile Google Play]<br />
#Go to the [https://ccdb.alliancecan.ca CCDB], connect to your account and select <i>My account → [https://ccdb.alliancecan.ca/multi_factor_authentications Multifactor authentication management]</i>.<br />
#Under <i>Register a device</i>, click on <i>Duo Mobile</i>.<br />
#Enter a name for your device.<br />
#In the Duo Mobile application, click the "+" sign to add a new account, and scan the QR code that is shown to you.<br />
<br />
== To use a YubiKey == <!--T:4--><br />
A YubiKey is a hardware token made by the [https://www.yubico.com/ Yubico] company. If you do not have a smartphone or tablet, do not wish to use your phone or tablet for multifactor authentication, or are often in a situation when using your phone or tablet is not possible, then a YubiKey is your best option.<br />
<br />
<!--T:23--><br />
A YubiKey is the size of a small USB stick and costs between $50 and $100. Different models can fit in USB-A, USB-C, or Lightning ports, and some also support near-field communication (NFC) for use with a phone or tablet.<br />
<br />
<!--T:5--><br />
Among the many protocols supported by YubiKeys, the one which works with SSH connections to our clusters is the Yubico One-Time Password (OTP). After you have registered a YubiKey for multifactor authentication, when you log on to one of our clusters you will be prompted for a one-time password (OTP). You respond by touching a button on your YubiKey, which generates and transmits a string of 32 characters to complete your authentication.<br />
<br />
<!--T:6--><br />
To register your YubiKey you will need its Public ID, Private ID, and Secret Key. If you have this information, go to the [https://ccdb.computecanada.ca/multi_factor_authentications Multifactor authentication management page]. If you do not have this information, configure your key using the steps below.<br />
<br />
=== Configuring your YubiKey for Yubico OTP === <!--T:7--><br />
<br />
<!--T:8--><br />
# Download and install the YubiKey Manager software from the [https://www.yubico.com/support/download/yubikey-manager/ Yubico website].<br />
# Insert your YubiKey and launch the YubiKey Manager software.<br />
# In the YubiKey Manager software, select <i>Applications</i>, then <i>OTP</i>. (Images below illustrate this and the next few steps.)<br />
# Select <i>Configure</i> for either slot 1 or slot 2. Slot 1 corresponds to a short touch (pressing for 1s to 2.5), while slot 2 is a long touch on the key (pressing for 3s to 5s). Slot 1 is typically pre-registered for Yubico cloud mode. If you are already using this slot for other services, either use slot 2, or click on <i>Swap</i> to transfer the configuration to slot 2 before configuring slot 1. <br />
# Select <i>Yubico OTP</i>.<br />
# Select <i>Use serial</i>, then generate a private ID and a secret key. <b>Securely save a copy of the data in the Public ID, Private ID, and Secret Key fields before you click on <i>Finish</i>, as you will need the data for the next step.</b><br />
# <b>IMPORTANT: Make sure you clicked on "Finish" in the previous step.</b><br />
# Log into the CCDB to register your YubiKey in the <i>[https://ccdb.alliancecan.ca/multi_factor_authentications Multifactor authentication management page]</i>.<br />
<gallery widths=300px heights=300px><br />
File:Yubico Manager OTP.png|Step 3<br />
File:Yubico Manager OTP configuration.png|Step 4<br />
File:Select Yubico OTP.png|Step 5<br />
File:Generate Yubikey IDs.png|Step 6, Step 7<br />
CCDB Yubikeys.png|Step 8<br />
</gallery><br />
<br />
= Using your second factor = <!--T:9--><br />
== When connecting via SSH == <br />
If your account has multifactor authentication enabled, when you connect via SSH to a cluster which supports MFA, you will be prompted to use your second factor after you first use either your password or your [[SSH Keys|SSH key]]. This prompt will look like this:<br />
{{Command|ssh cluster.computecanada.ca<br />
|result= Duo two-factor login for name<br />
<br />
<!--T:10--><br />
Enter a passcode or select one of the following options:<br />
<br />
<!--T:11--><br />
1. Duo Push to My phone (iOS)<br />
<br />
<!--T:12--><br />
Passcode or option (1-1):}}<br />
At this point, you can select which phone or tablet you want Duo to send a notification to. If you have multiple devices enrolled, you will be shown a list. You will then get a notification on your device, which you accept to complete the authentication.<br />
<br />
<!--T:13--><br />
If you are using a YubiKey, a backup code, or if you prefer to enter the time-based one-time password that the Duo Mobile application shows, you would write these instead of selecting an option. For example:<br />
{{Command|ssh cluster.computecanada.ca<br />
|result= Duo two-factor login for name<br />
<br />
<!--T:14--><br />
Enter a passcode or select one of the following options:<br />
<br />
<!--T:15--><br />
1. Duo Push to My phone (iOS)<br />
<br />
<!--T:16--><br />
Passcode or option (1-1):vvcccbhbllnuuebegkkbcfdftndjijlneejilrgiguki<br />
Success. Logging you in...}}<br />
<br />
=== Configuring your SSH client to only ask every so often === <!--T:17--><br />
If you use OpenSSH to connect, you can reduce the frequency with which you are asked for a second factor. To do so, edit your <code>.ssh/config</code> to add the lines:<br />
<br />
<!--T:24--><br />
<pre><br />
Host HOSTNAME<br />
ControlPath ~/.ssh/cm-%r@%h:%p<br />
ControlMaster auto<br />
ControlPersist 10m<br />
</pre><br />
where you would replace <code>HOSTNAME</code> with the host name of the server for which you want this configuration.<br />
<br />
== When authenticating to our account portal == <!--T:18--><br />
Once multifactor authentication is enabled on your account, you will be required to use it when connecting to our account portal. After entering your username and password, you will see a prompt similar to this, where you click on the option you want to use. <br><br />
(Note: <i>This screen will be updated</i>.)<br />
<gallery widths=300px heights=300px><br />
File:CCDB MFA prompt.png<br />
</gallery><br />
<br />
= Frequently asked questions = <!--T:19--><br />
== I have an Android phone which is older than Android 9. I do not find the Duo Mobile application. Can I still use Duo ? ==<br />
Yes. However, you have to download the application from the Duo website. See [https://help.duo.com/s/article/2211?language=en_US this page] for more details. <br />
<br />
== I do not have a smartphone or tablet, or they are too old. Can I still use multifactor authentication? == <!--T:25--><br />
Yes. In this case, you need [[#To use a YubiKey|to use a YubiKey]].<br />
<br />
== I have lost my second factor device. What can I do? == <!--T:20--><br />
* If you have backup codes, or if you have more than one device, use that other mechanism to connect to your account on our [https://ccdb.alliancecan.ca/multi_factor_authentications account portal], and then delete your lost device from the list. Then, register a new device. <br />
* If you do not have backup codes or have lost all of your devices, copy the following list providing answers to as many questions as you can. Email this information to support@tech.alliancecan.ca. <br />
<br />
<!--T:30--><br />
What is the primary email address registered in your account?<br />
For how long have you had an active account with us?<br />
What is your research area?<br />
What is your IP address? (to see your IP address, point your browser to this [[lhttps://whatismyipaddress.com/|link]] https://whatismyipaddress.com/)<br />
Who is the principal investigator sponsoring your account?<br />
Who are your group members?<br />
Who can we contact to validate your request?<br />
Which clusters do you use the most?<br />
Which modules do you load most often?<br />
When did you run your last job?<br />
Provide a few of your latest job IDs.<br />
Provide ticket topics and ticket IDs from your recent requests for technical support.<br />
<br />
== Which SSH clients can be used when multifactor authentication is configured? == <!--T:29--><br />
* MobaXTerm version 23.1 or more recent<br />
*<br />
<br />
= Advanced usage = <!--T:27--><br />
== Configuring your YubiKey for Yubico OTP using the Command Line (<code>ykman</code>)==<br />
# Install the command line YubiKey Manager software (<code>ykman</code>) following instructions for your OS from Yubico's [https://docs.yubico.com/software/yubikey/tools/ykman/Install_ykman.html#download-ykman ykman guide].<br />
# Insert your YubiKey and read key information with the command <code>ykman info</code>.<br />
# Read OTP information with the command <code>ykman otp info</code>.<br />
# Select the slot you wish to program and use the command <code>ykman otp yubiotp</code> to program it.<br />
# <b>Securely save a copy of the data in the Public ID, Private ID, and Secret Key fields. You will need the data for the next step.</b><br />
# Log into the CCDB to register your YubiKey in the <i>[https://ccdb.alliancecan.ca/multi_factor_authentications Multifactor authentication management page]</i>.<br />
<br />
<!--T:28--><br />
:<source lang="console"><br />
[name@yourLaptop]$ ykman otp yubiotp -uGgP vvcccctffclk 2<br />
Using a randomly generated private ID: bc3dd98eaa12<br />
Using a randomly generated secret key: ae012f11bc5a00d3cac00f1d57aa0b12<br />
Upload credential to YubiCloud? [y/N]: y<br />
Upload to YubiCloud initiated successfully.<br />
Program an OTP credential in slot 2? [y/N]: y<br />
Opening upload form in browser: https://upload.yubico.com/proceed/4567ad02-c3a2-1234-a1c3-abe3f4d21c69<br />
</source><br />
<br />
</translate></div>Kerrachehttps://docs.alliancecan.ca/mediawiki/index.php?title=Multifactor_authentication&diff=140423Multifactor authentication2023-07-11T15:37:52Z<p>Kerrache: </p>
<hr />
<div><languages /><br />
<br />
<translate><br />
<br />
<!--T:26--><br />
{{Panel<br />
|title=This topic is in testing phase<br />
|panelstyle=draft<br />
|content=<b>This article currently applies only to staff members</b>: Multifactor authentication is still being tested by staff members. It will be made available to all users as an option at some later date. <br />
[[Category:Draft]]<br />
}}<br />
<br />
<!--T:1--><br />
Multifactor authentication (MFA) allows you to protect your account with more than a password. Once your account is configured to use this feature, you will need to enter your username and password as usual, and then perform a second action (the <i>second factor</i>) to access most of our services. <br><br />
<br />
<!--T:21--><br />
You can choose any of these factors for this second authentication step:<br />
*Approving a notification on a smart device through the Duo Mobile application.<br />
*Entering a code generated on demand.<br />
*Pushing a button on a hardware key (YubiKey).<br />
<br />
<!--T:22--><br />
This feature will be progressively deployed, that is, it will not be immediately available for all our services.<br />
<br />
= Registering factors = <!--T:2--><br />
== Registering multiple factors ==<br />
When you enable multifactor authentication for your account, we <b>strongly recommend</b> that you configure at least two options for your second factor. For example, you can use a phone and single-use codes; a phone and a hardware key; or two hardware keys. This will ensure that if you lose one factor, you can still use your other one to access your account.<br />
<br />
== To use a smartphone or tablet == <!--T:3--><br />
#Install the Duo Mobile authentication application from the [https://itunes.apple.com/us/app/duo-mobile/id422663827 Apple Store] or on [https://play.google.com/store/apps/details?id=com.duosecurity.duomobile Google Play]<br />
#Go to the [https://ccdb.alliancecan.ca CCDB], connect to your account and select <i>My account → [https://ccdb.alliancecan.ca/multi_factor_authentications Multifactor authentication management]</i>.<br />
#Under <i>Register a device</i>, click on <i>Duo Mobile</i>.<br />
#Enter a name for your device.<br />
#In the Duo Mobile application, click the "+" sign to add a new account, and scan the QR code that is shown to you.<br />
<br />
== To use a YubiKey == <!--T:4--><br />
A YubiKey is a hardware token made by the [https://www.yubico.com/ Yubico] company. If you do not have a smartphone or tablet, do not wish to use your phone or tablet for multifactor authentication, or are often in a situation when using your phone or tablet is not possible, then a YubiKey is your best option.<br />
<br />
<!--T:23--><br />
A YubiKey is the size of a small USB stick and costs between $50 and $100. Different models can fit in USB-A, USB-C, or Lightning ports, and some also support near-field communication (NFC) for use with a phone or tablet.<br />
<br />
<!--T:5--><br />
Among the many protocols supported by YubiKeys, the one which works with SSH connections to our clusters is the Yubico One-Time Password (OTP). After you have registered a YubiKey for multifactor authentication, when you log on to one of our clusters you will be prompted for a one-time password (OTP). You respond by touching a button on your YubiKey, which generates and transmits a string of 32 characters to complete your authentication.<br />
<br />
<!--T:6--><br />
To register your YubiKey you will need its Public ID, Private ID, and Secret Key. If you have this information, go to the [https://ccdb.computecanada.ca/multi_factor_authentications Multifactor authentication management page]. If you do not have this information, configure your key using the steps below.<br />
<br />
=== Configuring your YubiKey for Yubico OTP === <!--T:7--><br />
<br />
<!--T:8--><br />
# Download and install the YubiKey Manager software from the [https://www.yubico.com/support/download/yubikey-manager/ Yubico website].<br />
# Insert your YubiKey and launch the YubiKey Manager software.<br />
# In the YubiKey Manager software, select <i>Applications</i>, then <i>OTP</i>. (Images below illustrate this and the next few steps.)<br />
# Select <i>Configure</i> for either slot 1 or slot 2. Slot 1 corresponds to a short touch (pressing for 1s to 2.5), while slot 2 is a long touch on the key (pressing for 3s to 5s). Slot 1 is typically pre-registered for Yubico cloud mode. If you are already using this slot for other services, either use slot 2, or click on <i>Swap</i> to transfer the configuration to slot 2 before configuring slot 1. <br />
# Select <i>Yubico OTP</i>.<br />
# Select <i>Use serial</i>, then generate a private ID and a secret key. <b>Securely save a copy of the data in the Public ID, Private ID, and Secret Key fields before you click on <i>Finish</i>, as you will need the data for the next step.</b><br />
# <b>IMPORTANT: Make sure you clicked on "Finish" in the previous step.</b><br />
# Log into the CCDB to register your YubiKey in the <i>[https://ccdb.alliancecan.ca/multi_factor_authentications Multifactor authentication management page]</i>.<br />
<gallery widths=300px heights=300px><br />
File:Yubico Manager OTP.png|Step 3<br />
File:Yubico Manager OTP configuration.png|Step 4<br />
File:Select Yubico OTP.png|Step 5<br />
File:Generate Yubikey IDs.png|Step 6, Step 7<br />
CCDB Yubikeys.png|Step 8<br />
</gallery><br />
<br />
= Using your second factor = <!--T:9--><br />
== When connecting via SSH == <br />
If your account has multifactor authentication enabled, when you connect via SSH to a cluster which supports MFA, you will be prompted to use your second factor after you first use either your password or your [[SSH Keys|SSH key]]. This prompt will look like this:<br />
{{Command|ssh cluster.computecanada.ca<br />
|result= Duo two-factor login for name<br />
<br />
<!--T:10--><br />
Enter a passcode or select one of the following options:<br />
<br />
<!--T:11--><br />
1. Duo Push to My phone (iOS)<br />
<br />
<!--T:12--><br />
Passcode or option (1-1):}}<br />
At this point, you can select which phone or tablet you want Duo to send a notification to. If you have multiple devices enrolled, you will be shown a list. You will then get a notification on your device, which you accept to complete the authentication.<br />
<br />
<!--T:13--><br />
If you are using a YubiKey, a backup code, or if you prefer to enter the time-based one-time password that the Duo Mobile application shows, you would write these instead of selecting an option. For example:<br />
{{Command|ssh cluster.computecanada.ca<br />
|result= Duo two-factor login for name<br />
<br />
<!--T:14--><br />
Enter a passcode or select one of the following options:<br />
<br />
<!--T:15--><br />
1. Duo Push to My phone (iOS)<br />
<br />
<!--T:16--><br />
Passcode or option (1-1):vvcccbhbllnuuebegkkbcfdftndjijlneejilrgiguki<br />
Success. Logging you in...}}<br />
<br />
=== Configuring your SSH client to only ask every so often === <!--T:17--><br />
If you use OpenSSH to connect, you can reduce the frequency with which you are asked for a second factor. To do so, edit your <code>.ssh/config</code> to add the lines:<br />
<br />
<!--T:24--><br />
<pre><br />
Host HOSTNAME<br />
ControlPath ~/.ssh/cm-%r@%h:%p<br />
ControlMaster auto<br />
ControlPersist 10m<br />
</pre><br />
where you would replace <code>HOSTNAME</code> with the host name of the server for which you want this configuration.<br />
<br />
== When authenticating to our account portal == <!--T:18--><br />
Once multifactor authentication is enabled on your account, you will be required to use it when connecting to our account portal. After entering your username and password, you will see a prompt similar to this, where you click on the option you want to use. <br><br />
(Note: <i>This screen will be updated</i>.)<br />
<gallery widths=300px heights=300px><br />
File:CCDB MFA prompt.png<br />
</gallery><br />
<br />
= Frequently asked questions = <!--T:19--><br />
== I have an Android phone which is older than Android 9. I do not find the Duo Mobile application. Can I still use Duo ? ==<br />
Yes. However, you have to download the application from the Duo website. See [https://help.duo.com/s/article/2211?language=en_US this page] for more details. <br />
<br />
== I do not have a smartphone or tablet, or they are too old. Can I still use multifactor authentication? == <!--T:25--><br />
Yes. In this case, you need [[#To use a YubiKey|to use a YubiKey]].<br />
<br />
== I have lost my second factor device. What can I do? == <!--T:20--><br />
* If you have backup codes, or if you have more than one device, use that other mechanism to connect to your account on our [https://ccdb.alliancecan.ca/multi_factor_authentications account portal], and then delete your lost device from the list. Then, register a new device. <br />
* If you do not have backup codes or have lost all of your devices, copy the following list providing answers to as many questions as you can. Email this information to support@tech.alliancecan.ca. <br />
<br />
<!--T:30--><br />
What is the primary email address registered in your account?<br />
For how long have you had an active account with us?<br />
What is your research area?<br />
What is your IP address? (to see your IP address, point your browser to https://whatismyipaddress.com/)<br />
Who is the principal investigator sponsoring your account?<br />
Who are your group members?<br />
Who can we contact to validate your request?<br />
Which clusters do you use the most?<br />
Which modules do you load most often?<br />
When did you run your last job?<br />
Provide a few of your latest job IDs.<br />
Provide ticket topics and ticket IDs from your recent requests for technical support.<br />
<br />
== Which SSH clients can be used when multifactor authentication is configured? == <!--T:29--><br />
* MobaXTerm version 23.1 or more recent<br />
*<br />
<br />
= Advanced usage = <!--T:27--><br />
== Configuring your YubiKey for Yubico OTP using the Command Line (<code>ykman</code>)==<br />
# Install the command line YubiKey Manager software (<code>ykman</code>) following instructions for your OS from Yubico's [https://docs.yubico.com/software/yubikey/tools/ykman/Install_ykman.html#download-ykman ykman guide].<br />
# Insert your YubiKey and read key information with the command <code>ykman info</code>.<br />
# Read OTP information with the command <code>ykman otp info</code>.<br />
# Select the slot you wish to program and use the command <code>ykman otp yubiotp</code> to program it.<br />
# <b>Securely save a copy of the data in the Public ID, Private ID, and Secret Key fields. You will need the data for the next step.</b><br />
# Log into the CCDB to register your YubiKey in the <i>[https://ccdb.alliancecan.ca/multi_factor_authentications Multifactor authentication management page]</i>.<br />
<br />
<!--T:28--><br />
:<source lang="console"><br />
[name@yourLaptop]$ ykman otp yubiotp -uGgP vvcccctffclk 2<br />
Using a randomly generated private ID: bc3dd98eaa12<br />
Using a randomly generated secret key: ae012f11bc5a00d3cac00f1d57aa0b12<br />
Upload credential to YubiCloud? [y/N]: y<br />
Upload to YubiCloud initiated successfully.<br />
Program an OTP credential in slot 2? [y/N]: y<br />
Opening upload form in browser: https://upload.yubico.com/proceed/4567ad02-c3a2-1234-a1c3-abe3f4d21c69<br />
</source><br />
<br />
</translate></div>Kerrachehttps://docs.alliancecan.ca/mediawiki/index.php?title=Diskusage_Explorer&diff=135478Diskusage Explorer2023-05-02T21:01:24Z<p>Kerrache: </p>
<hr />
<div><languages /><br />
<translate><br />
<br />
==Content of Folders== <!--T:1--><br />
<br />
<!--T:2--><br />
<span style="color:red">Warning: At the moment this feature is only available on [[Béluga/en|Béluga]] and [[Narval/en|Narval]].</span><br />
<br />
<!--T:3--><br />
You can get a breakdown by folder of how the diskspace is being consumed in your home, scratch and project spaces. That information is currently updated once a day and is stored in an [[SQLite]] format for fast access. <br />
<br />
<!--T:4--><br />
Here is a walkthrough of how to explore your disk consumption, using the example of the project space <code>def-professor</code> as the particular directory to investigate.<br />
<br />
=== Ncurse User Interface === <!--T:5--><br />
Choose a project space you have access to and which you wish to analyze, which for the purpose of this discussion we will assume to be <tt>def-professor</tt>.<br />
{{Command|diskusage_explorer /project/def-professor}}<br />
This command loads a browser that shows the resources consumed by all files under any directory tree<br />
[[File:Ncurse duc.png|thumb|using|450px|frame|left| Navigating your project space with duc's ncurse tool]]<br />
<br clear=all> <!-- This is to prevent the next section from filling to the right of the image. --><br />
<br />
<!--T:6--><br />
Type <code>c</code> to toggle between consumed disk space and the number of files, <code>q</code> or <code><esc></code> to quit and <code>h</code> for help.<br />
<br />
<!--T:7--><br />
If you are only interested in a sub-directory of this project space and do not want to navigate the whole tree in the ncurse user interface, <br />
{{Command|diskusage_explorer /project/def-professor/subdirectory/}}<br />
<br />
<!--T:8--><br />
A complete manual page is available with the <code>man duc</code> command.<br />
<br />
=== Graphical User Interface === <!--T:9--><br />
<br />
<!--T:10--><br />
Note that when the login node is especially busy or if you have an especially large amount of files in you project space, the graphical interface mode can be slow and choppy. For a better experience, you can read the section below to run <code>diskusage_explorer</code> on your own machine.<br />
<br />
<!--T:11--><br />
Note that Compute Canada recommends the use of the standard text-based ncurse mode on our cluster login nodes but <code>diskusage_explorer</code> does also include a nice graphical user interface (GUI). <br />
<br />
<!--T:12--><br />
First make sure that you are connected to the cluster in such a way that [[SSH]] is capable of correctly displaying GUI applications. You can then use a graphical interface by means of the command,<br />
{{Command|duc gui -d /project/.duc_databases/def-professor.sqlite /project/def-professor}}<br />
<br />
<!--T:13--><br />
You can navigate the folders with the mouse and still type <code>c</code> to toggle between the size of the files and the number of files.<br />
<br />
<!--T:14--><br />
[[File:Duc gui navigation.gif|thumb|using|450px|frame|left|Navigating your project space with duc's GUI tool]]<br />
<br clear=all> <!-- This is to prevent the next section from filling to the right of the image. --><br />
<br />
=== Browse faster on your own machine === <!--T:15--><br />
<br />
<!--T:16--><br />
First [http://duc.zevv.nl/#download install the diskusage_explorer software] on your local machine and then, still on your local machine, download the SQLite file from your cluster and run <code>duc</code>. <br />
<br />
<!--T:17--><br />
<pre><br />
rsync -v --progress username@beluga.calculcanada.ca:/project/.duc_databases/def-professor.sqlite .<br />
duc gui -d ./def-professor.sqlite /project/def-professor<br />
</pre><br />
<br />
<!--T:18--><br />
This immediately leads to a smoother and more satisfying browsing experience.<br />
<br />
== Space and inode usage per user on cedar == <!--T:19--><br />
<br />
<!--T:20--><br />
On cedar, it is possible for any member of a group to run <code>diskusage_report</code> with the following options <code>--per_user</code> and <code>--all_users</code> to have a break-down usage per user. The first option displays only heavy users. In other terms, members of the group who have more files and/or occupy more space. When both options are used, the command gives the break-down usage for all members of the group. This is a handy command that helps to identify the users within a group who have more files and/or a large amount of data and ask them to better manage their data by reducing their inode usage for example.<br />
<br />
<!--T:21--><br />
In the following example, a user '''user01''' runs the command and got the following output:<br />
<br />
<!--T:22--><br />
<source lang="bash"><br />
[user01@cedar1 ~]$ diskusage_report --per_user --all_users<br />
Description Space # of files<br />
/home (user user01) 109k/50G 12/500k<br />
/scratch (user user01) 4000/20T 1/1000k<br />
/project (group user01) 0/2048k 0/1025<br />
/project (group def-professor) 9434G/10T 497k/500k<br />
<br />
Breakdown for project def-professor (Last update: 2023-05-02 01:03:10)<br />
User File count Size Location<br />
-------------------------------------------------------------------------<br />
user01 28313 4.00 GiB On disk<br />
user02 11926 3.74 GiB On disk<br />
user03 14507 6121.03 GiB On disk<br />
user04 4010 377.86 GiB On disk<br />
user05 125929 262.75 GiB On disk<br />
user06 201099 60.51 GiB On disk<br />
user07 84806 1721.33 GiB On disk<br />
user08 26516 947.23 GiB On disk<br />
Total 497106 9510.43 GiB On disk<br />
<!--T:23--><br />
<br />
<!--T:24--><br />
Breakdown for nearline def-professor (Last update: 2023-05-02 01:01:30)<br />
User File count Size Location<br />
-------------------------------------------------------------------------<br />
user03 5 1197.90 GiB On disk and tape<br />
Total 5 1197.90 GiB On disk and tape<br />
</source><br />
<br />
<!--T:25--><br />
This group has 8 users and the above output shows clearly that at least 4 of them have a large number of files for a small amount of data:<br />
<br />
<!--T:26--><br />
<source lang="bash"><br />
User File count Size Location<br />
-------------------------------------------------------------------------<br />
user01 28313 4.00 GiB On disk<br />
user02 11926 3.74 GiB On disk<br />
user05 125929 262.75 GiB On disk<br />
user06 201099 60.51 GiB On disk<br />
</source><br />
<br />
</translate></div>Kerrachehttps://docs.alliancecan.ca/mediawiki/index.php?title=Diskusage_Explorer&diff=135477Diskusage Explorer2023-05-02T21:00:31Z<p>Kerrache: diskusage_report on cedar: changing the inode</p>
<hr />
<div><languages /><br />
<translate><br />
<br />
==Content of Folders== <!--T:1--><br />
<br />
<!--T:2--><br />
<span style="color:red">Warning: At the moment this feature is only available on [[Béluga/en|Béluga]] and [[Narval/en|Narval]].</span><br />
<br />
<!--T:3--><br />
You can get a breakdown by folder of how the diskspace is being consumed in your home, scratch and project spaces. That information is currently updated once a day and is stored in an [[SQLite]] format for fast access. <br />
<br />
<!--T:4--><br />
Here is a walkthrough of how to explore your disk consumption, using the example of the project space <code>def-professor</code> as the particular directory to investigate.<br />
<br />
=== Ncurse User Interface === <!--T:5--><br />
Choose a project space you have access to and which you wish to analyze, which for the purpose of this discussion we will assume to be <tt>def-professor</tt>.<br />
{{Command|diskusage_explorer /project/def-professor}}<br />
This command loads a browser that shows the resources consumed by all files under any directory tree<br />
[[File:Ncurse duc.png|thumb|using|450px|frame|left| Navigating your project space with duc's ncurse tool]]<br />
<br clear=all> <!-- This is to prevent the next section from filling to the right of the image. --><br />
<br />
<!--T:6--><br />
Type <code>c</code> to toggle between consumed disk space and the number of files, <code>q</code> or <code><esc></code> to quit and <code>h</code> for help.<br />
<br />
<!--T:7--><br />
If you are only interested in a sub-directory of this project space and do not want to navigate the whole tree in the ncurse user interface, <br />
{{Command|diskusage_explorer /project/def-professor/subdirectory/}}<br />
<br />
<!--T:8--><br />
A complete manual page is available with the <code>man duc</code> command.<br />
<br />
=== Graphical User Interface === <!--T:9--><br />
<br />
<!--T:10--><br />
Note that when the login node is especially busy or if you have an especially large amount of files in you project space, the graphical interface mode can be slow and choppy. For a better experience, you can read the section below to run <code>diskusage_explorer</code> on your own machine.<br />
<br />
<!--T:11--><br />
Note that Compute Canada recommends the use of the standard text-based ncurse mode on our cluster login nodes but <code>diskusage_explorer</code> does also include a nice graphical user interface (GUI). <br />
<br />
<!--T:12--><br />
First make sure that you are connected to the cluster in such a way that [[SSH]] is capable of correctly displaying GUI applications. You can then use a graphical interface by means of the command,<br />
{{Command|duc gui -d /project/.duc_databases/def-professor.sqlite /project/def-professor}}<br />
<br />
<!--T:13--><br />
You can navigate the folders with the mouse and still type <code>c</code> to toggle between the size of the files and the number of files.<br />
<br />
<!--T:14--><br />
[[File:Duc gui navigation.gif|thumb|using|450px|frame|left|Navigating your project space with duc's GUI tool]]<br />
<br clear=all> <!-- This is to prevent the next section from filling to the right of the image. --><br />
<br />
=== Browse faster on your own machine === <!--T:15--><br />
<br />
<!--T:16--><br />
First [http://duc.zevv.nl/#download install the diskusage_explorer software] on your local machine and then, still on your local machine, download the SQLite file from your cluster and run <code>duc</code>. <br />
<br />
<!--T:17--><br />
<pre><br />
rsync -v --progress username@beluga.calculcanada.ca:/project/.duc_databases/def-professor.sqlite .<br />
duc gui -d ./def-professor.sqlite /project/def-professor<br />
</pre><br />
<br />
<!--T:18--><br />
This immediately leads to a smoother and more satisfying browsing experience.<br />
<br />
== Space and inode usage per user on cedar == <!--T:19--><br />
<br />
<!--T:20--><br />
On cedar, it is possible for any member of a group to run <code>diskusage_report</code> with the following options <code>--per_user</code> and <code>--all_users</code> to have a break-down usage per user. The first option displays only heavy users. In other terms, members of the group who have more files and/or occupy more space. When both options are used, the command gives the break-down usage for all members of the group. This is a handy command that helps to identify the users within a group who have more files and/or a large amount of data and ask them to better manage their data by reducing their inode usage for example.<br />
<br />
<!--T:21--><br />
In the following example, a user '''user01''' runs the command and got the following output:<br />
<br />
<!--T:22--><br />
<source lang="bash"><br />
[user01@cedar1 ~]$ diskusage_report --per_user --all_users<br />
Description Space # of files<br />
/home (user user01) 109k/50G 12/500k<br />
/scratch (user user01) 4000/20T 1/1000k<br />
/project (group user01) 0/2048k 0/1025<br />
/project (group def-professor) 9434G/10T 497k/500k<br />
<br />
Breakdown for project def-professor (Last update: 2023-05-02 01:03:10)<br />
User File count Size Location<br />
-------------------------------------------------------------------------<br />
user01 28313 4.00 GiB On disk<br />
user02 11926 3.74 GiB On disk<br />
user03 14507 6121.03 GiB On disk<br />
user04 4010 377.86 GiB On disk<br />
user05 125929 262.75 GiB On disk<br />
user06 201099 60.51 GiB On disk<br />
user07 84806 1721.33 GiB On disk<br />
user08 26516 947.23 GiB On disk<br />
Total 497106 9510.43 GiB On disk<br />
<br />
Breakdown for nearline def-professor (Last update: 2023-05-02 01:01:30)<br />
User File count Size Location<br />
-------------------------------------------------------------------------<br />
sbelharb 5 1197.90 GiB On disk and tape<br />
Total 5 1197.90 GiB On disk and tape<br />
<!--T:23--><br />
<br />
<!--T:24--><br />
Breakdown for nearline def-professor (Last update: 2023-05-02 01:01:30)<br />
User File count Size Location<br />
-------------------------------------------------------------------------<br />
user03 5 1197.90 GiB On disk and tape<br />
Total 5 1197.90 GiB On disk and tape<br />
</source><br />
<br />
<!--T:25--><br />
This group has 8 users and the above output shows clearly that at least 4 of them have a large number of files for a small amount of data:<br />
<br />
<!--T:26--><br />
<source lang="bash"><br />
User File count Size Location<br />
-------------------------------------------------------------------------<br />
user01 28313 4.00 GiB On disk<br />
user02 11926 3.74 GiB On disk<br />
user05 125929 262.75 GiB On disk<br />
user06 201099 60.51 GiB On disk<br />
</source><br />
<br />
</translate></div>Kerrachehttps://docs.alliancecan.ca/mediawiki/index.php?title=Diskusage_Explorer&diff=135401Diskusage Explorer2023-05-02T19:48:59Z<p>Kerrache: /* Space and inode usage per user on cedar */</p>
<hr />
<div><languages /><br />
<translate><br />
<br />
==Content of Folders== <!--T:1--><br />
<br />
<!--T:2--><br />
<span style="color:red">Warning: At the moment this feature is only available on [[Béluga/en|Béluga]] and [[Narval/en|Narval]].</span><br />
<br />
<!--T:3--><br />
You can get a breakdown by folder of how the diskspace is being consumed in your home, scratch and project spaces. That information is currently updated once a day and is stored in an [[SQLite]] format for fast access. <br />
<br />
<!--T:4--><br />
Here is a walkthrough of how to explore your disk consumption, using the example of the project space <code>def-professor</code> as the particular directory to investigate.<br />
<br />
=== Ncurse User Interface === <!--T:5--><br />
Choose a project space you have access to and which you wish to analyze, which for the purpose of this discussion we will assume to be <tt>def-professor</tt>.<br />
{{Command|diskusage_explorer /project/def-professor}}<br />
This command loads a browser that shows the resources consumed by all files under any directory tree<br />
[[File:Ncurse duc.png|thumb|using|450px|frame|left| Navigating your project space with duc's ncurse tool]]<br />
<br clear=all> <!-- This is to prevent the next section from filling to the right of the image. --><br />
<br />
<!--T:6--><br />
Type <code>c</code> to toggle between consumed disk space and the number of files, <code>q</code> or <code><esc></code> to quit and <code>h</code> for help.<br />
<br />
<!--T:7--><br />
If you are only interested in a sub-directory of this project space and do not want to navigate the whole tree in the ncurse user interface, <br />
{{Command|diskusage_explorer /project/def-professor/subdirectory/}}<br />
<br />
<!--T:8--><br />
A complete manual page is available with the <code>man duc</code> command.<br />
<br />
=== Graphical User Interface === <!--T:9--><br />
<br />
<!--T:10--><br />
Note that when the login node is especially busy or if you have an especially large amount of files in you project space, the graphical interface mode can be slow and choppy. For a better experience, you can read the section below to run <code>diskusage_explorer</code> on your own machine.<br />
<br />
<!--T:11--><br />
Note that Compute Canada recommends the use of the standard text-based ncurse mode on our cluster login nodes but <code>diskusage_explorer</code> does also include a nice graphical user interface (GUI). <br />
<br />
<!--T:12--><br />
First make sure that you are connected to the cluster in such a way that [[SSH]] is capable of correctly displaying GUI applications. You can then use a graphical interface by means of the command,<br />
{{Command|duc gui -d /project/.duc_databases/def-professor.sqlite /project/def-professor}}<br />
<br />
<!--T:13--><br />
You can navigate the folders with the mouse and still type <code>c</code> to toggle between the size of the files and the number of files.<br />
<br />
<!--T:14--><br />
[[File:Duc gui navigation.gif|thumb|using|450px|frame|left|Navigating your project space with duc's GUI tool]]<br />
<br clear=all> <!-- This is to prevent the next section from filling to the right of the image. --><br />
<br />
=== Browse faster on your own machine === <!--T:15--><br />
<br />
<!--T:16--><br />
First [http://duc.zevv.nl/#download install the diskusage_explorer software] on your local machine and then, still on your local machine, download the SQLite file from your cluster and run <code>duc</code>. <br />
<br />
<!--T:17--><br />
<pre><br />
rsync -v --progress username@beluga.calculcanada.ca:/project/.duc_databases/def-professor.sqlite .<br />
duc gui -d ./def-professor.sqlite /project/def-professor<br />
</pre><br />
<br />
<!--T:18--><br />
This immediately leads to a smoother and more satisfying browsing experience.<br />
<br />
== Space and inode usage per user on cedar ==<br />
<br />
On cedar, it is possible for any member of a group to run <code>diskusage_report</code> with the following options <code>--per_user</code> and <code>--all_users</code> to have a break-down usage per user. The first option displays only heavy users. In other terms, members of the group who have more files and/or occupy more space. When both options are used, the command gives the break-down usage for all members of the group. This is a handy command that helps to identify the users within a group who have more files and/or a large amount of data and ask them to better manage their data by reducing their inode usage for example.<br />
<br />
In the following example, a user '''user01''' runs the command and got the following output:<br />
<br />
<source lang="bash"><br />
[user01@cedar1 ~]$ diskusage_report --per_user --all_users<br />
Description Space # of files<br />
/home (user user01) 109k/50G 12/500k<br />
/scratch (user user01) 4000/20T 1/1000k<br />
/project (group user01) 0/2048k 0/1025<br />
/project (group def-professor) 9434G/10T 4062k/5000k<br />
<br />
Breakdown for project def-professor (Last update: 2023-05-02 01:03:10)<br />
User File count Size Location<br />
-------------------------------------------------------------------------<br />
user01 141567 4.00 GiB On disk<br />
user02 11926 3.74 GiB On disk<br />
user03 725383 6121.03 GiB On disk<br />
user04 4010 377.86 GiB On disk<br />
user05 1296466 262.75 GiB On disk<br />
user06 1010996 60.51 GiB On disk<br />
user07 848066 1721.33 GiB On disk<br />
user08 26516 947.23 GiB On disk<br />
Total 4064930 9510.43 GiB On disk<br />
<br />
Breakdown for nearline def-professor (Last update: 2023-05-02 01:01:30)<br />
User File count Size Location<br />
-------------------------------------------------------------------------<br />
user03 5 1197.90 GiB On disk and tape<br />
Total 5 1197.90 GiB On disk and tape<br />
</source><br />
<br />
This group has 8 users and the above output shows clearly that at least 4 of them have a large number of files for a small amount of data:<br />
<br />
<source lang="bash"><br />
User File count Size Location<br />
-------------------------------------------------------------------------<br />
user01 141567 4.00 GiB On disk<br />
user02 11926 3.74 GiB On disk<br />
user05 1296466 262.75 GiB On disk<br />
user06 1010996 60.51 GiB On disk<br />
</source><br />
<br />
</translate></div>Kerrachehttps://docs.alliancecan.ca/mediawiki/index.php?title=Diskusage_Explorer&diff=135400Diskusage Explorer2023-05-02T19:42:06Z<p>Kerrache: /* Space and inode usage per user on cedar */</p>
<hr />
<div><languages /><br />
<translate><br />
<br />
==Content of Folders== <!--T:1--><br />
<br />
<!--T:2--><br />
<span style="color:red">Warning: At the moment this feature is only available on [[Béluga/en|Béluga]] and [[Narval/en|Narval]].</span><br />
<br />
<!--T:3--><br />
You can get a breakdown by folder of how the diskspace is being consumed in your home, scratch and project spaces. That information is currently updated once a day and is stored in an [[SQLite]] format for fast access. <br />
<br />
<!--T:4--><br />
Here is a walkthrough of how to explore your disk consumption, using the example of the project space <code>def-professor</code> as the particular directory to investigate.<br />
<br />
=== Ncurse User Interface === <!--T:5--><br />
Choose a project space you have access to and which you wish to analyze, which for the purpose of this discussion we will assume to be <tt>def-professor</tt>.<br />
{{Command|diskusage_explorer /project/def-professor}}<br />
This command loads a browser that shows the resources consumed by all files under any directory tree<br />
[[File:Ncurse duc.png|thumb|using|450px|frame|left| Navigating your project space with duc's ncurse tool]]<br />
<br clear=all> <!-- This is to prevent the next section from filling to the right of the image. --><br />
<br />
<!--T:6--><br />
Type <code>c</code> to toggle between consumed disk space and the number of files, <code>q</code> or <code><esc></code> to quit and <code>h</code> for help.<br />
<br />
<!--T:7--><br />
If you are only interested in a sub-directory of this project space and do not want to navigate the whole tree in the ncurse user interface, <br />
{{Command|diskusage_explorer /project/def-professor/subdirectory/}}<br />
<br />
<!--T:8--><br />
A complete manual page is available with the <code>man duc</code> command.<br />
<br />
=== Graphical User Interface === <!--T:9--><br />
<br />
<!--T:10--><br />
Note that when the login node is especially busy or if you have an especially large amount of files in you project space, the graphical interface mode can be slow and choppy. For a better experience, you can read the section below to run <code>diskusage_explorer</code> on your own machine.<br />
<br />
<!--T:11--><br />
Note that Compute Canada recommends the use of the standard text-based ncurse mode on our cluster login nodes but <code>diskusage_explorer</code> does also include a nice graphical user interface (GUI). <br />
<br />
<!--T:12--><br />
First make sure that you are connected to the cluster in such a way that [[SSH]] is capable of correctly displaying GUI applications. You can then use a graphical interface by means of the command,<br />
{{Command|duc gui -d /project/.duc_databases/def-professor.sqlite /project/def-professor}}<br />
<br />
<!--T:13--><br />
You can navigate the folders with the mouse and still type <code>c</code> to toggle between the size of the files and the number of files.<br />
<br />
<!--T:14--><br />
[[File:Duc gui navigation.gif|thumb|using|450px|frame|left|Navigating your project space with duc's GUI tool]]<br />
<br clear=all> <!-- This is to prevent the next section from filling to the right of the image. --><br />
<br />
=== Browse faster on your own machine === <!--T:15--><br />
<br />
<!--T:16--><br />
First [http://duc.zevv.nl/#download install the diskusage_explorer software] on your local machine and then, still on your local machine, download the SQLite file from your cluster and run <code>duc</code>. <br />
<br />
<!--T:17--><br />
<pre><br />
rsync -v --progress username@beluga.calculcanada.ca:/project/.duc_databases/def-professor.sqlite .<br />
duc gui -d ./def-professor.sqlite /project/def-professor<br />
</pre><br />
<br />
<!--T:18--><br />
This immediately leads to a smoother and more satisfying browsing experience.<br />
<br />
== Space and inode usage per user on cedar ==<br />
<br />
On cedar, it is possible for any member of a group to run <code>diskusage_report</code> with the following options <code>--per_user</code> and <code>--all_users</code> to have a break-down usage per user. The first option displays only heavy users. In other terms, members of the group who have more files and/or occupy more space. When both options are used, the command gives the break-down usage for all members of the group. This is a handy command that helps to identify the users within a group who have more files and/or a large amount of data and ask them to better manage their data by reducing their inode usage for example.<br />
<br />
In the following example, a user '''user01''' runs the command and got the following output:<br />
<br />
<source lang="bash"><br />
[user01@cedar1 ~]$ diskusage_report --per_user --all_users<br />
Description Space # of files<br />
/home (user user01) 109k/50G 12/500k<br />
/scratch (user user01) 4000/20T 1/1000k<br />
/project (group user01) 0/2048k 0/1025<br />
/project (group def-professor) 9434G/10T 4062k/5000k<br />
<br />
Breakdown for project def-professor (Last update: 2023-05-02 01:03:10)<br />
User File count Size Location<br />
-------------------------------------------------------------------------<br />
user01 141567 4.00 GiB On disk<br />
user02 11926 3.74 GiB On disk<br />
user03 725383 6121.03 GiB On disk<br />
user04 4010 377.86 GiB On disk<br />
user05 1296466 262.75 GiB On disk<br />
user06 1010996 60.51 GiB On disk<br />
user07 848066 1721.33 GiB On disk<br />
user08 26516 947.23 GiB On disk<br />
Total 4064930 9510.43 GiB On disk<br />
<br />
Breakdown for nearline def-professor (Last update: 2023-05-02 01:01:30)<br />
User File count Size Location<br />
-------------------------------------------------------------------------<br />
user03 5 1197.90 GiB On disk and tape<br />
Total 5 1197.90 GiB On disk and tape<br />
</source><br />
<br />
This group has 8 users and the output shows clearly that at least 4 of them have a large number of files for a small amount of data:<br />
<br />
<source lang="bash"><br />
User File count Size Location<br />
-------------------------------------------------------------------------<br />
user01 141567 4.00 GiB On disk<br />
user02 11926 3.74 GiB On disk<br />
user05 1296466 262.75 GiB On disk<br />
user06 1010996 60.51 GiB On disk<br />
</source><br />
<br />
</translate></div>Kerrachehttps://docs.alliancecan.ca/mediawiki/index.php?title=Diskusage_Explorer&diff=135399Diskusage Explorer2023-05-02T19:40:53Z<p>Kerrache: /* Space and inode usage per user on cedar */</p>
<hr />
<div><languages /><br />
<translate><br />
<br />
==Content of Folders== <!--T:1--><br />
<br />
<!--T:2--><br />
<span style="color:red">Warning: At the moment this feature is only available on [[Béluga/en|Béluga]] and [[Narval/en|Narval]].</span><br />
<br />
<!--T:3--><br />
You can get a breakdown by folder of how the diskspace is being consumed in your home, scratch and project spaces. That information is currently updated once a day and is stored in an [[SQLite]] format for fast access. <br />
<br />
<!--T:4--><br />
Here is a walkthrough of how to explore your disk consumption, using the example of the project space <code>def-professor</code> as the particular directory to investigate.<br />
<br />
=== Ncurse User Interface === <!--T:5--><br />
Choose a project space you have access to and which you wish to analyze, which for the purpose of this discussion we will assume to be <tt>def-professor</tt>.<br />
{{Command|diskusage_explorer /project/def-professor}}<br />
This command loads a browser that shows the resources consumed by all files under any directory tree<br />
[[File:Ncurse duc.png|thumb|using|450px|frame|left| Navigating your project space with duc's ncurse tool]]<br />
<br clear=all> <!-- This is to prevent the next section from filling to the right of the image. --><br />
<br />
<!--T:6--><br />
Type <code>c</code> to toggle between consumed disk space and the number of files, <code>q</code> or <code><esc></code> to quit and <code>h</code> for help.<br />
<br />
<!--T:7--><br />
If you are only interested in a sub-directory of this project space and do not want to navigate the whole tree in the ncurse user interface, <br />
{{Command|diskusage_explorer /project/def-professor/subdirectory/}}<br />
<br />
<!--T:8--><br />
A complete manual page is available with the <code>man duc</code> command.<br />
<br />
=== Graphical User Interface === <!--T:9--><br />
<br />
<!--T:10--><br />
Note that when the login node is especially busy or if you have an especially large amount of files in you project space, the graphical interface mode can be slow and choppy. For a better experience, you can read the section below to run <code>diskusage_explorer</code> on your own machine.<br />
<br />
<!--T:11--><br />
Note that Compute Canada recommends the use of the standard text-based ncurse mode on our cluster login nodes but <code>diskusage_explorer</code> does also include a nice graphical user interface (GUI). <br />
<br />
<!--T:12--><br />
First make sure that you are connected to the cluster in such a way that [[SSH]] is capable of correctly displaying GUI applications. You can then use a graphical interface by means of the command,<br />
{{Command|duc gui -d /project/.duc_databases/def-professor.sqlite /project/def-professor}}<br />
<br />
<!--T:13--><br />
You can navigate the folders with the mouse and still type <code>c</code> to toggle between the size of the files and the number of files.<br />
<br />
<!--T:14--><br />
[[File:Duc gui navigation.gif|thumb|using|450px|frame|left|Navigating your project space with duc's GUI tool]]<br />
<br clear=all> <!-- This is to prevent the next section from filling to the right of the image. --><br />
<br />
=== Browse faster on your own machine === <!--T:15--><br />
<br />
<!--T:16--><br />
First [http://duc.zevv.nl/#download install the diskusage_explorer software] on your local machine and then, still on your local machine, download the SQLite file from your cluster and run <code>duc</code>. <br />
<br />
<!--T:17--><br />
<pre><br />
rsync -v --progress username@beluga.calculcanada.ca:/project/.duc_databases/def-professor.sqlite .<br />
duc gui -d ./def-professor.sqlite /project/def-professor<br />
</pre><br />
<br />
<!--T:18--><br />
This immediately leads to a smoother and more satisfying browsing experience.<br />
<br />
== Space and inode usage per user on cedar ==<br />
<br />
On cedar, it is possible for any member of a group to run <code>diskusage_report</code> with the following options <code>--per_user</code> and <code>--all_users</code> to have a break-down usage per user. The first option displays only heavy users. In other terms, members of the group who have more files and/or occupy more space. When both options are used, the command gives the break-down usage for all members of the group. This is a handy command that helps to identify the users within a group who have more files and/or a large amount of data and ask them to better manage their data by reducing their inode usage for example.<br />
<br />
In the following example, a user '''user01''' runs the command and got the following output:<br />
<br />
<source lang="bash"><br />
[user01@cedar1 ~]$ diskusage_report --per_user --all_users<br />
Description Space # of files<br />
/home (user user01) 109k/50G 12/500k<br />
/scratch (user user01) 4000/20T 1/1000k<br />
/project (group user01) 0/2048k 0/1025<br />
/project (group def-professor) 9434G/10T 4062k/5000k<br />
<br />
Breakdown for project def-professor (Last update: 2023-05-02 01:03:10)<br />
User File count Size Location<br />
-------------------------------------------------------------------------<br />
user01 141567 4.00 GiB On disk<br />
user02 11926 3.74 GiB On disk<br />
user03 725383 6121.03 GiB On disk<br />
user04 4010 377.86 GiB On disk<br />
user05 1296466 262.75 GiB On disk<br />
user06 1010996 60.51 GiB On disk<br />
user07 848066 1721.33 GiB On disk<br />
user08 26516 947.23 GiB On disk<br />
Total 4064930 9510.43 GiB On disk<br />
<br />
Breakdown for nearline def-professor (Last update: 2023-05-02 01:01:30)<br />
User File count Size Location<br />
-------------------------------------------------------------------------<br />
sbelharb 5 1197.90 GiB On disk and tape<br />
Total 5 1197.90 GiB On disk and tape<br />
</source><br />
<br />
This group has 8 users and the output shows clearly that at least 4 of them have a large number of files for a small amount of data:<br />
<br />
<source lang="bash"><br />
User File count Size Location<br />
-------------------------------------------------------------------------<br />
user01 141567 4.00 GiB On disk<br />
user02 11926 3.74 GiB On disk<br />
user05 1296466 262.75 GiB On disk<br />
user06 1010996 60.51 GiB On disk<br />
</source><br />
<br />
</translate></div>Kerrachehttps://docs.alliancecan.ca/mediawiki/index.php?title=Diskusage_Explorer&diff=135398Diskusage Explorer2023-05-02T19:37:41Z<p>Kerrache: /* Space and inode usage per user on cedar */</p>
<hr />
<div><languages /><br />
<translate><br />
<br />
==Content of Folders== <!--T:1--><br />
<br />
<!--T:2--><br />
<span style="color:red">Warning: At the moment this feature is only available on [[Béluga/en|Béluga]] and [[Narval/en|Narval]].</span><br />
<br />
<!--T:3--><br />
You can get a breakdown by folder of how the diskspace is being consumed in your home, scratch and project spaces. That information is currently updated once a day and is stored in an [[SQLite]] format for fast access. <br />
<br />
<!--T:4--><br />
Here is a walkthrough of how to explore your disk consumption, using the example of the project space <code>def-professor</code> as the particular directory to investigate.<br />
<br />
=== Ncurse User Interface === <!--T:5--><br />
Choose a project space you have access to and which you wish to analyze, which for the purpose of this discussion we will assume to be <tt>def-professor</tt>.<br />
{{Command|diskusage_explorer /project/def-professor}}<br />
This command loads a browser that shows the resources consumed by all files under any directory tree<br />
[[File:Ncurse duc.png|thumb|using|450px|frame|left| Navigating your project space with duc's ncurse tool]]<br />
<br clear=all> <!-- This is to prevent the next section from filling to the right of the image. --><br />
<br />
<!--T:6--><br />
Type <code>c</code> to toggle between consumed disk space and the number of files, <code>q</code> or <code><esc></code> to quit and <code>h</code> for help.<br />
<br />
<!--T:7--><br />
If you are only interested in a sub-directory of this project space and do not want to navigate the whole tree in the ncurse user interface, <br />
{{Command|diskusage_explorer /project/def-professor/subdirectory/}}<br />
<br />
<!--T:8--><br />
A complete manual page is available with the <code>man duc</code> command.<br />
<br />
=== Graphical User Interface === <!--T:9--><br />
<br />
<!--T:10--><br />
Note that when the login node is especially busy or if you have an especially large amount of files in you project space, the graphical interface mode can be slow and choppy. For a better experience, you can read the section below to run <code>diskusage_explorer</code> on your own machine.<br />
<br />
<!--T:11--><br />
Note that Compute Canada recommends the use of the standard text-based ncurse mode on our cluster login nodes but <code>diskusage_explorer</code> does also include a nice graphical user interface (GUI). <br />
<br />
<!--T:12--><br />
First make sure that you are connected to the cluster in such a way that [[SSH]] is capable of correctly displaying GUI applications. You can then use a graphical interface by means of the command,<br />
{{Command|duc gui -d /project/.duc_databases/def-professor.sqlite /project/def-professor}}<br />
<br />
<!--T:13--><br />
You can navigate the folders with the mouse and still type <code>c</code> to toggle between the size of the files and the number of files.<br />
<br />
<!--T:14--><br />
[[File:Duc gui navigation.gif|thumb|using|450px|frame|left|Navigating your project space with duc's GUI tool]]<br />
<br clear=all> <!-- This is to prevent the next section from filling to the right of the image. --><br />
<br />
=== Browse faster on your own machine === <!--T:15--><br />
<br />
<!--T:16--><br />
First [http://duc.zevv.nl/#download install the diskusage_explorer software] on your local machine and then, still on your local machine, download the SQLite file from your cluster and run <code>duc</code>. <br />
<br />
<!--T:17--><br />
<pre><br />
rsync -v --progress username@beluga.calculcanada.ca:/project/.duc_databases/def-professor.sqlite .<br />
duc gui -d ./def-professor.sqlite /project/def-professor<br />
</pre><br />
<br />
<!--T:18--><br />
This immediately leads to a smoother and more satisfying browsing experience.<br />
<br />
== Space and inode usage per user on cedar ==<br />
<br />
On cedar, it is possible for any member of a group to run <code>diskusage_report</code> with the following options <code>--per_user</code> and <code>--all_users</code> to have a break-down usage per user. The first option displays only heavy users. In other terms, members of the group who have more files and/or occupy more space. When both options are used, the command gives the break-down usage for all members of the group. This is a handy command that helps to identify the users within a group who have more files and/or a large amount of data and ask them to better manage their data by reducing their inode usage for example.<br />
<br />
In the following example, a user **user01** run the command and got the following output:<br />
<br />
<source lang="bash"><br />
[user01@cedar1 ~]$ diskusage_report --per_user --all_users<br />
Description Space # of files<br />
/home (user user01) 109k/50G 12/500k<br />
/scratch (user user01) 4000/20T 1/1000k<br />
/project (group user01) 0/2048k 0/1025<br />
/project (group def-professor) 9434G/10T 4062k/5000k<br />
<br />
Breakdown for project def-professor (Last update: 2023-05-02 01:03:10)<br />
User File count Size Location<br />
-------------------------------------------------------------------------<br />
user01 141567 4.00 GiB On disk<br />
user02 11926 3.74 GiB On disk<br />
user03 725383 6121.03 GiB On disk<br />
user04 4010 377.86 GiB On disk<br />
user05 1296466 262.75 GiB On disk<br />
user06 1010996 60.51 GiB On disk<br />
user07 848066 1721.33 GiB On disk<br />
user08 26516 947.23 GiB On disk<br />
Total 4064930 9510.43 GiB On disk<br />
<br />
Breakdown for nearline def-professor (Last update: 2023-05-02 01:01:30)<br />
User File count Size Location<br />
-------------------------------------------------------------------------<br />
sbelharb 5 1197.90 GiB On disk and tape<br />
Total 5 1197.90 GiB On disk and tape<br />
</source><br />
<br />
</translate></div>Kerrachehttps://docs.alliancecan.ca/mediawiki/index.php?title=Diskusage_Explorer&diff=135397Diskusage Explorer2023-05-02T19:36:19Z<p>Kerrache: /* Space and inode usage per user on cedar */</p>
<hr />
<div><languages /><br />
<translate><br />
<br />
==Content of Folders== <!--T:1--><br />
<br />
<!--T:2--><br />
<span style="color:red">Warning: At the moment this feature is only available on [[Béluga/en|Béluga]] and [[Narval/en|Narval]].</span><br />
<br />
<!--T:3--><br />
You can get a breakdown by folder of how the diskspace is being consumed in your home, scratch and project spaces. That information is currently updated once a day and is stored in an [[SQLite]] format for fast access. <br />
<br />
<!--T:4--><br />
Here is a walkthrough of how to explore your disk consumption, using the example of the project space <code>def-professor</code> as the particular directory to investigate.<br />
<br />
=== Ncurse User Interface === <!--T:5--><br />
Choose a project space you have access to and which you wish to analyze, which for the purpose of this discussion we will assume to be <tt>def-professor</tt>.<br />
{{Command|diskusage_explorer /project/def-professor}}<br />
This command loads a browser that shows the resources consumed by all files under any directory tree<br />
[[File:Ncurse duc.png|thumb|using|450px|frame|left| Navigating your project space with duc's ncurse tool]]<br />
<br clear=all> <!-- This is to prevent the next section from filling to the right of the image. --><br />
<br />
<!--T:6--><br />
Type <code>c</code> to toggle between consumed disk space and the number of files, <code>q</code> or <code><esc></code> to quit and <code>h</code> for help.<br />
<br />
<!--T:7--><br />
If you are only interested in a sub-directory of this project space and do not want to navigate the whole tree in the ncurse user interface, <br />
{{Command|diskusage_explorer /project/def-professor/subdirectory/}}<br />
<br />
<!--T:8--><br />
A complete manual page is available with the <code>man duc</code> command.<br />
<br />
=== Graphical User Interface === <!--T:9--><br />
<br />
<!--T:10--><br />
Note that when the login node is especially busy or if you have an especially large amount of files in you project space, the graphical interface mode can be slow and choppy. For a better experience, you can read the section below to run <code>diskusage_explorer</code> on your own machine.<br />
<br />
<!--T:11--><br />
Note that Compute Canada recommends the use of the standard text-based ncurse mode on our cluster login nodes but <code>diskusage_explorer</code> does also include a nice graphical user interface (GUI). <br />
<br />
<!--T:12--><br />
First make sure that you are connected to the cluster in such a way that [[SSH]] is capable of correctly displaying GUI applications. You can then use a graphical interface by means of the command,<br />
{{Command|duc gui -d /project/.duc_databases/def-professor.sqlite /project/def-professor}}<br />
<br />
<!--T:13--><br />
You can navigate the folders with the mouse and still type <code>c</code> to toggle between the size of the files and the number of files.<br />
<br />
<!--T:14--><br />
[[File:Duc gui navigation.gif|thumb|using|450px|frame|left|Navigating your project space with duc's GUI tool]]<br />
<br clear=all> <!-- This is to prevent the next section from filling to the right of the image. --><br />
<br />
=== Browse faster on your own machine === <!--T:15--><br />
<br />
<!--T:16--><br />
First [http://duc.zevv.nl/#download install the diskusage_explorer software] on your local machine and then, still on your local machine, download the SQLite file from your cluster and run <code>duc</code>. <br />
<br />
<!--T:17--><br />
<pre><br />
rsync -v --progress username@beluga.calculcanada.ca:/project/.duc_databases/def-professor.sqlite .<br />
duc gui -d ./def-professor.sqlite /project/def-professor<br />
</pre><br />
<br />
<!--T:18--><br />
This immediately leads to a smoother and more satisfying browsing experience.<br />
<br />
== Space and inode usage per user on cedar ==<br />
<br />
On cedar, it is possible for any member of a group to run <code>diskusage_report</code> with the following options <code>--per_user</code> and <code>--all_users</code> to have a break-down usage per user. The first option displays only heavy users. In other terms, members of the group who have more files and/or occupy more space. When both options are used, the command gives the break-down usage for all members of the group. This is a handy command that helps to identify the users within a group who have more files and/or a large amount of data and ask them to better manage their data by reducing their inode usage for example.<br />
<br />
<source lang="bash"><br />
[user01@cedar1 ~]$ diskusage_report --per_user --all_users<br />
Description Space # of files<br />
/home (user user01) 109k/50G 12/500k<br />
/scratch (user user01) 4000/20T 1/1000k<br />
/project (group user01) 0/2048k 0/1025<br />
/project (group def-professor) 9434G/10T 4062k/5000k<br />
<br />
Breakdown for project def-professor (Last update: 2023-05-02 01:03:10)<br />
User File count Size Location<br />
-------------------------------------------------------------------------<br />
user01 141567 4.00 GiB On disk<br />
user02 11926 3.74 GiB On disk<br />
user03 725383 6121.03 GiB On disk<br />
user04 4010 377.86 GiB On disk<br />
user05 1296466 262.75 GiB On disk<br />
user06 1010996 60.51 GiB On disk<br />
user07 848066 1721.33 GiB On disk<br />
user08 26516 947.23 GiB On disk<br />
Total 4064930 9510.43 GiB On disk<br />
<br />
Breakdown for nearline def-professor (Last update: 2023-05-02 01:01:30)<br />
User File count Size Location<br />
-------------------------------------------------------------------------<br />
sbelharb 5 1197.90 GiB On disk and tape<br />
Total 5 1197.90 GiB On disk and tape<br />
</source><br />
<br />
</translate></div>Kerrachehttps://docs.alliancecan.ca/mediawiki/index.php?title=Diskusage_Explorer&diff=135396Diskusage Explorer2023-05-02T19:35:02Z<p>Kerrache: /* Space and inode usage per user on cedar */</p>
<hr />
<div><languages /><br />
<translate><br />
<br />
==Content of Folders== <!--T:1--><br />
<br />
<!--T:2--><br />
<span style="color:red">Warning: At the moment this feature is only available on [[Béluga/en|Béluga]] and [[Narval/en|Narval]].</span><br />
<br />
<!--T:3--><br />
You can get a breakdown by folder of how the diskspace is being consumed in your home, scratch and project spaces. That information is currently updated once a day and is stored in an [[SQLite]] format for fast access. <br />
<br />
<!--T:4--><br />
Here is a walkthrough of how to explore your disk consumption, using the example of the project space <code>def-professor</code> as the particular directory to investigate.<br />
<br />
=== Ncurse User Interface === <!--T:5--><br />
Choose a project space you have access to and which you wish to analyze, which for the purpose of this discussion we will assume to be <tt>def-professor</tt>.<br />
{{Command|diskusage_explorer /project/def-professor}}<br />
This command loads a browser that shows the resources consumed by all files under any directory tree<br />
[[File:Ncurse duc.png|thumb|using|450px|frame|left| Navigating your project space with duc's ncurse tool]]<br />
<br clear=all> <!-- This is to prevent the next section from filling to the right of the image. --><br />
<br />
<!--T:6--><br />
Type <code>c</code> to toggle between consumed disk space and the number of files, <code>q</code> or <code><esc></code> to quit and <code>h</code> for help.<br />
<br />
<!--T:7--><br />
If you are only interested in a sub-directory of this project space and do not want to navigate the whole tree in the ncurse user interface, <br />
{{Command|diskusage_explorer /project/def-professor/subdirectory/}}<br />
<br />
<!--T:8--><br />
A complete manual page is available with the <code>man duc</code> command.<br />
<br />
=== Graphical User Interface === <!--T:9--><br />
<br />
<!--T:10--><br />
Note that when the login node is especially busy or if you have an especially large amount of files in you project space, the graphical interface mode can be slow and choppy. For a better experience, you can read the section below to run <code>diskusage_explorer</code> on your own machine.<br />
<br />
<!--T:11--><br />
Note that Compute Canada recommends the use of the standard text-based ncurse mode on our cluster login nodes but <code>diskusage_explorer</code> does also include a nice graphical user interface (GUI). <br />
<br />
<!--T:12--><br />
First make sure that you are connected to the cluster in such a way that [[SSH]] is capable of correctly displaying GUI applications. You can then use a graphical interface by means of the command,<br />
{{Command|duc gui -d /project/.duc_databases/def-professor.sqlite /project/def-professor}}<br />
<br />
<!--T:13--><br />
You can navigate the folders with the mouse and still type <code>c</code> to toggle between the size of the files and the number of files.<br />
<br />
<!--T:14--><br />
[[File:Duc gui navigation.gif|thumb|using|450px|frame|left|Navigating your project space with duc's GUI tool]]<br />
<br clear=all> <!-- This is to prevent the next section from filling to the right of the image. --><br />
<br />
=== Browse faster on your own machine === <!--T:15--><br />
<br />
<!--T:16--><br />
First [http://duc.zevv.nl/#download install the diskusage_explorer software] on your local machine and then, still on your local machine, download the SQLite file from your cluster and run <code>duc</code>. <br />
<br />
<!--T:17--><br />
<pre><br />
rsync -v --progress username@beluga.calculcanada.ca:/project/.duc_databases/def-professor.sqlite .<br />
duc gui -d ./def-professor.sqlite /project/def-professor<br />
</pre><br />
<br />
<!--T:18--><br />
This immediately leads to a smoother and more satisfying browsing experience.<br />
<br />
== Space and inode usage per user on cedar ==<br />
<br />
On cedar, it is possible for any member of a group to run <code>diskusage_report</code> with the following options <code>--per_user</code> and <code>--all_users</code> to have a break-down usage per user. The first option displays only heavy users. In other terms, members of the group who have more files and/or occupy more space. When both options are used, the command gives the break-down usage for all members of the group. This is a handy command that helps to identify the users within a group who have more files and/or a large amount of data and ask them to better manage their data by reducing their inode usage for example.<br />
<br />
<table><br />
[user01@cedar1 ~]$ diskusage_report --per_user --all_users<br />
Description Space # of files<br />
/home (user user01) 109k/50G 12/500k<br />
/scratch (user user01) 4000/20T 1/1000k<br />
/project (group user01) 0/2048k 0/1025<br />
/project (group def-professor) 9434G/10T 4062k/5000k<br />
<br />
Breakdown for project def-professor (Last update: 2023-05-02 01:03:10)<br />
User File count Size Location<br />
-------------------------------------------------------------------------<br />
user01 141567 4.00 GiB On disk<br />
user02 11926 3.74 GiB On disk<br />
user03 725383 6121.03 GiB On disk<br />
user04 4010 377.86 GiB On disk<br />
user05 1296466 262.75 GiB On disk<br />
user06 1010996 60.51 GiB On disk<br />
user07 848066 1721.33 GiB On disk<br />
user08 26516 947.23 GiB On disk<br />
Total 4064930 9510.43 GiB On disk<br />
<br />
Breakdown for nearline def-professor (Last update: 2023-05-02 01:01:30)<br />
User File count Size Location<br />
-------------------------------------------------------------------------<br />
sbelharb 5 1197.90 GiB On disk and tape<br />
Total 5 1197.90 GiB On disk and tape<br />
</table> <br />
<br />
</translate></div>Kerrachehttps://docs.alliancecan.ca/mediawiki/index.php?title=Diskusage_Explorer&diff=135395Diskusage Explorer2023-05-02T19:33:21Z<p>Kerrache: diskusage_report on cedar</p>
<hr />
<div><languages /><br />
<translate><br />
<br />
==Content of Folders== <!--T:1--><br />
<br />
<!--T:2--><br />
<span style="color:red">Warning: At the moment this feature is only available on [[Béluga/en|Béluga]] and [[Narval/en|Narval]].</span><br />
<br />
<!--T:3--><br />
You can get a breakdown by folder of how the diskspace is being consumed in your home, scratch and project spaces. That information is currently updated once a day and is stored in an [[SQLite]] format for fast access. <br />
<br />
<!--T:4--><br />
Here is a walkthrough of how to explore your disk consumption, using the example of the project space <code>def-professor</code> as the particular directory to investigate.<br />
<br />
=== Ncurse User Interface === <!--T:5--><br />
Choose a project space you have access to and which you wish to analyze, which for the purpose of this discussion we will assume to be <tt>def-professor</tt>.<br />
{{Command|diskusage_explorer /project/def-professor}}<br />
This command loads a browser that shows the resources consumed by all files under any directory tree<br />
[[File:Ncurse duc.png|thumb|using|450px|frame|left| Navigating your project space with duc's ncurse tool]]<br />
<br clear=all> <!-- This is to prevent the next section from filling to the right of the image. --><br />
<br />
<!--T:6--><br />
Type <code>c</code> to toggle between consumed disk space and the number of files, <code>q</code> or <code><esc></code> to quit and <code>h</code> for help.<br />
<br />
<!--T:7--><br />
If you are only interested in a sub-directory of this project space and do not want to navigate the whole tree in the ncurse user interface, <br />
{{Command|diskusage_explorer /project/def-professor/subdirectory/}}<br />
<br />
<!--T:8--><br />
A complete manual page is available with the <code>man duc</code> command.<br />
<br />
=== Graphical User Interface === <!--T:9--><br />
<br />
<!--T:10--><br />
Note that when the login node is especially busy or if you have an especially large amount of files in you project space, the graphical interface mode can be slow and choppy. For a better experience, you can read the section below to run <code>diskusage_explorer</code> on your own machine.<br />
<br />
<!--T:11--><br />
Note that Compute Canada recommends the use of the standard text-based ncurse mode on our cluster login nodes but <code>diskusage_explorer</code> does also include a nice graphical user interface (GUI). <br />
<br />
<!--T:12--><br />
First make sure that you are connected to the cluster in such a way that [[SSH]] is capable of correctly displaying GUI applications. You can then use a graphical interface by means of the command,<br />
{{Command|duc gui -d /project/.duc_databases/def-professor.sqlite /project/def-professor}}<br />
<br />
<!--T:13--><br />
You can navigate the folders with the mouse and still type <code>c</code> to toggle between the size of the files and the number of files.<br />
<br />
<!--T:14--><br />
[[File:Duc gui navigation.gif|thumb|using|450px|frame|left|Navigating your project space with duc's GUI tool]]<br />
<br clear=all> <!-- This is to prevent the next section from filling to the right of the image. --><br />
<br />
=== Browse faster on your own machine === <!--T:15--><br />
<br />
<!--T:16--><br />
First [http://duc.zevv.nl/#download install the diskusage_explorer software] on your local machine and then, still on your local machine, download the SQLite file from your cluster and run <code>duc</code>. <br />
<br />
<!--T:17--><br />
<pre><br />
rsync -v --progress username@beluga.calculcanada.ca:/project/.duc_databases/def-professor.sqlite .<br />
duc gui -d ./def-professor.sqlite /project/def-professor<br />
</pre><br />
<br />
<!--T:18--><br />
This immediately leads to a smoother and more satisfying browsing experience.<br />
<br />
== Space and inode usage per user on cedar ==<br />
<br />
On cedar, it is possible for any member of a group to run <code>diskusage_report</code> with the following options <code>--per_user</code> and <code>--all_users<code> to have a break-down usage per user. The first option gives displays only heavy users. In other terms, members of the group who have more files and/or occupy more space. When both options are used, the command gives the break-down usage for all members of the group. This is a handy command that helps to identify the users within a group who have more files and/or a large amount of data and ask them to better manage their data by reducing their inode usage for example.<br />
<br />
<table><br />
[user01@cedar1 ~]$ diskusage_report --per_user --all_users<br />
Description Space # of files<br />
/home (user user01) 109k/50G 12/500k<br />
/scratch (user user01) 4000/20T 1/1000k<br />
/project (group user01) 0/2048k 0/1025<br />
/project (group def-professor) 9434G/10T 4062k/5000k<br />
<br />
Breakdown for project def-professor (Last update: 2023-05-02 01:03:10)<br />
User File count Size Location<br />
-------------------------------------------------------------------------<br />
user01 141567 4.00 GiB On disk<br />
user02 11926 3.74 GiB On disk<br />
user03 725383 6121.03 GiB On disk<br />
user04 4010 377.86 GiB On disk<br />
user05 1296466 262.75 GiB On disk<br />
user06 1010996 60.51 GiB On disk<br />
user07 848066 1721.33 GiB On disk<br />
user08 26516 947.23 GiB On disk<br />
Total 4064930 9510.43 GiB On disk<br />
<br />
Breakdown for nearline def-professor (Last update: 2023-05-02 01:01:30)<br />
User File count Size Location<br />
-------------------------------------------------------------------------<br />
sbelharb 5 1197.90 GiB On disk and tape<br />
Total 5 1197.90 GiB On disk and tape<br />
</table> <br />
<br />
</translate></div>Kerrachehttps://docs.alliancecan.ca/mediawiki/index.php?title=CFOUR&diff=131465CFOUR2023-03-13T14:51:16Z<p>Kerrache: Switch the license from Compute Canada to the Alliance.</p>
<hr />
<div><languages/><br />
[[Category:Software]]<br />
<translate><br />
<br />
= Introduction = <!--T:1--><br />
<br />
<!--T:2--><br />
"'''CFOUR''' (Coupled-Cluster techniques for Computational Chemistry) is a program package for performing high-level quantum chemical calculations on atoms and molecules. The major strength of the program suite is its rather sophisticated arsenal of high-level ''ab-initio'' methods for the calculation of atomic and molecular properties. Virtually all approaches based on Møller-Plesset (MP) perturbation theory and the coupled-cluster approximation (CC) are available; most of these have complementary analytic derivative approaches within the package as well."<br />
<br />
<!--T:3--><br />
"'''CFOUR''' is not a commercial code. It is rather a program that is undergoing development; new techniques and improvements are constantly being made." See [http://slater.chemie.uni-mainz.de/cfour/index.php?n=Main.HomePage the CFOUR web site] for more information.<br />
<br />
= License limitations = <!--T:4--><br />
<br />
<!--T:5--><br />
The Alliance has signed a [http://slater.chemie.uni-mainz.de/cfour/index.php?n=Main.Download license] agreement with [https://www.tc.uni-mainz.de/prof-dr-juergen-gauss/ Prof. Dr. J. Gauss] who acts for the developers of the CFOUR Software. <br />
<br />
<!--T:6--><br />
In order to use the current installed version on the Alliance systems, each user must agree to certain conditions. Please [[Technical_support | contact support]] with a copy of the following statement:<br />
<br />
<!--T:7--><br />
# I will use CFOUR only for academic research.<br />
# I will not copy the CFOUR software, nor make it available to anyone else.<br />
# I will properly acknowledge original papers related to CFOUR and to the Alliance in my publications (see the license form for more details).<br />
# I understand that the agreement for using CFOUR can be terminated by one of the parties: CFOUR developers or the Alliance.<br />
# I will notify the Alliance of any change in the above acknowledgement.<br />
<br />
<!--T:8--><br />
When your statement is received, we will allow you to access the program.<br />
<br />
= Module = <!--T:9--><br />
<br />
<!--T:10--><br />
You can access the MPI version of CFOUR by loading a [[Utiliser des modules/en|module]]. <br />
<br />
<!--T:11--><br />
<source lang="bash"><br />
module load intel/2020.1.217 openmpi/4.0.3 cfour-mpi/2.1<br />
</source><br />
<br />
<!--T:17--><br />
For the serial version, use:<br />
<br />
<!--T:18--><br />
<source lang="bash"><br />
module load intel/2020.1.217 cfour/2.1<br />
</source><br />
<br />
<!--T:14--><br />
There is a mailing list as a forum for user experiences with the CFOUR program system. For how to subscribe and other information, see [http://slater.chemie.uni-mainz.de/cfour/index.php?n=Main.MailingList this page].<br />
<br />
== Examples and job scripts == <!--T:33--><br />
<br />
<!--T:34--><br />
To run CFOUR, you need to have at least the input file [http://slater.chemie.uni-mainz.de/cfour/index.php?n=Main.InputFileZMAT ZMAT] with all information concerning geometry, requested quantum-chemical method, basis set, etc. The second file is [http://slater.chemie.uni-mainz.de/cfour/index.php?n=Main.Basis-setFileGENBAS GENBAS] that contains the required information for the basis sets available to the user. If GENBAS is not present in the directory from where you start your job, CFOUR will create a symlink and use the existing file provided by the module. The file is located at: <code>$EBROOTCFOUR/basis/GENBAS</code>.<br />
<br />
<!--T:35--><br />
<tabs><br />
<tab name="INPUT"><br />
{{File<br />
|name=ZMAT<br />
|lang="txt"<br />
|contents=<br />
Acetylene, CCSD/DZP excited-state geometry optimization<br />
C <br />
C 1 RCC*<br />
H 1 RCH* 2 A*<br />
H 2 RCH* 1 A* 3 D180<br />
<br />
RCC=1.36<br />
RCH=1.08<br />
A=124.<br />
D180=180.<br />
<br />
*ACES2(CALC=CCSD,BASIS=DZP,EXCITE=EOMEE <br />
ESTATE_CONV=10,CONV=10,SCF_CONV=10,CC_CONV=10,LINEQ_CONV=10,ZETA_CONV=10) <br />
<br />
%excite* <br />
1 <br />
1 <br />
1 7 0 8 0 1.0 <br />
<br />
<!--T:36--><br />
}}<br />
</tab><br />
<br />
<!--T:37--><br />
<tab name="Serial job"><br />
{{File<br />
|name=run_cfour_serial.sh<br />
|lang="bash"<br />
|contents=<br />
#!/bin/bash<br />
#SBATCH --account=def-someacct # replace this with your own account<br />
#SBATCH --ntasks=1<br />
#SBATCH --mem-per-cpu=2500M # memory; default unit is megabytes.<br />
#SBATCH --time=0-00:30 # time (DD-HH:MM).<br />
<br />
<!--T:38--><br />
# Load the module:<br />
<br />
<!--T:39--><br />
module load intel/2020.1.217 cfour/2.1<br />
<br />
<!--T:40--><br />
echo "Starting run at: `date`"<br />
<br />
<!--T:41--><br />
CFOUROUTPUT="cfour-output.txt"<br />
export CFOUR_NUM_CORES=1<br />
<br />
<!--T:42--><br />
xcfour > ${CFOUROUTPUT} <br />
<br />
<!--T:43--><br />
# Clean the symlink:<br />
if [[ -L "GENBAS" ]]; then unlink GENBAS; fi<br />
<br />
<!--T:44--><br />
echo "Program finished with exit code $? at: `date`"<br />
}}<br />
</tab><br />
<br />
<!--T:45--><br />
<tab name="MPI job"><br />
{{File<br />
|name=run-cfour-mpi.sh<br />
|lang="bash"<br />
|contents=<br />
#!/bin/bash<br />
#SBATCH --account=def-someacct # replace this with your own account<br />
#SBATCH --ntasks-per-node=4<br />
#SBATCH --mem-per-cpu=2500M # memory; default unit is megabytes.<br />
#SBATCH --time=0-00:30 # time (DD-HH:MM).<br />
<br />
<!--T:46--><br />
# Load the module:<br />
<br />
<!--T:47--><br />
module load intel/2020.1.217 openmpi/4.0.3 cfour-mpi/2.1<br />
<br />
<!--T:48--><br />
echo "Starting run at: `date`"<br />
<br />
<!--T:49--><br />
CFOUROUTPUT="cfour-output.txt"<br />
export CFOUR_NUM_CORES=${SLURM_NTASKS}<br />
<br />
<!--T:50--><br />
xcfour > ${CFOUROUTPUT} <br />
<br />
<!--T:51--><br />
# Clean the symlink:<br />
if [[ -L "GENBAS" ]]; then unlink GENBAS; fi<br />
<br />
<!--T:52--><br />
echo "Program finished with exit code $? at: `date`"<br />
}}<br />
</tab><br />
</tabs><br />
<br />
<br />
= Related links = <!--T:12--><br />
<br />
<!--T:53--><br />
* [http://slater.chemie.uni-mainz.de/cfour/index.php?n=Main.Manual Manual]<br />
* [http://slater.chemie.uni-mainz.de/cfour/index.php?n=Main.Features Features]<br />
</translate></div>Kerrachehttps://docs.alliancecan.ca/mediawiki/index.php?title=ORCA&diff=130827ORCA2023-03-06T21:57:36Z<p>Kerrache: Updating the version to 5.0.4</p>
<hr />
<div><languages /><br />
[[Category:Software]][[Category:ComputationalChemistry]]<br />
<br />
<translate><br />
<br />
==Introduction== <!--T:1--><br />
ORCA is a flexible, efficient and easy-to-use general-purpose tool for quantum chemistry with specific emphasis on spectroscopic properties of open-shell molecules. It features a wide variety of standard quantum chemical methods ranging from semiempirical methods to DFT to single- and multireference correlated ab initio methods. It can also treat environmental and relativistic effects.<br />
<br />
== Licensing == <!--T:2--><br />
If you wish to use prebuilt ORCA executables:<br />
# You have to register at https://orcaforum.kofo.mpg.de/<br />
# You will receive a first email to verify the email address and activate the account. Follow the instructions in that email.<br />
# Once the registration is complete, you will get a '''second email''' stating that the "''registration for ORCA download and usage has been completed''".<br />
# [[Technical support | Contact us]] requesting access to ORCA with a copy of the '''second email'''.<br />
<br />
== ORCA versions == <!--T:21--><br />
<br />
<!--T:22--><br />
On July 2021, a first version 5 of ORCA was released. This is a major upgrade of ORCA 4.<br />
<br />
=== ORCA 5 === <!--T:23--><br />
<br />
<!--T:24--><br />
The first released versions 5.0 through 5.0.3 have few bugs that were fixed in the following 5.0.4 version. Even if all the versions (5.0.1, 5.0.2, 5.0.3) are installed on our clusters, we recommend that you use the latest version 5.0.4. This version has a fix for D4 dispersion gradients. <br />
<br />
<!--T:25--><br />
To load version 5.0.4, use:<br />
<br />
<!--T:26--><br />
<code><br />
module load StdEnv/2020 gcc/10.3.0 openmpi/4.1.1 orca/5.0.4<br />
</code><br />
<br />
<!--T:27--><br />
'''Note:''' Versions 5.0.1, 5.0.2 and 5.0.3 are in our software stack but could be removed at any time in the future.<br />
<br />
=== ORCA 4 === <!--T:28--><br />
<br />
<!--T:29--><br />
The latest released version of ORCA 4 is 4.2.1. Other versions prior to this one are also available in our software stack.<br />
<br />
<!--T:30--><br />
To load version 4.2.1, use:<br />
<br />
<!--T:31--><br />
<code><br />
module load StdEnv/2020 gcc/9.3.0 openmpi/4.0.3 orca/4.2.1<br />
</code><br />
<br />
<!--T:32--><br />
or<br />
<br />
<!--T:33--><br />
<code><br />
module load nixpkgs/16.09 gcc/7.3.0 openmpi/3.1.4 orca/4.2.1<br />
</code><br />
<br />
== Setting ORCA input files == <!--T:34--><br />
<br />
<!--T:35--><br />
In addition to the different keywords required to run a given simulation, you should make sure to set two additional parameters:<br />
<br />
<!--T:36--><br />
* number of CPUs<br />
<br />
<!--T:37--><br />
* maxcore<br />
<br />
==Using the software== <!--T:3--><br />
To see which versions of ORCA are currently available, type <code>module spider orca</code>. For detailed information about a specific version, including the other modules that must be loaded first, use the module's full name. For example, <code>module spider orca/4.0.1.2</code>.<br />
<br />
<!--T:11--><br />
See [[Using modules]] for general guidance.<br />
<br />
===Job submission=== <!--T:4--><br />
For a general discussion about submitting jobs, see [[Running jobs]].<br />
<br />
<!--T:12--><br />
'''NOTE''': If you run into MPI errors with some of the ORCA executables, you can try to define the following variables:<br />
<br />
<!--T:13--><br />
export OMPI_MCA_mtl='^mxm'<br />
export OMPI_MCA_pml='^yalla'<br />
<br />
<!--T:5--><br />
The following is a job script to run ORCA using [[MPI]]:<br />
<br />
</translate><br />
{{File<br />
|name=run_orca.sh<br />
|lang="bash"<br />
|contents=<br />
#!/bin/bash<br />
#SBATCH --account=def-youPIs<br />
#SBATCH --ntasks=8 # cpus, the nprocs defined in the input file<br />
#SBATCH --mem-per-cpu=3G # memory per cpu<br />
#SBATCH --time=00-03:00 # time (DD-HH:MM)<br />
#SBATCH --output=benzene.log # output .log file<br />
<br />
module load StdEnv/2020 gcc/9.3.0 openmpi/4.0.3<br />
module load orca/4.2.1<br />
$EBROOTORCA/orca benzene.inp<br />
}}<br />
<translate><br />
<!--T:10--><br />
Example of the input file, benzene.inp:<br />
</translate><br />
{{File<br />
|name=benzene.inp<br />
|lang="text"<br />
|contents=<br />
# Benzene RHF Opt Calculation<br />
%pal nprocs 8 end<br />
! RHF TightSCF PModel<br />
! opt<br />
<br />
* xyz 0 1<br />
C 0.000000000000 1.398696930758 0.000000000000<br />
C 0.000000000000 -1.398696930758 0.000000000000<br />
C 1.211265339156 0.699329968382 0.000000000000<br />
C 1.211265339156 -0.699329968382 0.000000000000<br />
C -1.211265339156 0.699329968382 0.000000000000<br />
C -1.211265339156 -0.699329968382 0.000000000000<br />
H 0.000000000000 2.491406946734 0.000000000000<br />
H 0.000000000000 -2.491406946734 0.000000000000<br />
H 2.157597486829 1.245660462400 0.000000000000<br />
H 2.157597486829 -1.245660462400 0.000000000000<br />
H -2.157597486829 1.245660462400 0.000000000000<br />
H -2.157597486829 -1.245660462400 0.000000000000<br />
*<br />
}}<br />
<translate><br />
<br />
===Notes=== <!--T:15--><br />
* To make sure that the program runs efficiently and makes use of all the resources or the cores asked for in your job script, please add this line <code>%pal nprocs <ncores> end</code> to your input file as shown in the above example. Replace <code><ncores></code> by the number of cores you used in your script.<br />
<br />
===(Sep. 6 2019) Temporary fix to OpenMPI version inconsistency issue=== <!--T:16--><br />
For some type of calculations (DLPNO-STEOM-CCSD in particular), you could receive unknown openmpi related fatal errors. This could be due to using an older version of openmpi (''i.e.'' 3.1.2 as suggested by 'module' for both orca/4.1.0 and 4.2.0) than recommended officially (3.1.3 for orca/4.1.0 and 3.1.4 for orca/4.2.0). To temporarily fix this issue, one can build a custom version of openmpi.<br />
<br />
<!--T:17--><br />
The following two commands prepare a custom openmpi/3.1.4 for orca/4.2.0:<br />
module load gcc/7.3.0<br />
eb OpenMPI-3.1.2-GCC-7.3.0.eb --try-software-version=3.1.4<br />
When the building is finished, you can load the custom openmpi using module:<br />
module load openmpi/3.1.4<br />
At this step, one can manually install orca/4.2.0 binaries from the official forum under the home directory after finishing the registration on the official orca forum and being granted access to the orca program on our clusters.<br />
<br />
<br />
<!--T:18--><br />
Additional notes from the contributor:<br />
<br />
<!--T:19--><br />
This is a '''temporary''' fix prior to the official upgrade of openmpi on our clusters. Please remember to delete the manually installed orca binaries once the official openmpi version is up to date.<br />
<br />
<!--T:20--><br />
The compiling command does not seem to apply to openmpi/2.1.x.<br />
<br />
== Related links == <!--T:38--><br />
<br />
<!--T:39--><br />
* [https://orcaforum.kofo.mpg.de/app.php/portal ORCA Forum]<br />
<br />
</translate></div>Kerrachehttps://docs.alliancecan.ca/mediawiki/index.php?title=Symbolic_algebra_software&diff=125363Symbolic algebra software2022-12-13T15:41:11Z<p>Kerrache: Fix a link for Macaulay2</p>
<hr />
<div><languages /><br />
[[Category:Software]]<br />
<br />
<translate><br />
<!--T:1--><br />
Symbolic algebra software is a program, often accessible as an interactive environment, that is able to work directly with symbolic expressions (derivatives, integrals and so forth) and permits exact arithmetic (e.g. <tt>exp(-i*pi/2) = -i</tt>) as well as other formal operations that arise in domains like number theory, group theory, differential geometry, commutative algebra and so forth. Most such programs also permit the use of approximate numerical calculations using floating point numbers for handling problems that are analytically intractable. Some well-known symbolic algebra software packages are the commercial products [http://www.wolfram.com/mathematica/ Mathematica] and [http://www.maplesoft.com/ Maple], neither of which is available on our clusters but which you can install in your home directory if your license for the software allows this. An open source alternative, [https://www.sagemath.org/ SageMath], can however be used by loading the appropriate module:<br />
{{Command|module load sagemath/9.3}}<br />
Afterwards you can then run the software interactively, e.g.<br />
{{Command<br />
|sage<br />
|result=<br />
┌────────────────────────────────────────────────────────────────────┐<br />
│ SageMath version 9.3, Release Date: 2021-05-09 │<br />
│ Using Python 3.8.10. Type "help()" for help. │<br />
└────────────────────────────────────────────────────────────────────┘<br />
sage: <br />
}}<br />
Additional open source software that may be of interest and which is available on the clusters as a [[Utiliser_des_modules/en|module]] includes the [https://www.shoup.net/ntl/ Number Theory Library (NTL)] (<code>ntl</code>), [https://www.singular.uni-kl.de/ SINGULAR] (<code>singular</code>), [https://faculty.math.illinois.edu/Macaulay2/ Macaulay2] (<code>m2</code>) and [http://pari.math.u-bordeaux.fr/ PARI/GP] (<code>pari-gp</code>).<br />
</translate></div>Kerrachehttps://docs.alliancecan.ca/mediawiki/index.php?title=Storage_and_file_management&diff=119351Storage and file management2022-09-12T21:59:47Z<p>Kerrache: Fix links to RAS and RAC</p>
<hr />
<div><languages /><br />
<translate><br />
==Overview== <!--T:1--><br />
<br />
<!--T:2--><br />
Compute Canada provides a wide range of storage options to cover the needs of our very diverse users. These storage solutions range from high-speed temporary local storage to different kinds of long-term storage, so you can choose the storage medium that best corresponds to your needs and usage patterns. In most cases the [https://en.wikipedia.org/wiki/File_system filesystems] on Compute Canada systems are a ''shared'' resource and for this reason should be used responsibly - unwise behaviour can negatively affect dozens or hundreds of other users. These filesystems are also designed to store a limited number of very large files, which are typically binary since very large (hundreds of MB or more) text files lose most of their interest in being human-readable. You should therefore avoid storing tens of thousands of small files, where small means less than a few megabytes, particularly in the same directory. A better approach is to use commands like [[Archiving and compressing files|<tt>tar</tt>]] or <tt>zip</tt> to convert a directory containing many small files into a single very large archive file. <br />
<br />
<!--T:3--><br />
It is also your responsibility to manage the age of your stored data: most of the filesystems are not intended to provide an indefinite archiving service so when a given file or directory is no longer needed, you need to move it to a more appropriate filesystem which may well mean your personal workstation or some other storage system under your control. Moving significant amounts of data between your workstation and a Compute Canada system or between two Compute Canada systems should generally be done using [[Globus]]. <br />
<br />
<!--T:4--><br />
Note that Compute Canada storage systems are not for personal use and should only be used to store research data.<br />
<br />
<!--T:17--><br />
When your account is created on a Compute Canada cluster, your home directory will not be entirely empty. It will contain references to your scratch and [[Project layout|project]] spaces through the mechanism of a [https://en.wikipedia.org/wiki/Symbolic_link symbolic link], a kind of shortcut that allows easy access to these other filesystems from your home directory. Note that these symbolic links may appear up to a few hours after you first connect to the cluster. While your home and scratch spaces are unique to you as an individual user, the project space is a shared by a research group. This group may consist of those individuals with a Compute Canada account sponsored by a particular faculty member or members of a [https://www.computecanada.ca/research-portal/accessing-resources/resource-allocation-competitions/ RAC allocation]. A given individual may thus have access to several different project spaces, associated with one or more faculty members, with symbolic links to these different project spaces in the directory projects of your home. Every account has one or many projects. In the folder <tt>projects</tt> within their home directory, each user has a link to each of the projects they have access to. For users with a single active sponsored role is the default project of your sponsor while users with more than one active sponsored role will have a default project that corresponds to the default project of the faculty member with the most sponsored accounts.<br />
<br />
<!--T:16--><br />
All users can check the available disk space and the current disk utilization for the ''project'', ''home'' and ''scratch'' file systems with the command line utility '''''diskusage_report''''', available on Compute Canada clusters. To use this utility, log into the cluster using SSH, at the command prompt type diskusage_report, and press the Enter key. Following is a typical output of this utility:<br />
<pre><br />
# diskusage_report<br />
Description Space # of files<br />
Home (username) 280 kB/47 GB 25/500k<br />
Scratch (username) 4096 B/18 TB 1/1000k<br />
Project (def-username-ab) 4096 B/9536 GB 2/500k<br />
Project (def-username) 4096 B/9536 GB 2/500k<br />
</pre><br />
<br />
== Storage types == <!--T:5--><br />
Unlike your personal computer, a Compute Canada system will typically have several storage spaces or filesystems and you should ensure that you are using the right space for the right task. In this section we will discuss the principal filesystems available on most Compute Canada systems and the intended use of each one along with some of its characteristics. <br />
* '''HOME:''' While your home directory may seem like the logical place to store all your files and do all your work, in general this isn't the case - your home normally has a relatively small quota and doesn't have especially good performance for the writing and reading of large amounts of data. The most logical use of your home directory is typically source code, small parameter files and job submission scripts. <br />
* '''PROJECT:''' The project space has a significantly larger quota and is well-adapted to [[Sharing data | sharing data]] among members of a research group since it, unlike the home or scratch, is linked to a professor's account rather than an individual user. The data stored in the project space should be fairly static, that is to say the data are not likely to be changed many times in a month. Otherwise, frequently changing data - including just moving and renaming directories - in project can become a heavy burden on the tape-based backup system. <br />
* '''SCRATCH''': For intensive read/write operations on large files (> 100 MB per file), scratch is the best choice. Remember however that important files must be copied off scratch since they are not backed up there, and older files are subject to [[Scratch purging policy|purging]]. The scratch storage should therefore be used for temporary files: checkpoint files, output from jobs and other data that can easily be recreated.<br />
* '''SLURM_TMPDIR''': While a job is running, <code>$SLURM_TMPDIR</code> is a unique path to a temporary folder on a local fast filesystem on each compute node reserved for the job. This is the best location to temporarily store large collections of small files (< 1 MB per file). Note: this space is shared between jobs on each node, and the total available space depends on the node specifications. Finally, when the job ends, this folder is deleted. A more detailed discussion of using <code>$SLURM_TMPDIR</code> is available at [[Using_$SLURM_TMPDIR | this page]].<br />
<br />
==Project space consumption per user== <!--T:23--> <br />
<br />
<!--T:24--><br />
While the command '''diskusage_report''' gives the space and inode usage per user on ''home'' and ''scratch'', it shows the total quota of the group on project. It includes all the files from each member of the group. Since the files that belong to a user could however be anywhere in the project space, it is difficult to obtain correct figures per user and per given project in case a user has access to more than one project. However, users can obtain an estimate of their space and inode use on the entire project space by running the command,<br />
<br />
<!--T:26--><br />
<code>lfs quota -u $USER /project</code><br />
<br />
<!--T:25--><br />
In addition to that, users can obtain an estimate for the number of files in a given directory (and its sub-directories) using the command <code>lfs find</code>, e.g.<br />
<source lang="console"><br />
lfs find <path to the directory> -type f | wc -l<br />
</source><br />
<br />
== Best practices == <!--T:9--><br />
* Regularly clean up your data in the scratch and project spaces, because those filesystems are used for huge data collections.<br />
* Only use text format for files that are smaller than a few megabytes.<br />
* As far as possible, use scratch and local storage for temporary files. For local storage you can use the temporary directory created by the [[Running jobs|job scheduler]] for this, named <code>$SLURM_TMPDIR</code>.<br />
* If your program must search within a file, it is fastest to do it by first reading it completely before searching.<br />
* If you no longer use certain files but they must be retained, [[Archiving and compressing files|archive and compress]] them, and if possible move them to an alternative location like [[Using nearline storage|nearline]].<br />
* For more notes on managing many files, see [[Handling large collections of files]], especially if you are limited by a quota on the number of files. <br />
* Having any sort of parallel write access to a file stored on a shared filesystem like home, scratch and project is likely to create problems unless you are using a specialized tool such as [https://en.wikipedia.org/wiki/Message_Passing_Interface#I/O MPI-IO]. <br />
* If your needs are not well served by the available storage options please contact [[technical support]].<br />
<br />
==Filesystem quotas and policies== <!--T:10--><br />
<br />
<!--T:11--><br />
In order to ensure that there is adequate space for all Compute Canada users, there are a variety of quotas and policy restrictions concerning back-ups and automatic purging of certain filesystems. <br />
By default on our clusters each user has access to the home and scratch spaces, and each group has access to 1 TB of project space. Small increases in project and scratch spaces are available through our Rapid Access Service ([https://alliancecan.ca/en/services/advanced-research-computing/accessing-resources/rapid-access-service RAS]). Larger increases in project spaces are available through the annual Resource Allocation Competitions ([https://alliancecan.ca/en/services/advanced-research-computing/accessing-resources/resource-allocation-competition RAC]). You can see your current quota usage for various filesystems on Cedar and Graham using the command [[Storage and file management#Overview|<tt>diskusage_report</tt>]].<br />
<br />
<!--T:12--><br />
<tabs><br />
<tab name="Cedar"><br />
{| class="wikitable" style="font-size: 95%; text-align: center;"<br />
|+Filesystem Characteristics <br />
! Filesystem<br />
! Default Quota<br />
! Lustre-based?<br />
! Backed up?<br />
! Purged?<br />
! Available by Default?<br />
! Mounted on Compute Nodes?<br />
|-<br />
|Home Space<br />
|50 GB and 500K files per user<ref>This quota is fixed and cannot be changed.</ref><br />
|Yes<br />
|Yes<br />
|No<br />
|Yes<br />
|Yes<br />
|-<br />
|Scratch Space<br />
|20 TB and 1M files per user<br />
|Yes<br />
|No<br />
|Files older than 60 days are purged.<ref>See [[Scratch purging policy]] for more information.</ref><br />
|Yes<br />
|Yes<br />
|-<br />
|Project Space<br />
|1 TB and 500K files per group<ref>Project space can be increased to 10 TB per group by a RAS request. The group's sponsoring PI should write to [[technical support]] to make the request.</ref><br />
|Yes<br />
|Yes<br />
|No<br />
|Yes<br />
|Yes<br />
|-<br />
|Nearline Space<br />
|2 TB and 5000 files per group<br />
|Yes<br />
|Yes<br />
|No<br />
|Yes<br />
|No<br />
|}<br />
<references /><br />
</tab><br />
<tab name="Graham"><br />
{| class="wikitable" style="font-size: 95%; text-align: center;"<br />
|+Filesystem Characteristics <br />
! Filesystem<br />
! Default Quota<br />
! Lustre-based?<br />
! Backed up?<br />
! Purged?<br />
! Available by Default?<br />
! Mounted on Compute Nodes?<br />
|-<br />
|Home Space<br />
|50 GB and 500K files per user<ref>This quota is fixed and cannot be changed.</ref><br />
|No<br />
|Yes<br />
|No<br />
|Yes<br />
|Yes<br />
|-<br />
|Scratch Space<br />
|20 TB and 1M files per user<br />
|Yes<br />
|No<br />
|Files older than 60 days are purged.<ref>See [[Scratch purging policy]] for more information.</ref><br />
|Yes<br />
|Yes<br />
|-<br />
|Project Space<br />
|1 TB and 500K files per group<ref>Project space can be increased to 10 TB per group by a RAS request. The group's sponsoring PI should write to [[technical support]] to make the request.</ref><br />
|Yes<br />
|Yes<br />
|No<br />
|Yes<br />
|Yes<br />
|-<br />
|Nearline Space<br />
|10 TB and 5000 files per group<br />
|Yes<br />
|Yes<br />
|No<br />
|Yes<br />
|No<br />
|}<br />
<references /><br />
</tab><br />
<tab name="Béluga and Narval"><br />
{| class="wikitable" style="font-size: 95%; text-align: center;"<br />
|+Filesystem Characteristics <br />
! Filesystem<br />
! Default Quota<br />
! Lustre-based?<br />
! Backed up?<br />
! Purged?<br />
! Available by Default?<br />
! Mounted on Compute Nodes?<br />
|-<br />
|Home Space<br />
|50 GB and 500K files per user<ref>This quota is fixed and cannot be changed.</ref><br />
|Yes<br />
|Yes<br />
|No<br />
|Yes<br />
|Yes<br />
|-<br />
|Scratch Space<br />
|20 TB and 1M files per user<br />
|Yes<br />
|No<br />
|Files older than 60 days are purged.<ref>See [[Scratch purging policy]] for more information.</ref><br />
|Yes<br />
|Yes<br />
|-<br />
|Project Space<br />
|1 TB and 500K files per group<ref>Project space can be increased to 10 TB per group by a RAS request. The group's sponsoring PI should write to [[technical support]] to make the request.</ref><br />
|Yes<br />
|Yes<br />
|No<br />
|Yes<br />
|Yes<br />
|-<br />
|Nearline Space<br />
|1 TB and 5000 files per group<br />
|Yes<br />
|Yes<br />
|No<br />
|Yes<br />
|No<br />
|}<br />
<references /><br />
</tab><br />
<tab name="Niagara"><br />
{| class="wikitable"<br />
! location<br />
!colspan="2"| quota<br />
!align="right"| block size<br />
! expiration time<br />
! backed up<br />
! on login nodes<br />
! on compute nodes<br />
|-<br />
| $HOME<br />
|colspan="2"| 100 GB per user<br />
|align="right"| 1 MB<br />
| <br />
| yes<br />
| yes<br />
| read-only<br />
|-<br />
|rowspan="6"| $SCRATCH<br />
|colspan="2"| 25 TB per user (dynamic per group)<br />
|align="right" rowspan="6" | 16 MB<br />
|rowspan="6"| 2 months<br />
|rowspan="6"| no<br />
|rowspan="6"| yes<br />
|rowspan="6"| yes<br />
|-<br />
|align="right"|up to 4 users per group<br />
|align="right"|50TB<br />
|-<br />
|align="right"|up to 11 users per group<br />
|align="right"|125TB<br />
|-<br />
|align="right"|up to 28 users per group<br />
|align="right"|250TB<br />
|-<br />
|align="right"|up to 60 users per group<br />
|align="right"|400TB<br />
|-<br />
|align="right"|above 60 users per group<br />
|align="right"|500TB<br />
|-<br />
| $PROJECT<br />
|colspan="2"| by group allocation (RRG or RPP)<br />
|align="right"| 16 MB<br />
| <br />
| yes<br />
| yes<br />
| yes<br />
|-<br />
| $ARCHIVE<br />
|colspan="2"| by group allocation<br />
|align="right"| <br />
|<br />
| dual-copy<br />
| no<br />
| no<br />
|-<br />
| $BBUFFER<br />
|colspan="2"| 10 TB per user<br />
|align="right"| 1 MB<br />
| very short<br />
| no<br />
| yes<br />
| yes<br />
|}<br />
<ul><br />
<li>[https://docs.scinet.utoronto.ca/images/9/9a/Inode_vs._Space_quota_-_v2x.pdf Inode vs. Space quota (PROJECT and SCRATCH)]</li><br />
<li>[https://docs.scinet.utoronto.ca/images/0/0e/Scratch-quota.pdf dynamic quota per group (SCRATCH)]</li><br />
<li>Compute nodes do not have local storage.</li><br />
<li>Archive(a.k.a. nearline) space is on [https://docs.scinet.utoronto.ca/index.php/HPSS HPSS]</li><br />
<li>Backup means a recent snapshot, not an archive of all data that ever was.</li><br />
<li><code>$BBUFFER</code> stands for [https://docs.scinet.utoronto.ca/index.php/Burst_Buffer Burst Buffer], a faster parallel storage tier for temporary data.</li></ul><br />
<br />
<!--T:21--><br />
</tab><br />
</tabs><br />
<br />
<!--T:22--><br />
The backup policy on the home and project space is nightly backups which are retained for 30 days, while deleted files are retained for a further 60 days - note that is entirely distinct from the age limit for purging files from the scratch space. If you wish to recover a previous version of a file or directory, you should contact [[technical support]] with the full path for the file(s) and desired version (by date).<br />
<br />
== See also == <!--T:13--><br />
<br />
<!--T:14--><br />
* [[Diskusage Explorer]]<br />
* [[Project layout]]<br />
* [[Sharing data]]<br />
* [[Transferring data]]<br />
* [[Tuning Lustre]]<br />
* [[Archiving and compressing files]]<br />
* [[Handling large collections of files]]<br />
</translate></div>Kerrachehttps://docs.alliancecan.ca/mediawiki/index.php?title=Gaussian&diff=112959Gaussian2022-03-15T15:09:30Z<p>Kerrache: </p>
<hr />
<div><languages /><br />
[[Category:Software]][[Category:ComputationalChemistry]]<br />
<br />
<translate><br />
<!--T:1--><br />
''See also [[Gaussian error messages]].''<br><br><br />
Gaussian is a computational chemistry application produced by [http://gaussian.com/ Gaussian, Inc.]<br />
<br />
== Limitations == <!--T:46--><br />
<br />
<!--T:3--><br />
Compute Canada currently supports Gaussian only on [[Graham]] and [[Cedar]]. <br />
<br />
<!--T:47--><br />
[https://gaussian.com/running/?tabid=4 Cluster/network parallel execution] of Gaussian, also known as "Linda parallelism", is not supported at any Compute Canada site.<br />
Only [https://gaussian.com/running/?tabid=4 "shared-memory multiprocessor parallel execution"] is supported.<br />
Therefore no Gaussian job can use more than a single compute node.<br />
<br />
== License agreement == <!--T:2--><br />
<br />
<!--T:4--><br />
In order to use Gaussian you must agree to certain conditions. Please [[Technical_support | contact support]] with a copy of the following statement:<br />
# I am not a member of a research group developing software competitive to Gaussian.<br />
# I will not copy the Gaussian software, nor make it available to anyone else.<br />
# I will properly acknowledge Gaussian Inc. and Compute Canada in publications.<br />
# I will notify Compute Canada of any change in the above acknowledgement.<br />
If you are a sponsored user, your sponsor (PI) must also have such a statement on file with us.<br />
<br />
<!--T:5--><br />
We will then grant you access to Gaussian.<br />
<br />
==Running Gaussian on Graham and Cedar== <!--T:6--><br />
The <code>gaussian</code> module is installed on [[Graham]] and [[Cedar]]. To check what versions are available use the <code>module spider</code> command as follows:<br />
<br />
<!--T:36--><br />
[name@server $] module spider gaussian<br />
<br />
<!--T:37--><br />
For module commands, please see [[Utiliser des modules/en|Using modules]].<br />
<br />
</translate> <br />
<br />
<translate><br />
===Job submission=== <!--T:7--><br />
The national clusters use the Slurm scheduler; for details about submitting jobs, see [[Running jobs]].<br />
<br />
Since only "shared-memory multiprocessor parallel version of Gaussian is supported on cedar and graham, your jobs could use only one node and up to a maximum cores per node: 48 on cedar and 32 on graham. If your jobs require more memory, you may use large memory nodes. Please refer to the site pages [[Cedar/en#Node_characteristics|cedar]] and [[Graham/en#Node_characteristics|graham]] for more information about the hardware. <br />
<br />
<!--T:8--><br />
Besides your input file (in our example name.com), you have to prepare a job script to define the compute resources for the job; both input file and job script must be in the same directory.<br />
<br />
<!--T:9--><br />
There are two options to run your Gaussian job on Graham and Cedar, based on the location of the default runtime files and the job size.<br />
<br />
====G16 (G09, G03)==== <!--T:10--><br />
<br />
<!--T:11--><br />
This option will save the default runtime files (unnamed .rwf, .inp, .d2e, .int, .skr files) to /scratch/username/jobid/. Those files will stay there when the job is unfinished or failed for whatever reason, you could locate the .rwf file for restart purpose later.<br />
<br />
<!--T:12--><br />
The following example is a G16 job script:<br />
<br />
<!--T:31--><br />
Note that for coherence, we use the same name for each files, changing only the extension (name.sh, name.com, name.log).<br />
</translate><br />
{{File<br />
|name=mysub.sh<br />
|lang="bash"<br />
|contents=<br />
#!/bin/bash<br />
#SBATCH --mem=16G # <translate><!--T:13--><br />
memory, roughly 2 times %mem defined in the input name.com file</translate><br />
#SBATCH --time=02-00:00 # <translate><!--T:14--><br />
expect run time (DD-HH:MM)</translate><br />
#SBATCH --cpus-per-task=16 # <translate><!--T:15--><br />
No. of cpus for the job as defined by %nprocs in the name.com file</translate><br />
module load gaussian/g16.c01<br />
G16 name.com # <translate><!--T:16--><br />
G16 command, input: name.com, output: name.log</translate><br />
}}<br />
<translate><br />
<!--T:17--><br />
To use Gaussian 09 or Gaussian 03, simply modify the module load gaussian/g16.b01 to gaussian/g09.e01 or gaussian/g03.d01, and change G16 to G09 or G03. You can modify the --mem, --time, --cpus-per-task to match your job's requirements for compute resources.<br />
<br />
====g16 (g09, g03)==== <!--T:18--><br />
<br />
<!--T:19--><br />
This option will save the default runtime files (unnamed .rwf, .inp, .d2e, .int, .skr files) temporarily in $SLURM_TMPDIR (/localscratch/username.jobid.0/) on the compute node where the job was scheduled to. The files will be removed by the scheduler when a job is done (successful or not). If you do not expect to use the .rwf file to restart in a later time, you can use this option.<br />
<br />
<!--T:20--><br />
/localscratch is ~800G shared by all jobs running on the same node. If your job files would be bigger than or close to that size range, you would instead use the G16 (G09, G03) option.<br />
<br />
<!--T:21--><br />
The following example is a g16 job script:<br />
</translate><br />
{{File<br />
|name=mysub.sh<br />
|lang="bash"<br />
|contents=<br />
#!/bin/bash<br />
#SBATCH --mem=16G # <translate><!--T:22--><br />
memory, roughly 2 times %mem defined in the input name.com file</translate><br />
#SBATCH --time=02-00:00 # <translate><!--T:23--><br />
expect run time (DD-HH:MM)</translate><br />
#SBATCH --cpus-per-task=16 # <translate><!--T:24--><br />
No. of cpus for the job as defined by %nprocs in the name.com file</translate><br />
module load gaussian/g16.c01<br />
g16 < name.com # <translate><!--T:25--><br />
g16 command, input: name.com, output: slurm-<jobid>.out by default</translate><br />
}}<br />
<translate><br />
<br />
====Submit the job==== <!--T:33--><br />
sbatch mysub.sh<br />
<br />
=== Interactive jobs === <!--T:26--><br />
You can run interactive Gaussian job for testing purpose on Graham and Cedar. It's not a good practice to run interactive Gaussian jobs on a login node. You can start an interactive session on a compute node with salloc, the example for an hour, 8 cpus and 10G memory Gaussian job is like<br />
Goto the input file directory first, then use salloc command:<br />
</translate><br />
{{Command|salloc --time{{=}}1:0:0 --cpus-per-task{{=}}8 --mem{{=}}10g}}<br />
<br />
<translate><br />
<!--T:27--><br />
Then use either<br />
</translate><br />
{{Commands<br />
|module load gaussian/g16.c01<br />
|G16 g16_test2.com # <translate><!--T:28--><br />
G16 saves runtime file (.rwf etc.) to /scratch/yourid/93288/</translate><br />
}}<br />
<br />
<translate><!--T:29--><br />
or </translate><br />
{{Commands<br />
|module load gaussian/g16.c01<br />
|g16 < g16_test2.com >& g16_test2.log & # <translate><!--T:30--><br />
g16 saves runtime file to /localscratch/yourid/</translate><br />
}}<br />
<translate><br />
=== Restart jobs === <!--T:38--><br />
Gaussian jobs can always be restarted from the previous <tt>rwf</tt> file.<br />
<br />
<!--T:39--><br />
Geometry optimization can be restarted from the <tt>chk</tt> file as usual.<br />
One-step computation, such as Analytic frequency calculations, including properties like ROA and VCD with ONIOM; CCSD and EOM-CCSD calculations; NMR; Polar=OptRot; CID, CISD, CCD, QCISD and BD energies, can be restarted from the <tt>rwf</tt> file.<br />
<br />
<!--T:40--><br />
To restart a job from previous <tt>rwf</tt> file, you need to know the location of this <tt>rwf</tt> file from your previous run. <br />
<br />
<!--T:41--><br />
The restart input is simple: first you need to specify %rwf path to the previous <tt>rwf</tt> file, secondly change the keywords line to be #p restart, then leave a blank line at the end.<br />
<br />
<!--T:42--><br />
A sample restart input is like:<br />
{{File<br />
|name=restart.com<br />
|lang="bash"<br />
|contents=<br />
%rwf=/scratch/yourid/jobid/name.rwf<br />
%NoSave<br />
%chk=name.chk<br />
%mem=5000mb<br />
%nprocs=16<br />
#p restart<br />
(one blank line)<br />
<br />
<!--T:43--><br />
}}<br />
<br />
===Examples=== <!--T:34--><br />
An example input file and the run scripts <tt>*.sh</tt> can be found in<br />
<tt>/opt/software/gaussian/version/examples/</tt><br />
where version is either g03.d10, g09.e01, or g16.b01<br />
<br />
== Notes == <!--T:35--><br />
# NBO7 is included in g16.c01 version only, both nbo6 and nbo7 keywords will run NBO7 in g16.c01<br />
# NBO6 is available in g09.e01 and g16.b01 versions.<br />
# You can watch a recorded webinar/tutorial: [https://www.youtube.com/watch?v=xpBhPnRbeQo Gaussian16 and NBO7 on Graham and Cedar]<br />
<br />
== Errors == <!--T:44--><br />
Some of the error messages produced by Gaussian have been collected, with suggestions for their resolution. See [[Gaussian error messages]].<br />
</translate></div>Kerrachehttps://docs.alliancecan.ca/mediawiki/index.php?title=Gaussian&diff=112958Gaussian2022-03-15T15:08:14Z<p>Kerrache: </p>
<hr />
<div><languages /><br />
[[Category:Software]][[Category:ComputationalChemistry]]<br />
<br />
<translate><br />
<!--T:1--><br />
''See also [[Gaussian error messages]].''<br><br><br />
Gaussian is a computational chemistry application produced by [http://gaussian.com/ Gaussian, Inc.]<br />
<br />
== Limitations == <!--T:46--><br />
<br />
<!--T:3--><br />
Compute Canada currently supports Gaussian only on [[Graham]] and [[Cedar]]. <br />
<br />
<!--T:47--><br />
[https://gaussian.com/running/?tabid=4 Cluster/network parallel execution] of Gaussian, also known as "Linda parallelism", is not supported at any Compute Canada site.<br />
Only [https://gaussian.com/running/?tabid=4 "shared-memory multiprocessor parallel execution"] is supported.<br />
Therefore no Gaussian job can use more than a single compute node.<br />
<br />
== License agreement == <!--T:2--><br />
<br />
<!--T:4--><br />
In order to use Gaussian you must agree to certain conditions. Please [[Technical_support | contact support]] with a copy of the following statement:<br />
# I am not a member of a research group developing software competitive to Gaussian.<br />
# I will not copy the Gaussian software, nor make it available to anyone else.<br />
# I will properly acknowledge Gaussian Inc. and Compute Canada in publications.<br />
# I will notify Compute Canada of any change in the above acknowledgement.<br />
If you are a sponsored user, your sponsor (PI) must also have such a statement on file with us.<br />
<br />
<!--T:5--><br />
We will then grant you access to Gaussian.<br />
<br />
==Running Gaussian on Graham and Cedar== <!--T:6--><br />
The <code>gaussian</code> module is installed on [[Graham]] and [[Cedar]]. To check what versions are available use the <code>module spider</code> command as follows:<br />
<br />
<!--T:36--><br />
[name@server $] module spider gaussian<br />
<br />
<!--T:37--><br />
For module commands, please see [[Utiliser des modules/en|Using modules]].<br />
<br />
</translate> <br />
<br />
<translate><br />
===Job submission=== <!--T:7--><br />
The national clusters use the Slurm scheduler; for details about submitting jobs, see [[Running jobs]].<br />
<br />
Since only "shared-memory multiprocessor parallel version of Gaussian is supported on cedar and graham, your jobs could use only one node and up to a maximum cores per node: 48 on cedar and 32 on graham. If your jobs require more memory, you may use large memory nodes. Please refer to the sites pages [[Cedar/en#Node_characteristics|cedar]] and [[Graham/en#Node_characteristics|graham]] for more information about the hardware. <br />
<br />
<!--T:8--><br />
Besides your input file (in our example name.com), you have to prepare a job script to define the compute resources for the job; both input file and job script must be in the same directory.<br />
<br />
<!--T:9--><br />
There are two options to run your Gaussian job on Graham and Cedar, based on the location of the default runtime files and the job size.<br />
<br />
====G16 (G09, G03)==== <!--T:10--><br />
<br />
<!--T:11--><br />
This option will save the default runtime files (unnamed .rwf, .inp, .d2e, .int, .skr files) to /scratch/username/jobid/. Those files will stay there when the job is unfinished or failed for whatever reason, you could locate the .rwf file for restart purpose later.<br />
<br />
<!--T:12--><br />
The following example is a G16 job script:<br />
<br />
<!--T:31--><br />
Note that for coherence, we use the same name for each files, changing only the extension (name.sh, name.com, name.log).<br />
</translate><br />
{{File<br />
|name=mysub.sh<br />
|lang="bash"<br />
|contents=<br />
#!/bin/bash<br />
#SBATCH --mem=16G # <translate><!--T:13--><br />
memory, roughly 2 times %mem defined in the input name.com file</translate><br />
#SBATCH --time=02-00:00 # <translate><!--T:14--><br />
expect run time (DD-HH:MM)</translate><br />
#SBATCH --cpus-per-task=16 # <translate><!--T:15--><br />
No. of cpus for the job as defined by %nprocs in the name.com file</translate><br />
module load gaussian/g16.c01<br />
G16 name.com # <translate><!--T:16--><br />
G16 command, input: name.com, output: name.log</translate><br />
}}<br />
<translate><br />
<!--T:17--><br />
To use Gaussian 09 or Gaussian 03, simply modify the module load gaussian/g16.b01 to gaussian/g09.e01 or gaussian/g03.d01, and change G16 to G09 or G03. You can modify the --mem, --time, --cpus-per-task to match your job's requirements for compute resources.<br />
<br />
====g16 (g09, g03)==== <!--T:18--><br />
<br />
<!--T:19--><br />
This option will save the default runtime files (unnamed .rwf, .inp, .d2e, .int, .skr files) temporarily in $SLURM_TMPDIR (/localscratch/username.jobid.0/) on the compute node where the job was scheduled to. The files will be removed by the scheduler when a job is done (successful or not). If you do not expect to use the .rwf file to restart in a later time, you can use this option.<br />
<br />
<!--T:20--><br />
/localscratch is ~800G shared by all jobs running on the same node. If your job files would be bigger than or close to that size range, you would instead use the G16 (G09, G03) option.<br />
<br />
<!--T:21--><br />
The following example is a g16 job script:<br />
</translate><br />
{{File<br />
|name=mysub.sh<br />
|lang="bash"<br />
|contents=<br />
#!/bin/bash<br />
#SBATCH --mem=16G # <translate><!--T:22--><br />
memory, roughly 2 times %mem defined in the input name.com file</translate><br />
#SBATCH --time=02-00:00 # <translate><!--T:23--><br />
expect run time (DD-HH:MM)</translate><br />
#SBATCH --cpus-per-task=16 # <translate><!--T:24--><br />
No. of cpus for the job as defined by %nprocs in the name.com file</translate><br />
module load gaussian/g16.c01<br />
g16 < name.com # <translate><!--T:25--><br />
g16 command, input: name.com, output: slurm-<jobid>.out by default</translate><br />
}}<br />
<translate><br />
<br />
====Submit the job==== <!--T:33--><br />
sbatch mysub.sh<br />
<br />
=== Interactive jobs === <!--T:26--><br />
You can run interactive Gaussian job for testing purpose on Graham and Cedar. It's not a good practice to run interactive Gaussian jobs on a login node. You can start an interactive session on a compute node with salloc, the example for an hour, 8 cpus and 10G memory Gaussian job is like<br />
Goto the input file directory first, then use salloc command:<br />
</translate><br />
{{Command|salloc --time{{=}}1:0:0 --cpus-per-task{{=}}8 --mem{{=}}10g}}<br />
<br />
<translate><br />
<!--T:27--><br />
Then use either<br />
</translate><br />
{{Commands<br />
|module load gaussian/g16.c01<br />
|G16 g16_test2.com # <translate><!--T:28--><br />
G16 saves runtime file (.rwf etc.) to /scratch/yourid/93288/</translate><br />
}}<br />
<br />
<translate><!--T:29--><br />
or </translate><br />
{{Commands<br />
|module load gaussian/g16.c01<br />
|g16 < g16_test2.com >& g16_test2.log & # <translate><!--T:30--><br />
g16 saves runtime file to /localscratch/yourid/</translate><br />
}}<br />
<translate><br />
=== Restart jobs === <!--T:38--><br />
Gaussian jobs can always be restarted from the previous <tt>rwf</tt> file.<br />
<br />
<!--T:39--><br />
Geometry optimization can be restarted from the <tt>chk</tt> file as usual.<br />
One-step computation, such as Analytic frequency calculations, including properties like ROA and VCD with ONIOM; CCSD and EOM-CCSD calculations; NMR; Polar=OptRot; CID, CISD, CCD, QCISD and BD energies, can be restarted from the <tt>rwf</tt> file.<br />
<br />
<!--T:40--><br />
To restart a job from previous <tt>rwf</tt> file, you need to know the location of this <tt>rwf</tt> file from your previous run. <br />
<br />
<!--T:41--><br />
The restart input is simple: first you need to specify %rwf path to the previous <tt>rwf</tt> file, secondly change the keywords line to be #p restart, then leave a blank line at the end.<br />
<br />
<!--T:42--><br />
A sample restart input is like:<br />
{{File<br />
|name=restart.com<br />
|lang="bash"<br />
|contents=<br />
%rwf=/scratch/yourid/jobid/name.rwf<br />
%NoSave<br />
%chk=name.chk<br />
%mem=5000mb<br />
%nprocs=16<br />
#p restart<br />
(one blank line)<br />
<br />
<!--T:43--><br />
}}<br />
<br />
===Examples=== <!--T:34--><br />
An example input file and the run scripts <tt>*.sh</tt> can be found in<br />
<tt>/opt/software/gaussian/version/examples/</tt><br />
where version is either g03.d10, g09.e01, or g16.b01<br />
<br />
== Notes == <!--T:35--><br />
# NBO7 is included in g16.c01 version only, both nbo6 and nbo7 keywords will run NBO7 in g16.c01<br />
# NBO6 is available in g09.e01 and g16.b01 versions.<br />
# You can watch a recorded webinar/tutorial: [https://www.youtube.com/watch?v=xpBhPnRbeQo Gaussian16 and NBO7 on Graham and Cedar]<br />
<br />
== Errors == <!--T:44--><br />
Some of the error messages produced by Gaussian have been collected, with suggestions for their resolution. See [[Gaussian error messages]].<br />
</translate></div>Kerrachehttps://docs.alliancecan.ca/mediawiki/index.php?title=Gaussian&diff=112957Gaussian2022-03-15T15:06:27Z<p>Kerrache: </p>
<hr />
<div><languages /><br />
[[Category:Software]][[Category:ComputationalChemistry]]<br />
<br />
<translate><br />
<!--T:1--><br />
''See also [[Gaussian error messages]].''<br><br><br />
Gaussian is a computational chemistry application produced by [http://gaussian.com/ Gaussian, Inc.]<br />
<br />
== Limitations == <!--T:46--><br />
<br />
<!--T:3--><br />
Compute Canada currently supports Gaussian only on [[Graham]] and [[Cedar]]. <br />
<br />
<!--T:47--><br />
[https://gaussian.com/running/?tabid=4 Cluster/network parallel execution] of Gaussian, also known as "Linda parallelism", is not supported at any Compute Canada site.<br />
Only [https://gaussian.com/running/?tabid=4 "shared-memory multiprocessor parallel execution"] is supported.<br />
Therefore no Gaussian job can use more than a single compute node.<br />
<br />
== License agreement == <!--T:2--><br />
<br />
<!--T:4--><br />
In order to use Gaussian you must agree to certain conditions. Please [[Technical_support | contact support]] with a copy of the following statement:<br />
# I am not a member of a research group developing software competitive to Gaussian.<br />
# I will not copy the Gaussian software, nor make it available to anyone else.<br />
# I will properly acknowledge Gaussian Inc. and Compute Canada in publications.<br />
# I will notify Compute Canada of any change in the above acknowledgement.<br />
If you are a sponsored user, your sponsor (PI) must also have such a statement on file with us.<br />
<br />
<!--T:5--><br />
We will then grant you access to Gaussian.<br />
<br />
==Running Gaussian on Graham and Cedar== <!--T:6--><br />
The <code>gaussian</code> module is installed on [[Graham]] and [[Cedar]]. To check what versions are available use the <code>module spider</code> command as follows:<br />
<br />
<!--T:36--><br />
[name@server $] module spider gaussian<br />
<br />
<!--T:37--><br />
For module commands, please see [[Utiliser des modules/en|Using modules]].<br />
<br />
</translate> <br />
<br />
<translate><br />
===Job submission=== <!--T:7--><br />
The national clusters use the Slurm scheduler; for details about submitting jobs, see [[Running jobs]].<br />
<br />
Since only "shared-memory multiprocessor parallel version of Gaussian is supported on cedar and graham, your jobs could use only one node and up to a maximum cores per node: 48 on cedar and 32 on graham. If your jobs require more memory, you may use large memory nodes. Please refer to the sites pages [[Cedar/en|cedar]] and [[Graham/en#Node_characteristics|graham]] for more information about the hardware. <br />
<br />
<!--T:8--><br />
Besides your input file (in our example name.com), you have to prepare a job script to define the compute resources for the job; both input file and job script must be in the same directory.<br />
<br />
<!--T:9--><br />
There are two options to run your Gaussian job on Graham and Cedar, based on the location of the default runtime files and the job size.<br />
<br />
====G16 (G09, G03)==== <!--T:10--><br />
<br />
<!--T:11--><br />
This option will save the default runtime files (unnamed .rwf, .inp, .d2e, .int, .skr files) to /scratch/username/jobid/. Those files will stay there when the job is unfinished or failed for whatever reason, you could locate the .rwf file for restart purpose later.<br />
<br />
<!--T:12--><br />
The following example is a G16 job script:<br />
<br />
<!--T:31--><br />
Note that for coherence, we use the same name for each files, changing only the extension (name.sh, name.com, name.log).<br />
</translate><br />
{{File<br />
|name=mysub.sh<br />
|lang="bash"<br />
|contents=<br />
#!/bin/bash<br />
#SBATCH --mem=16G # <translate><!--T:13--><br />
memory, roughly 2 times %mem defined in the input name.com file</translate><br />
#SBATCH --time=02-00:00 # <translate><!--T:14--><br />
expect run time (DD-HH:MM)</translate><br />
#SBATCH --cpus-per-task=16 # <translate><!--T:15--><br />
No. of cpus for the job as defined by %nprocs in the name.com file</translate><br />
module load gaussian/g16.c01<br />
G16 name.com # <translate><!--T:16--><br />
G16 command, input: name.com, output: name.log</translate><br />
}}<br />
<translate><br />
<!--T:17--><br />
To use Gaussian 09 or Gaussian 03, simply modify the module load gaussian/g16.b01 to gaussian/g09.e01 or gaussian/g03.d01, and change G16 to G09 or G03. You can modify the --mem, --time, --cpus-per-task to match your job's requirements for compute resources.<br />
<br />
====g16 (g09, g03)==== <!--T:18--><br />
<br />
<!--T:19--><br />
This option will save the default runtime files (unnamed .rwf, .inp, .d2e, .int, .skr files) temporarily in $SLURM_TMPDIR (/localscratch/username.jobid.0/) on the compute node where the job was scheduled to. The files will be removed by the scheduler when a job is done (successful or not). If you do not expect to use the .rwf file to restart in a later time, you can use this option.<br />
<br />
<!--T:20--><br />
/localscratch is ~800G shared by all jobs running on the same node. If your job files would be bigger than or close to that size range, you would instead use the G16 (G09, G03) option.<br />
<br />
<!--T:21--><br />
The following example is a g16 job script:<br />
</translate><br />
{{File<br />
|name=mysub.sh<br />
|lang="bash"<br />
|contents=<br />
#!/bin/bash<br />
#SBATCH --mem=16G # <translate><!--T:22--><br />
memory, roughly 2 times %mem defined in the input name.com file</translate><br />
#SBATCH --time=02-00:00 # <translate><!--T:23--><br />
expect run time (DD-HH:MM)</translate><br />
#SBATCH --cpus-per-task=16 # <translate><!--T:24--><br />
No. of cpus for the job as defined by %nprocs in the name.com file</translate><br />
module load gaussian/g16.c01<br />
g16 < name.com # <translate><!--T:25--><br />
g16 command, input: name.com, output: slurm-<jobid>.out by default</translate><br />
}}<br />
<translate><br />
<br />
====Submit the job==== <!--T:33--><br />
sbatch mysub.sh<br />
<br />
=== Interactive jobs === <!--T:26--><br />
You can run interactive Gaussian job for testing purpose on Graham and Cedar. It's not a good practice to run interactive Gaussian jobs on a login node. You can start an interactive session on a compute node with salloc, the example for an hour, 8 cpus and 10G memory Gaussian job is like<br />
Goto the input file directory first, then use salloc command:<br />
</translate><br />
{{Command|salloc --time{{=}}1:0:0 --cpus-per-task{{=}}8 --mem{{=}}10g}}<br />
<br />
<translate><br />
<!--T:27--><br />
Then use either<br />
</translate><br />
{{Commands<br />
|module load gaussian/g16.c01<br />
|G16 g16_test2.com # <translate><!--T:28--><br />
G16 saves runtime file (.rwf etc.) to /scratch/yourid/93288/</translate><br />
}}<br />
<br />
<translate><!--T:29--><br />
or </translate><br />
{{Commands<br />
|module load gaussian/g16.c01<br />
|g16 < g16_test2.com >& g16_test2.log & # <translate><!--T:30--><br />
g16 saves runtime file to /localscratch/yourid/</translate><br />
}}<br />
<translate><br />
=== Restart jobs === <!--T:38--><br />
Gaussian jobs can always be restarted from the previous <tt>rwf</tt> file.<br />
<br />
<!--T:39--><br />
Geometry optimization can be restarted from the <tt>chk</tt> file as usual.<br />
One-step computation, such as Analytic frequency calculations, including properties like ROA and VCD with ONIOM; CCSD and EOM-CCSD calculations; NMR; Polar=OptRot; CID, CISD, CCD, QCISD and BD energies, can be restarted from the <tt>rwf</tt> file.<br />
<br />
<!--T:40--><br />
To restart a job from previous <tt>rwf</tt> file, you need to know the location of this <tt>rwf</tt> file from your previous run. <br />
<br />
<!--T:41--><br />
The restart input is simple: first you need to specify %rwf path to the previous <tt>rwf</tt> file, secondly change the keywords line to be #p restart, then leave a blank line at the end.<br />
<br />
<!--T:42--><br />
A sample restart input is like:<br />
{{File<br />
|name=restart.com<br />
|lang="bash"<br />
|contents=<br />
%rwf=/scratch/yourid/jobid/name.rwf<br />
%NoSave<br />
%chk=name.chk<br />
%mem=5000mb<br />
%nprocs=16<br />
#p restart<br />
(one blank line)<br />
<br />
<!--T:43--><br />
}}<br />
<br />
===Examples=== <!--T:34--><br />
An example input file and the run scripts <tt>*.sh</tt> can be found in<br />
<tt>/opt/software/gaussian/version/examples/</tt><br />
where version is either g03.d10, g09.e01, or g16.b01<br />
<br />
== Notes == <!--T:35--><br />
# NBO7 is included in g16.c01 version only, both nbo6 and nbo7 keywords will run NBO7 in g16.c01<br />
# NBO6 is available in g09.e01 and g16.b01 versions.<br />
# You can watch a recorded webinar/tutorial: [https://www.youtube.com/watch?v=xpBhPnRbeQo Gaussian16 and NBO7 on Graham and Cedar]<br />
<br />
== Errors == <!--T:44--><br />
Some of the error messages produced by Gaussian have been collected, with suggestions for their resolution. See [[Gaussian error messages]].<br />
</translate></div>Kerrachehttps://docs.alliancecan.ca/mediawiki/index.php?title=ORCA&diff=110929ORCA2022-02-01T19:28:15Z<p>Kerrache: /* Related links */</p>
<hr />
<div><languages /><br />
[[Category:Software]][[Category:ComputationalChemistry]]<br />
<br />
<translate><br />
<br />
==Introduction== <!--T:1--><br />
ORCA is a flexible, efficient and easy-to-use general purpose tool for quantum chemistry with specific emphasis on spectroscopic properties of open-shell molecules. It features a wide variety of standard quantum chemical methods ranging from semiempirical methods to DFT to single- and multireference correlated ab initio methods. It can also treat environmental and relativistic effects.<br />
<br />
== Licensing == <!--T:2--><br />
If you wish to use pre-built ORCA executables:<br />
# You have to register at https://orcaforum.kofo.mpg.de/<br />
# You will receive a first email to verify the email address and activate the account. Follow the instructions in that email.<br />
# Once the registration is complete you will get a '''second email''' stating that the "''registration for ORCA download and usage has been completed''".<br />
# [[Technical support | Contact us]] requesting access to ORCA with a copy of the '''second email''' mentioned above.<br />
<br />
== ORCA versions ==<br />
<br />
On July 2021, a version 5 of ORCA was released. This is a major upgrade of ORCA 4.<br />
<br />
=== ORCA 5 ===<br />
<br />
The first released versions 5.0 and 5.0.1 have few bugs that were fixed in the later version 5.0.2. Even if the version 5.0.1 is installed on Compute Canada clusters, users should use the latest one 5.0.2.<br />
<br />
To load the version 5.0.2, use:<br />
<br />
<code><br />
module load StdEnv/2020 gcc/10.3.0 openmpi/4.1.1 orca/5.0.2<br />
</code><br />
<br />
'''Note:''' The version 5.0.1 is kept on Compute Canada software stack but it may be removed any time.<br />
<br />
=== ORCA 4 ===<br />
<br />
The latest released version of ORCA 4 is "4.2.1". Other versions prior to this one are also available on Compute Canada software stack.<br />
<br />
To load the version 4.2.1, use:<br />
<br />
<code><br />
module load StdEnv/2020 gcc/9.3.0 openmpi/4.0.3 orca/4.2.1<br />
</code><br />
<br />
or<br />
<br />
<code><br />
module load nixpkgs/16.09 gcc/7.3.0 openmpi/3.1.4 orca/4.2.1<br />
</code><br />
<br />
== Setting ORCA input fies ==<br />
<br />
In addition to the different keywords required to run a given simulation, users should make sure to set two additional parameters:<br />
<br />
* Number of CPUs:<br />
<br />
* maxcore:<br />
<br />
==Using the software== <!--T:3--><br />
To see what versions of ORCA are currently available, type <code>module spider orca</code>. For detailed information about a specific version, including what other modules must be loaded first, use the module's full name. For example, <code>module spider orca/4.0.1.2</code>.<br />
<br />
<!--T:11--><br />
See [[Using modules]] for general guidance.<br />
<br />
===Job submission=== <!--T:4--><br />
For a general discussion about submitting jobs, see [[Running jobs]].<br />
<br />
<!--T:12--><br />
'''NOTE''': If you run into MPI errors with some of the ORCA executables, you can try to define the following variables:<br />
<br />
<!--T:13--><br />
export OMPI_MCA_mtl='^mxm'<br />
export OMPI_MCA_pml='^yalla'<br />
<br />
<!--T:5--><br />
The following is a job script to run ORCA using [[MPI]]:<br />
<br />
</translate><br />
{{File<br />
|name=run_orca.sh<br />
|lang="bash"<br />
|contents=<br />
#!/bin/bash<br />
#SBATCH --account=def-youPIs<br />
#SBATCH --ntasks=8 # cpus, the nprocs defined in the input file<br />
#SBATCH --mem-per-cpu=3G # memory per cpu<br />
#SBATCH --time=00-03:00 # time (DD-HH:MM)<br />
#SBATCH --output=benzene.log # output .log file<br />
<br />
module load StdEnv/2020 gcc/9.3.0 openmpi/4.0.3<br />
module load orca/4.2.1<br />
$EBROOTORCA/orca benzene.inp<br />
}}<br />
<translate><br />
<!--T:10--><br />
Example of the input file, benzene.inp:<br />
</translate><br />
{{File<br />
|name=benzene.inp<br />
|lang="text"<br />
|contents=<br />
# Benzene RHF Opt Calculation<br />
%pal nprocs 8 end<br />
! RHF TightSCF PModel<br />
! opt<br />
<br />
* xyz 0 1<br />
C 0.000000000000 1.398696930758 0.000000000000<br />
C 0.000000000000 -1.398696930758 0.000000000000<br />
C 1.211265339156 0.699329968382 0.000000000000<br />
C 1.211265339156 -0.699329968382 0.000000000000<br />
C -1.211265339156 0.699329968382 0.000000000000<br />
C -1.211265339156 -0.699329968382 0.000000000000<br />
H 0.000000000000 2.491406946734 0.000000000000<br />
H 0.000000000000 -2.491406946734 0.000000000000<br />
H 2.157597486829 1.245660462400 0.000000000000<br />
H 2.157597486829 -1.245660462400 0.000000000000<br />
H -2.157597486829 1.245660462400 0.000000000000<br />
H -2.157597486829 -1.245660462400 0.000000000000<br />
*<br />
}}<br />
<translate><br />
<br />
===Notes=== <!--T:15--><br />
* To make sure that the program runs efficiently and makes use of all the resources or the cores asked for in your job script, please add this line <code>%pal nprocs <ncores> end</code> to your input file as shown in the above example. Replace <code><ncores></code> by the number of cores you used in your script.<br />
<br />
===(Sep. 6 2019) Temporary fix to OpenMPI version inconsistency issue=== <!--T:16--><br />
For some type of calculations (DLPNO-STEOM-CCSD in particular), one could receive unknown openmpi related fatal errors. This could be due to using an older version of openmpi (''i.e.'' 3.1.2 as suggested by 'module' for both orca/4.1.0 and 4.2.0) than recommended officially (3.1.3 for orca/4.1.0 and 3.1.4 for orca/4.2.0). To temporarily fix this issue, one can build a custom version of openmpi.<br />
<br />
<!--T:17--><br />
The following two commands prepares a custom openmpi/3.1.4 for orca/4.2.0:<br />
module load gcc/7.3.0<br />
eb OpenMPI-3.1.2-GCC-7.3.0.eb --try-software-version=3.1.4<br />
When the building is finished, one can load the custom openmpi using module:<br />
module load openmpi/3.1.4<br />
At this step, one can manually install orca/4.2.0 binaries from the official forum under the home directory after finishing the registration on the official orca forum and being granted access to the orca program on Compute Canada clusters.<br />
<br />
<br />
<!--T:18--><br />
Additional notes from the contributor:<br />
<br />
<!--T:19--><br />
This is a '''temporary''' fix prior to the official upgrade of openmpi on Compute Canada clusters. Please remember to delete the manually installed orca binaries once the official openmpi version is up to date.<br />
<br />
<!--T:20--><br />
The compiling command does not seem to apply to openmpi/2.1.x.<br />
<br />
== Related links ==<br />
<br />
* ORCA Forum<br />
<br />
</translate></div>Kerrachehttps://docs.alliancecan.ca/mediawiki/index.php?title=ORCA&diff=110925ORCA2022-02-01T19:26:00Z<p>Kerrache: /* ORCA performance */</p>
<hr />
<div><languages /><br />
[[Category:Software]][[Category:ComputationalChemistry]]<br />
<br />
<translate><br />
<br />
==Introduction== <!--T:1--><br />
ORCA is a flexible, efficient and easy-to-use general purpose tool for quantum chemistry with specific emphasis on spectroscopic properties of open-shell molecules. It features a wide variety of standard quantum chemical methods ranging from semiempirical methods to DFT to single- and multireference correlated ab initio methods. It can also treat environmental and relativistic effects.<br />
<br />
== Licensing == <!--T:2--><br />
If you wish to use pre-built ORCA executables:<br />
# You have to register at https://orcaforum.kofo.mpg.de/<br />
# You will receive a first email to verify the email address and activate the account. Follow the instructions in that email.<br />
# Once the registration is complete you will get a '''second email''' stating that the "''registration for ORCA download and usage has been completed''".<br />
# [[Technical support | Contact us]] requesting access to ORCA with a copy of the '''second email''' mentioned above.<br />
<br />
== ORCA versions ==<br />
<br />
On July 2021, a version 5 of ORCA was released. This is a major upgrade of ORCA 4.<br />
<br />
=== ORCA 5 ===<br />
<br />
The first released versions 5.0 and 5.0.1 have few bugs that were fixed in the later version 5.0.2. Even if the version 5.0.1 is installed on Compute Canada clusters, users should use the latest one 5.0.2.<br />
<br />
To load the version 5.0.2, use:<br />
<br />
<code><br />
module load StdEnv/2020 gcc/10.3.0 openmpi/4.1.1 orca/5.0.2<br />
</code><br />
<br />
'''Note:''' The version 5.0.1 is kept on Compute Canada software stack but it may be removed any time.<br />
<br />
=== ORCA 4 ===<br />
<br />
The latest released version of ORCA 4 is "4.2.1". Other versions prior to this one are also available on Compute Canada software stack.<br />
<br />
To load the version 4.2.1, use:<br />
<br />
<code><br />
module load StdEnv/2020 gcc/9.3.0 openmpi/4.0.3 orca/4.2.1<br />
</code><br />
<br />
or<br />
<br />
<code><br />
module load nixpkgs/16.09 gcc/7.3.0 openmpi/3.1.4 orca/4.2.1<br />
</code><br />
<br />
== Setting ORCA input fies ==<br />
<br />
In addition to the different keywords required to run a given simulation, users should make sure to set two additional parameters:<br />
<br />
* Number of CPUs:<br />
<br />
* maxcore:<br />
<br />
==Using the software== <!--T:3--><br />
To see what versions of ORCA are currently available, type <code>module spider orca</code>. For detailed information about a specific version, including what other modules must be loaded first, use the module's full name. For example, <code>module spider orca/4.0.1.2</code>.<br />
<br />
<!--T:11--><br />
See [[Using modules]] for general guidance.<br />
<br />
===Job submission=== <!--T:4--><br />
For a general discussion about submitting jobs, see [[Running jobs]].<br />
<br />
<!--T:12--><br />
'''NOTE''': If you run into MPI errors with some of the ORCA executables, you can try to define the following variables:<br />
<br />
<!--T:13--><br />
export OMPI_MCA_mtl='^mxm'<br />
export OMPI_MCA_pml='^yalla'<br />
<br />
<!--T:5--><br />
The following is a job script to run ORCA using [[MPI]]:<br />
<br />
</translate><br />
{{File<br />
|name=run_orca.sh<br />
|lang="bash"<br />
|contents=<br />
#!/bin/bash<br />
#SBATCH --account=def-youPIs<br />
#SBATCH --ntasks=8 # cpus, the nprocs defined in the input file<br />
#SBATCH --mem-per-cpu=3G # memory per cpu<br />
#SBATCH --time=00-03:00 # time (DD-HH:MM)<br />
#SBATCH --output=benzene.log # output .log file<br />
<br />
module load StdEnv/2020 gcc/9.3.0 openmpi/4.0.3<br />
module load orca/4.2.1<br />
$EBROOTORCA/orca benzene.inp<br />
}}<br />
<translate><br />
<!--T:10--><br />
Example of the input file, benzene.inp:<br />
</translate><br />
{{File<br />
|name=benzene.inp<br />
|lang="text"<br />
|contents=<br />
# Benzene RHF Opt Calculation<br />
%pal nprocs 8 end<br />
! RHF TightSCF PModel<br />
! opt<br />
<br />
* xyz 0 1<br />
C 0.000000000000 1.398696930758 0.000000000000<br />
C 0.000000000000 -1.398696930758 0.000000000000<br />
C 1.211265339156 0.699329968382 0.000000000000<br />
C 1.211265339156 -0.699329968382 0.000000000000<br />
C -1.211265339156 0.699329968382 0.000000000000<br />
C -1.211265339156 -0.699329968382 0.000000000000<br />
H 0.000000000000 2.491406946734 0.000000000000<br />
H 0.000000000000 -2.491406946734 0.000000000000<br />
H 2.157597486829 1.245660462400 0.000000000000<br />
H 2.157597486829 -1.245660462400 0.000000000000<br />
H -2.157597486829 1.245660462400 0.000000000000<br />
H -2.157597486829 -1.245660462400 0.000000000000<br />
*<br />
}}<br />
<translate><br />
<br />
===Notes=== <!--T:15--><br />
* To make sure that the program runs efficiently and makes use of all the resources or the cores asked for in your job script, please add this line <code>%pal nprocs <ncores> end</code> to your input file as shown in the above example. Replace <code><ncores></code> by the number of cores you used in your script.<br />
<br />
===(Sep. 6 2019) Temporary fix to OpenMPI version inconsistency issue=== <!--T:16--><br />
For some type of calculations (DLPNO-STEOM-CCSD in particular), one could receive unknown openmpi related fatal errors. This could be due to using an older version of openmpi (''i.e.'' 3.1.2 as suggested by 'module' for both orca/4.1.0 and 4.2.0) than recommended officially (3.1.3 for orca/4.1.0 and 3.1.4 for orca/4.2.0). To temporarily fix this issue, one can build a custom version of openmpi.<br />
<br />
<!--T:17--><br />
The following two commands prepares a custom openmpi/3.1.4 for orca/4.2.0:<br />
module load gcc/7.3.0<br />
eb OpenMPI-3.1.2-GCC-7.3.0.eb --try-software-version=3.1.4<br />
When the building is finished, one can load the custom openmpi using module:<br />
module load openmpi/3.1.4<br />
At this step, one can manually install orca/4.2.0 binaries from the official forum under the home directory after finishing the registration on the official orca forum and being granted access to the orca program on Compute Canada clusters.<br />
<br />
<br />
<!--T:18--><br />
Additional notes from the contributor:<br />
<br />
<!--T:19--><br />
This is a '''temporary''' fix prior to the official upgrade of openmpi on Compute Canada clusters. Please remember to delete the manually installed orca binaries once the official openmpi version is up to date.<br />
<br />
<!--T:20--><br />
The compiling command does not seem to apply to openmpi/2.1.x.<br />
<br />
== Related links ==<br />
<br />
</translate></div>Kerrachehttps://docs.alliancecan.ca/mediawiki/index.php?title=ORCA&diff=110924ORCA2022-02-01T19:20:14Z<p>Kerrache: </p>
<hr />
<div><languages /><br />
[[Category:Software]][[Category:ComputationalChemistry]]<br />
<br />
<translate><br />
<br />
==Introduction== <!--T:1--><br />
ORCA is a flexible, efficient and easy-to-use general purpose tool for quantum chemistry with specific emphasis on spectroscopic properties of open-shell molecules. It features a wide variety of standard quantum chemical methods ranging from semiempirical methods to DFT to single- and multireference correlated ab initio methods. It can also treat environmental and relativistic effects.<br />
<br />
== Licensing == <!--T:2--><br />
If you wish to use pre-built ORCA executables:<br />
# You have to register at https://orcaforum.kofo.mpg.de/<br />
# You will receive a first email to verify the email address and activate the account. Follow the instructions in that email.<br />
# Once the registration is complete you will get a '''second email''' stating that the "''registration for ORCA download and usage has been completed''".<br />
# [[Technical support | Contact us]] requesting access to ORCA with a copy of the '''second email''' mentioned above.<br />
<br />
== ORCA versions ==<br />
<br />
On July 2021, a version 5 of ORCA was released. This is a major upgrade of ORCA 4.<br />
<br />
=== ORCA 5 ===<br />
<br />
The first released versions 5.0 and 5.0.1 have few bugs that were fixed in the later version 5.0.2. Even if the version 5.0.1 is installed on Compute Canada clusters, users should use the latest one 5.0.2.<br />
<br />
To load the version 5.0.2, use:<br />
<br />
<code><br />
module load StdEnv/2020 gcc/10.3.0 openmpi/4.1.1 orca/5.0.2<br />
</code><br />
<br />
'''Note:''' The version 5.0.1 is kept on Compute Canada software stack but it may be removed any time.<br />
<br />
=== ORCA 4 ===<br />
<br />
The latest released version of ORCA 4 is "4.2.1". Other versions prior to this one are also available on Compute Canada software stack.<br />
<br />
To load the version 4.2.1, use:<br />
<br />
<code><br />
module load StdEnv/2020 gcc/9.3.0 openmpi/4.0.3 orca/4.2.1<br />
</code><br />
<br />
or<br />
<br />
<code><br />
module load nixpkgs/16.09 gcc/7.3.0 openmpi/3.1.4 orca/4.2.1<br />
</code><br />
<br />
== ORCA performance ==<br />
<br />
==Using the software== <!--T:3--><br />
To see what versions of ORCA are currently available, type <code>module spider orca</code>. For detailed information about a specific version, including what other modules must be loaded first, use the module's full name. For example, <code>module spider orca/4.0.1.2</code>.<br />
<br />
<!--T:11--><br />
See [[Using modules]] for general guidance.<br />
<br />
===Job submission=== <!--T:4--><br />
For a general discussion about submitting jobs, see [[Running jobs]].<br />
<br />
<!--T:12--><br />
'''NOTE''': If you run into MPI errors with some of the ORCA executables, you can try to define the following variables:<br />
<br />
<!--T:13--><br />
export OMPI_MCA_mtl='^mxm'<br />
export OMPI_MCA_pml='^yalla'<br />
<br />
<!--T:5--><br />
The following is a job script to run ORCA using [[MPI]]:<br />
<br />
</translate><br />
{{File<br />
|name=run_orca.sh<br />
|lang="bash"<br />
|contents=<br />
#!/bin/bash<br />
#SBATCH --account=def-youPIs<br />
#SBATCH --ntasks=8 # cpus, the nprocs defined in the input file<br />
#SBATCH --mem-per-cpu=3G # memory per cpu<br />
#SBATCH --time=00-03:00 # time (DD-HH:MM)<br />
#SBATCH --output=benzene.log # output .log file<br />
<br />
module load StdEnv/2020 gcc/9.3.0 openmpi/4.0.3<br />
module load orca/4.2.1<br />
$EBROOTORCA/orca benzene.inp<br />
}}<br />
<translate><br />
<!--T:10--><br />
Example of the input file, benzene.inp:<br />
</translate><br />
{{File<br />
|name=benzene.inp<br />
|lang="text"<br />
|contents=<br />
# Benzene RHF Opt Calculation<br />
%pal nprocs 8 end<br />
! RHF TightSCF PModel<br />
! opt<br />
<br />
* xyz 0 1<br />
C 0.000000000000 1.398696930758 0.000000000000<br />
C 0.000000000000 -1.398696930758 0.000000000000<br />
C 1.211265339156 0.699329968382 0.000000000000<br />
C 1.211265339156 -0.699329968382 0.000000000000<br />
C -1.211265339156 0.699329968382 0.000000000000<br />
C -1.211265339156 -0.699329968382 0.000000000000<br />
H 0.000000000000 2.491406946734 0.000000000000<br />
H 0.000000000000 -2.491406946734 0.000000000000<br />
H 2.157597486829 1.245660462400 0.000000000000<br />
H 2.157597486829 -1.245660462400 0.000000000000<br />
H -2.157597486829 1.245660462400 0.000000000000<br />
H -2.157597486829 -1.245660462400 0.000000000000<br />
*<br />
}}<br />
<translate><br />
<br />
===Notes=== <!--T:15--><br />
* To make sure that the program runs efficiently and makes use of all the resources or the cores asked for in your job script, please add this line <code>%pal nprocs <ncores> end</code> to your input file as shown in the above example. Replace <code><ncores></code> by the number of cores you used in your script.<br />
<br />
===(Sep. 6 2019) Temporary fix to OpenMPI version inconsistency issue=== <!--T:16--><br />
For some type of calculations (DLPNO-STEOM-CCSD in particular), one could receive unknown openmpi related fatal errors. This could be due to using an older version of openmpi (''i.e.'' 3.1.2 as suggested by 'module' for both orca/4.1.0 and 4.2.0) than recommended officially (3.1.3 for orca/4.1.0 and 3.1.4 for orca/4.2.0). To temporarily fix this issue, one can build a custom version of openmpi.<br />
<br />
<!--T:17--><br />
The following two commands prepares a custom openmpi/3.1.4 for orca/4.2.0:<br />
module load gcc/7.3.0<br />
eb OpenMPI-3.1.2-GCC-7.3.0.eb --try-software-version=3.1.4<br />
When the building is finished, one can load the custom openmpi using module:<br />
module load openmpi/3.1.4<br />
At this step, one can manually install orca/4.2.0 binaries from the official forum under the home directory after finishing the registration on the official orca forum and being granted access to the orca program on Compute Canada clusters.<br />
<br />
<br />
<!--T:18--><br />
Additional notes from the contributor:<br />
<br />
<!--T:19--><br />
This is a '''temporary''' fix prior to the official upgrade of openmpi on Compute Canada clusters. Please remember to delete the manually installed orca binaries once the official openmpi version is up to date.<br />
<br />
<!--T:20--><br />
The compiling command does not seem to apply to openmpi/2.1.x.<br />
<br />
== Related links ==<br />
<br />
</translate></div>Kerrachehttps://docs.alliancecan.ca/mediawiki/index.php?title=ORCA&diff=110923ORCA2022-02-01T19:19:11Z<p>Kerrache: /* ORCA 4 */</p>
<hr />
<div><languages /><br />
[[Category:Software]][[Category:ComputationalChemistry]]<br />
<br />
<translate><br />
<br />
==Introduction== <!--T:1--><br />
ORCA is a flexible, efficient and easy-to-use general purpose tool for quantum chemistry with specific emphasis on spectroscopic properties of open-shell molecules. It features a wide variety of standard quantum chemical methods ranging from semiempirical methods to DFT to single- and multireference correlated ab initio methods. It can also treat environmental and relativistic effects.<br />
<br />
== Licensing == <!--T:2--><br />
If you wish to use pre-built ORCA executables:<br />
# You have to register at https://orcaforum.kofo.mpg.de/<br />
# You will receive a first email to verify the email address and activate the account. Follow the instructions in that email.<br />
# Once the registration is complete you will get a '''second email''' stating that the "''registration for ORCA download and usage has been completed''".<br />
# [[Technical support | Contact us]] requesting access to ORCA with a copy of the '''second email''' mentioned above.<br />
<br />
== ORCA versions ==<br />
<br />
On July 2021, a version 5 of ORCA was released. This is a major upgrade of ORCA 4.<br />
<br />
=== ORCA 5 ===<br />
<br />
The first released versions 5.0 and 5.0.1 have few bugs that were fixed in the later version 5.0.2. Even if the version 5.0.1 is installed on Compute Canada clusters, users should use the latest one 5.0.2.<br />
<br />
To load the version 5.0.2, use:<br />
<br />
<code><br />
module load StdEnv/2020 gcc/10.3.0 openmpi/4.1.1 orca/5.0.2<br />
</code><br />
<br />
'''Note:''' The version 5.0.1 is kept on Compute Canada software stack but it may be removed any time.<br />
<br />
=== ORCA 4 ===<br />
<br />
The latest released version of ORCA 4 is "4.2.1". Other versions prior to this one are also available on Compute Canada software stack.<br />
<br />
To load the version 4.2.1, use:<br />
<br />
<code><br />
module load StdEnv/2020 gcc/9.3.0 openmpi/4.0.3 orca/4.2.1<br />
</code><br />
<br />
or<br />
<br />
<code><br />
module load nixpkgs/16.09 gcc/7.3.0 openmpi/3.1.4 orca/4.2.1<br />
</code><br />
<br />
== ORCA performance ==<br />
<br />
==Using the software== <!--T:3--><br />
To see what versions of ORCA are currently available, type <code>module spider orca</code>. For detailed information about a specific version, including what other modules must be loaded first, use the module's full name. For example, <code>module spider orca/4.0.1.2</code>.<br />
<br />
<!--T:11--><br />
See [[Using modules]] for general guidance.<br />
<br />
===Job submission=== <!--T:4--><br />
For a general discussion about submitting jobs, see [[Running jobs]].<br />
<br />
<!--T:12--><br />
'''NOTE''': If you run into MPI errors with some of the ORCA executables, you can try to define the following variables:<br />
<br />
<!--T:13--><br />
export OMPI_MCA_mtl='^mxm'<br />
export OMPI_MCA_pml='^yalla'<br />
<br />
<!--T:5--><br />
The following is a job script to run ORCA using [[MPI]]:<br />
<br />
</translate><br />
{{File<br />
|name=run_orca.sh<br />
|lang="bash"<br />
|contents=<br />
#!/bin/bash<br />
#SBATCH --account=def-youPIs<br />
#SBATCH --ntasks=8 # cpus, the nprocs defined in the input file<br />
#SBATCH --mem-per-cpu=3G # memory per cpu<br />
#SBATCH --time=00-03:00 # time (DD-HH:MM)<br />
#SBATCH --output=benzene.log # output .log file<br />
<br />
module load StdEnv/2020 gcc/9.3.0 openmpi/4.0.3<br />
module load orca/4.2.1<br />
$EBROOTORCA/orca benzene.inp<br />
}}<br />
<translate><br />
<!--T:10--><br />
Example of the input file, benzene.inp:<br />
</translate><br />
{{File<br />
|name=benzene.inp<br />
|lang="text"<br />
|contents=<br />
# Benzene RHF Opt Calculation<br />
%pal nprocs 8 end<br />
! RHF TightSCF PModel<br />
! opt<br />
<br />
* xyz 0 1<br />
C 0.000000000000 1.398696930758 0.000000000000<br />
C 0.000000000000 -1.398696930758 0.000000000000<br />
C 1.211265339156 0.699329968382 0.000000000000<br />
C 1.211265339156 -0.699329968382 0.000000000000<br />
C -1.211265339156 0.699329968382 0.000000000000<br />
C -1.211265339156 -0.699329968382 0.000000000000<br />
H 0.000000000000 2.491406946734 0.000000000000<br />
H 0.000000000000 -2.491406946734 0.000000000000<br />
H 2.157597486829 1.245660462400 0.000000000000<br />
H 2.157597486829 -1.245660462400 0.000000000000<br />
H -2.157597486829 1.245660462400 0.000000000000<br />
H -2.157597486829 -1.245660462400 0.000000000000<br />
*<br />
}}<br />
<translate><br />
<br />
===Notes=== <!--T:15--><br />
* To make sure that the program runs efficiently and makes use of all the resources or the cores asked for in your job script, please add this line <code>%pal nprocs <ncores> end</code> to your input file as shown in the above example. Replace <code><ncores></code> by the number of cores you used in your script.<br />
<br />
===(Sep. 6 2019) Temporary fix to OpenMPI version inconsistency issue=== <!--T:16--><br />
For some type of calculations (DLPNO-STEOM-CCSD in particular), one could receive unknown openmpi related fatal errors. This could be due to using an older version of openmpi (''i.e.'' 3.1.2 as suggested by 'module' for both orca/4.1.0 and 4.2.0) than recommended officially (3.1.3 for orca/4.1.0 and 3.1.4 for orca/4.2.0). To temporarily fix this issue, one can build a custom version of openmpi.<br />
<br />
<!--T:17--><br />
The following two commands prepares a custom openmpi/3.1.4 for orca/4.2.0:<br />
module load gcc/7.3.0<br />
eb OpenMPI-3.1.2-GCC-7.3.0.eb --try-software-version=3.1.4<br />
When the building is finished, one can load the custom openmpi using module:<br />
module load openmpi/3.1.4<br />
At this step, one can manually install orca/4.2.0 binaries from the official forum under the home directory after finishing the registration on the official orca forum and being granted access to the orca program on Compute Canada clusters.<br />
<br />
<br />
<!--T:18--><br />
Additional notes from the contributor:<br />
<br />
<!--T:19--><br />
This is a '''temporary''' fix prior to the official upgrade of openmpi on Compute Canada clusters. Please remember to delete the manually installed orca binaries once the official openmpi version is up to date.<br />
<br />
<!--T:20--><br />
The compiling command does not seem to apply to openmpi/2.1.x.<br />
<br />
</translate></div>Kerrachehttps://docs.alliancecan.ca/mediawiki/index.php?title=ORCA&diff=110922ORCA2022-02-01T19:18:51Z<p>Kerrache: /* ORCA 5 */</p>
<hr />
<div><languages /><br />
[[Category:Software]][[Category:ComputationalChemistry]]<br />
<br />
<translate><br />
<br />
==Introduction== <!--T:1--><br />
ORCA is a flexible, efficient and easy-to-use general purpose tool for quantum chemistry with specific emphasis on spectroscopic properties of open-shell molecules. It features a wide variety of standard quantum chemical methods ranging from semiempirical methods to DFT to single- and multireference correlated ab initio methods. It can also treat environmental and relativistic effects.<br />
<br />
== Licensing == <!--T:2--><br />
If you wish to use pre-built ORCA executables:<br />
# You have to register at https://orcaforum.kofo.mpg.de/<br />
# You will receive a first email to verify the email address and activate the account. Follow the instructions in that email.<br />
# Once the registration is complete you will get a '''second email''' stating that the "''registration for ORCA download and usage has been completed''".<br />
# [[Technical support | Contact us]] requesting access to ORCA with a copy of the '''second email''' mentioned above.<br />
<br />
== ORCA versions ==<br />
<br />
On July 2021, a version 5 of ORCA was released. This is a major upgrade of ORCA 4.<br />
<br />
=== ORCA 5 ===<br />
<br />
The first released versions 5.0 and 5.0.1 have few bugs that were fixed in the later version 5.0.2. Even if the version 5.0.1 is installed on Compute Canada clusters, users should use the latest one 5.0.2.<br />
<br />
To load the version 5.0.2, use:<br />
<br />
<code><br />
module load StdEnv/2020 gcc/10.3.0 openmpi/4.1.1 orca/5.0.2<br />
</code><br />
<br />
'''Note:''' The version 5.0.1 is kept on Compute Canada software stack but it may be removed any time.<br />
<br />
== ORCA 4 ==<br />
<br />
The latest released version of ORCA 4 is "4.2.1". Other versions prior to this one are also available on Compute Canada software stack.<br />
<br />
To load the version 4.2.1, use:<br />
<br />
<code><br />
module load StdEnv/2020 gcc/9.3.0 openmpi/4.0.3 orca/4.2.1<br />
</code><br />
<br />
or<br />
<br />
<code><br />
module load nixpkgs/16.09 gcc/7.3.0 openmpi/3.1.4 orca/4.2.1<br />
</code><br />
<br />
== ORCA performance ==<br />
<br />
==Using the software== <!--T:3--><br />
To see what versions of ORCA are currently available, type <code>module spider orca</code>. For detailed information about a specific version, including what other modules must be loaded first, use the module's full name. For example, <code>module spider orca/4.0.1.2</code>.<br />
<br />
<!--T:11--><br />
See [[Using modules]] for general guidance.<br />
<br />
===Job submission=== <!--T:4--><br />
For a general discussion about submitting jobs, see [[Running jobs]].<br />
<br />
<!--T:12--><br />
'''NOTE''': If you run into MPI errors with some of the ORCA executables, you can try to define the following variables:<br />
<br />
<!--T:13--><br />
export OMPI_MCA_mtl='^mxm'<br />
export OMPI_MCA_pml='^yalla'<br />
<br />
<!--T:5--><br />
The following is a job script to run ORCA using [[MPI]]:<br />
<br />
</translate><br />
{{File<br />
|name=run_orca.sh<br />
|lang="bash"<br />
|contents=<br />
#!/bin/bash<br />
#SBATCH --account=def-youPIs<br />
#SBATCH --ntasks=8 # cpus, the nprocs defined in the input file<br />
#SBATCH --mem-per-cpu=3G # memory per cpu<br />
#SBATCH --time=00-03:00 # time (DD-HH:MM)<br />
#SBATCH --output=benzene.log # output .log file<br />
<br />
module load StdEnv/2020 gcc/9.3.0 openmpi/4.0.3<br />
module load orca/4.2.1<br />
$EBROOTORCA/orca benzene.inp<br />
}}<br />
<translate><br />
<!--T:10--><br />
Example of the input file, benzene.inp:<br />
</translate><br />
{{File<br />
|name=benzene.inp<br />
|lang="text"<br />
|contents=<br />
# Benzene RHF Opt Calculation<br />
%pal nprocs 8 end<br />
! RHF TightSCF PModel<br />
! opt<br />
<br />
* xyz 0 1<br />
C 0.000000000000 1.398696930758 0.000000000000<br />
C 0.000000000000 -1.398696930758 0.000000000000<br />
C 1.211265339156 0.699329968382 0.000000000000<br />
C 1.211265339156 -0.699329968382 0.000000000000<br />
C -1.211265339156 0.699329968382 0.000000000000<br />
C -1.211265339156 -0.699329968382 0.000000000000<br />
H 0.000000000000 2.491406946734 0.000000000000<br />
H 0.000000000000 -2.491406946734 0.000000000000<br />
H 2.157597486829 1.245660462400 0.000000000000<br />
H 2.157597486829 -1.245660462400 0.000000000000<br />
H -2.157597486829 1.245660462400 0.000000000000<br />
H -2.157597486829 -1.245660462400 0.000000000000<br />
*<br />
}}<br />
<translate><br />
<br />
===Notes=== <!--T:15--><br />
* To make sure that the program runs efficiently and makes use of all the resources or the cores asked for in your job script, please add this line <code>%pal nprocs <ncores> end</code> to your input file as shown in the above example. Replace <code><ncores></code> by the number of cores you used in your script.<br />
<br />
===(Sep. 6 2019) Temporary fix to OpenMPI version inconsistency issue=== <!--T:16--><br />
For some type of calculations (DLPNO-STEOM-CCSD in particular), one could receive unknown openmpi related fatal errors. This could be due to using an older version of openmpi (''i.e.'' 3.1.2 as suggested by 'module' for both orca/4.1.0 and 4.2.0) than recommended officially (3.1.3 for orca/4.1.0 and 3.1.4 for orca/4.2.0). To temporarily fix this issue, one can build a custom version of openmpi.<br />
<br />
<!--T:17--><br />
The following two commands prepares a custom openmpi/3.1.4 for orca/4.2.0:<br />
module load gcc/7.3.0<br />
eb OpenMPI-3.1.2-GCC-7.3.0.eb --try-software-version=3.1.4<br />
When the building is finished, one can load the custom openmpi using module:<br />
module load openmpi/3.1.4<br />
At this step, one can manually install orca/4.2.0 binaries from the official forum under the home directory after finishing the registration on the official orca forum and being granted access to the orca program on Compute Canada clusters.<br />
<br />
<br />
<!--T:18--><br />
Additional notes from the contributor:<br />
<br />
<!--T:19--><br />
This is a '''temporary''' fix prior to the official upgrade of openmpi on Compute Canada clusters. Please remember to delete the manually installed orca binaries once the official openmpi version is up to date.<br />
<br />
<!--T:20--><br />
The compiling command does not seem to apply to openmpi/2.1.x.<br />
<br />
</translate></div>Kerrachehttps://docs.alliancecan.ca/mediawiki/index.php?title=ORCA&diff=110920ORCA2022-02-01T19:18:29Z<p>Kerrache: </p>
<hr />
<div><languages /><br />
[[Category:Software]][[Category:ComputationalChemistry]]<br />
<br />
<translate><br />
<br />
==Introduction== <!--T:1--><br />
ORCA is a flexible, efficient and easy-to-use general purpose tool for quantum chemistry with specific emphasis on spectroscopic properties of open-shell molecules. It features a wide variety of standard quantum chemical methods ranging from semiempirical methods to DFT to single- and multireference correlated ab initio methods. It can also treat environmental and relativistic effects.<br />
<br />
== Licensing == <!--T:2--><br />
If you wish to use pre-built ORCA executables:<br />
# You have to register at https://orcaforum.kofo.mpg.de/<br />
# You will receive a first email to verify the email address and activate the account. Follow the instructions in that email.<br />
# Once the registration is complete you will get a '''second email''' stating that the "''registration for ORCA download and usage has been completed''".<br />
# [[Technical support | Contact us]] requesting access to ORCA with a copy of the '''second email''' mentioned above.<br />
<br />
== ORCA versions ==<br />
<br />
On July 2021, a version 5 of ORCA was released. This is a major upgrade of ORCA 4.<br />
<br />
== ORCA 5 ==<br />
<br />
The first released versions 5.0 and 5.0.1 have few bugs that were fixed in the later version 5.0.2. Even if the version 5.0.1 is installed on Compute Canada clusters, users should use the latest one 5.0.2.<br />
<br />
To load the version 5.0.2, use:<br />
<br />
<code><br />
module load StdEnv/2020 gcc/10.3.0 openmpi/4.1.1 orca/5.0.2<br />
</code><br />
<br />
'''Note:''' The version 5.0.1 is kept on Compute Canada software stack but it may be removed any time.<br />
<br />
== ORCA 4 ==<br />
<br />
The latest released version of ORCA 4 is "4.2.1". Other versions prior to this one are also available on Compute Canada software stack.<br />
<br />
To load the version 4.2.1, use:<br />
<br />
<code><br />
module load StdEnv/2020 gcc/9.3.0 openmpi/4.0.3 orca/4.2.1<br />
</code><br />
<br />
or<br />
<br />
<code><br />
module load nixpkgs/16.09 gcc/7.3.0 openmpi/3.1.4 orca/4.2.1<br />
</code><br />
<br />
== ORCA performance ==<br />
<br />
==Using the software== <!--T:3--><br />
To see what versions of ORCA are currently available, type <code>module spider orca</code>. For detailed information about a specific version, including what other modules must be loaded first, use the module's full name. For example, <code>module spider orca/4.0.1.2</code>.<br />
<br />
<!--T:11--><br />
See [[Using modules]] for general guidance.<br />
<br />
===Job submission=== <!--T:4--><br />
For a general discussion about submitting jobs, see [[Running jobs]].<br />
<br />
<!--T:12--><br />
'''NOTE''': If you run into MPI errors with some of the ORCA executables, you can try to define the following variables:<br />
<br />
<!--T:13--><br />
export OMPI_MCA_mtl='^mxm'<br />
export OMPI_MCA_pml='^yalla'<br />
<br />
<!--T:5--><br />
The following is a job script to run ORCA using [[MPI]]:<br />
<br />
</translate><br />
{{File<br />
|name=run_orca.sh<br />
|lang="bash"<br />
|contents=<br />
#!/bin/bash<br />
#SBATCH --account=def-youPIs<br />
#SBATCH --ntasks=8 # cpus, the nprocs defined in the input file<br />
#SBATCH --mem-per-cpu=3G # memory per cpu<br />
#SBATCH --time=00-03:00 # time (DD-HH:MM)<br />
#SBATCH --output=benzene.log # output .log file<br />
<br />
module load StdEnv/2020 gcc/9.3.0 openmpi/4.0.3<br />
module load orca/4.2.1<br />
$EBROOTORCA/orca benzene.inp<br />
}}<br />
<translate><br />
<!--T:10--><br />
Example of the input file, benzene.inp:<br />
</translate><br />
{{File<br />
|name=benzene.inp<br />
|lang="text"<br />
|contents=<br />
# Benzene RHF Opt Calculation<br />
%pal nprocs 8 end<br />
! RHF TightSCF PModel<br />
! opt<br />
<br />
* xyz 0 1<br />
C 0.000000000000 1.398696930758 0.000000000000<br />
C 0.000000000000 -1.398696930758 0.000000000000<br />
C 1.211265339156 0.699329968382 0.000000000000<br />
C 1.211265339156 -0.699329968382 0.000000000000<br />
C -1.211265339156 0.699329968382 0.000000000000<br />
C -1.211265339156 -0.699329968382 0.000000000000<br />
H 0.000000000000 2.491406946734 0.000000000000<br />
H 0.000000000000 -2.491406946734 0.000000000000<br />
H 2.157597486829 1.245660462400 0.000000000000<br />
H 2.157597486829 -1.245660462400 0.000000000000<br />
H -2.157597486829 1.245660462400 0.000000000000<br />
H -2.157597486829 -1.245660462400 0.000000000000<br />
*<br />
}}<br />
<translate><br />
<br />
===Notes=== <!--T:15--><br />
* To make sure that the program runs efficiently and makes use of all the resources or the cores asked for in your job script, please add this line <code>%pal nprocs <ncores> end</code> to your input file as shown in the above example. Replace <code><ncores></code> by the number of cores you used in your script.<br />
<br />
===(Sep. 6 2019) Temporary fix to OpenMPI version inconsistency issue=== <!--T:16--><br />
For some type of calculations (DLPNO-STEOM-CCSD in particular), one could receive unknown openmpi related fatal errors. This could be due to using an older version of openmpi (''i.e.'' 3.1.2 as suggested by 'module' for both orca/4.1.0 and 4.2.0) than recommended officially (3.1.3 for orca/4.1.0 and 3.1.4 for orca/4.2.0). To temporarily fix this issue, one can build a custom version of openmpi.<br />
<br />
<!--T:17--><br />
The following two commands prepares a custom openmpi/3.1.4 for orca/4.2.0:<br />
module load gcc/7.3.0<br />
eb OpenMPI-3.1.2-GCC-7.3.0.eb --try-software-version=3.1.4<br />
When the building is finished, one can load the custom openmpi using module:<br />
module load openmpi/3.1.4<br />
At this step, one can manually install orca/4.2.0 binaries from the official forum under the home directory after finishing the registration on the official orca forum and being granted access to the orca program on Compute Canada clusters.<br />
<br />
<br />
<!--T:18--><br />
Additional notes from the contributor:<br />
<br />
<!--T:19--><br />
This is a '''temporary''' fix prior to the official upgrade of openmpi on Compute Canada clusters. Please remember to delete the manually installed orca binaries once the official openmpi version is up to date.<br />
<br />
<!--T:20--><br />
The compiling command does not seem to apply to openmpi/2.1.x.<br />
<br />
</translate></div>Kerrachehttps://docs.alliancecan.ca/mediawiki/index.php?title=ORCA&diff=110912ORCA2022-02-01T19:15:01Z<p>Kerrache: /* ORCA 4 */</p>
<hr />
<div><languages /><br />
[[Category:Software]][[Category:ComputationalChemistry]]<br />
<br />
<translate><br />
<br />
==Introduction== <!--T:1--><br />
ORCA is a flexible, efficient and easy-to-use general purpose tool for quantum chemistry with specific emphasis on spectroscopic properties of open-shell molecules. It features a wide variety of standard quantum chemical methods ranging from semiempirical methods to DFT to single- and multireference correlated ab initio methods. It can also treat environmental and relativistic effects.<br />
<br />
== Licensing == <!--T:2--><br />
If you wish to use pre-built ORCA executables:<br />
# You have to register at https://orcaforum.kofo.mpg.de/<br />
# You will receive a first email to verify the email address and activate the account. Follow the instructions in that email.<br />
# Once the registration is complete you will get a '''second email''' stating that the "''registration for ORCA download and usage has been completed''".<br />
# [[Technical support | Contact us]] requesting access to ORCA with a copy of the '''second email''' mentioned above.<br />
<br />
== ORCA versions ==<br />
<br />
On July 2021, a version 5 of ORCA was released. This is a major upgrade of ORCA 4.<br />
<br />
== ORCA 5 ==<br />
<br />
The first released versions 5.0 and 5.0.1 have few bugs that were fixed in the later version 5.0.2. Even if the version 5.0.1 is installed on Compute Canada clusters, users should use the latest one 5.0.2.<br />
<br />
To load the version 5.0.2, use:<br />
<br />
<code><br />
module load StdEnv/2020 gcc/10.3.0 openmpi/4.1.1 orca/5.0.2<br />
</code><br />
<br />
'''Note:''' The version 5.0.1 is kept on our software stack but it may be removed any time.<br />
<br />
== ORCA 4 ==<br />
<br />
The latest released version of ORCA 4 is "4.2.1". Other versions prior to this one are also available on Compute Canada software stack.<br />
<br />
== ORCA performance ==<br />
<br />
==Using the software== <!--T:3--><br />
To see what versions of ORCA are currently available, type <code>module spider orca</code>. For detailed information about a specific version, including what other modules must be loaded first, use the module's full name. For example, <code>module spider orca/4.0.1.2</code>.<br />
<br />
<!--T:11--><br />
See [[Using modules]] for general guidance.<br />
<br />
===Job submission=== <!--T:4--><br />
For a general discussion about submitting jobs, see [[Running jobs]].<br />
<br />
<!--T:12--><br />
'''NOTE''': If you run into MPI errors with some of the ORCA executables, you can try to define the following variables:<br />
<br />
<!--T:13--><br />
export OMPI_MCA_mtl='^mxm'<br />
export OMPI_MCA_pml='^yalla'<br />
<br />
<!--T:5--><br />
The following is a job script to run ORCA using [[MPI]]:<br />
<br />
</translate><br />
{{File<br />
|name=run_orca.sh<br />
|lang="bash"<br />
|contents=<br />
#!/bin/bash<br />
#SBATCH --account=def-youPIs<br />
#SBATCH --ntasks=8 # cpus, the nprocs defined in the input file<br />
#SBATCH --mem-per-cpu=3G # memory per cpu<br />
#SBATCH --time=00-03:00 # time (DD-HH:MM)<br />
#SBATCH --output=benzene.log # output .log file<br />
<br />
module load StdEnv/2020 gcc/9.3.0 openmpi/4.0.3<br />
module load orca/4.2.1<br />
$EBROOTORCA/orca benzene.inp<br />
}}<br />
<translate><br />
<!--T:10--><br />
Example of the input file, benzene.inp:<br />
</translate><br />
{{File<br />
|name=benzene.inp<br />
|lang="text"<br />
|contents=<br />
# Benzene RHF Opt Calculation<br />
%pal nprocs 8 end<br />
! RHF TightSCF PModel<br />
! opt<br />
<br />
* xyz 0 1<br />
C 0.000000000000 1.398696930758 0.000000000000<br />
C 0.000000000000 -1.398696930758 0.000000000000<br />
C 1.211265339156 0.699329968382 0.000000000000<br />
C 1.211265339156 -0.699329968382 0.000000000000<br />
C -1.211265339156 0.699329968382 0.000000000000<br />
C -1.211265339156 -0.699329968382 0.000000000000<br />
H 0.000000000000 2.491406946734 0.000000000000<br />
H 0.000000000000 -2.491406946734 0.000000000000<br />
H 2.157597486829 1.245660462400 0.000000000000<br />
H 2.157597486829 -1.245660462400 0.000000000000<br />
H -2.157597486829 1.245660462400 0.000000000000<br />
H -2.157597486829 -1.245660462400 0.000000000000<br />
*<br />
}}<br />
<translate><br />
<br />
===Notes=== <!--T:15--><br />
* To make sure that the program runs efficiently and makes use of all the resources or the cores asked for in your job script, please add this line <code>%pal nprocs <ncores> end</code> to your input file as shown in the above example. Replace <code><ncores></code> by the number of cores you used in your script.<br />
<br />
===(Sep. 6 2019) Temporary fix to OpenMPI version inconsistency issue=== <!--T:16--><br />
For some type of calculations (DLPNO-STEOM-CCSD in particular), one could receive unknown openmpi related fatal errors. This could be due to using an older version of openmpi (''i.e.'' 3.1.2 as suggested by 'module' for both orca/4.1.0 and 4.2.0) than recommended officially (3.1.3 for orca/4.1.0 and 3.1.4 for orca/4.2.0). To temporarily fix this issue, one can build a custom version of openmpi.<br />
<br />
<!--T:17--><br />
The following two commands prepares a custom openmpi/3.1.4 for orca/4.2.0:<br />
module load gcc/7.3.0<br />
eb OpenMPI-3.1.2-GCC-7.3.0.eb --try-software-version=3.1.4<br />
When the building is finished, one can load the custom openmpi using module:<br />
module load openmpi/3.1.4<br />
At this step, one can manually install orca/4.2.0 binaries from the official forum under the home directory after finishing the registration on the official orca forum and being granted access to the orca program on Compute Canada clusters.<br />
<br />
<br />
<!--T:18--><br />
Additional notes from the contributor:<br />
<br />
<!--T:19--><br />
This is a '''temporary''' fix prior to the official upgrade of openmpi on Compute Canada clusters. Please remember to delete the manually installed orca binaries once the official openmpi version is up to date.<br />
<br />
<!--T:20--><br />
The compiling command does not seem to apply to openmpi/2.1.x.<br />
<br />
</translate></div>Kerrachehttps://docs.alliancecan.ca/mediawiki/index.php?title=ORCA&diff=110911ORCA2022-02-01T19:11:52Z<p>Kerrache: /* ORCA 5 */</p>
<hr />
<div><languages /><br />
[[Category:Software]][[Category:ComputationalChemistry]]<br />
<br />
<translate><br />
<br />
==Introduction== <!--T:1--><br />
ORCA is a flexible, efficient and easy-to-use general purpose tool for quantum chemistry with specific emphasis on spectroscopic properties of open-shell molecules. It features a wide variety of standard quantum chemical methods ranging from semiempirical methods to DFT to single- and multireference correlated ab initio methods. It can also treat environmental and relativistic effects.<br />
<br />
== Licensing == <!--T:2--><br />
If you wish to use pre-built ORCA executables:<br />
# You have to register at https://orcaforum.kofo.mpg.de/<br />
# You will receive a first email to verify the email address and activate the account. Follow the instructions in that email.<br />
# Once the registration is complete you will get a '''second email''' stating that the "''registration for ORCA download and usage has been completed''".<br />
# [[Technical support | Contact us]] requesting access to ORCA with a copy of the '''second email''' mentioned above.<br />
<br />
== ORCA versions ==<br />
<br />
On July 2021, a version 5 of ORCA was released. This is a major upgrade of ORCA 4.<br />
<br />
== ORCA 5 ==<br />
<br />
The first released versions 5.0 and 5.0.1 have few bugs that were fixed in the later version 5.0.2. Even if the version 5.0.1 is installed on Compute Canada clusters, users should use the latest one 5.0.2.<br />
<br />
To load the version 5.0.2, use:<br />
<br />
<code><br />
module load StdEnv/2020 gcc/10.3.0 openmpi/4.1.1 orca/5.0.2<br />
</code><br />
<br />
'''Note:''' The version 5.0.1 is kept on our software stack but it may be removed any time.<br />
<br />
== ORCA 4 ==<br />
<br />
== ORCA performance ==<br />
<br />
==Using the software== <!--T:3--><br />
To see what versions of ORCA are currently available, type <code>module spider orca</code>. For detailed information about a specific version, including what other modules must be loaded first, use the module's full name. For example, <code>module spider orca/4.0.1.2</code>.<br />
<br />
<!--T:11--><br />
See [[Using modules]] for general guidance.<br />
<br />
===Job submission=== <!--T:4--><br />
For a general discussion about submitting jobs, see [[Running jobs]].<br />
<br />
<!--T:12--><br />
'''NOTE''': If you run into MPI errors with some of the ORCA executables, you can try to define the following variables:<br />
<br />
<!--T:13--><br />
export OMPI_MCA_mtl='^mxm'<br />
export OMPI_MCA_pml='^yalla'<br />
<br />
<!--T:5--><br />
The following is a job script to run ORCA using [[MPI]]:<br />
<br />
</translate><br />
{{File<br />
|name=run_orca.sh<br />
|lang="bash"<br />
|contents=<br />
#!/bin/bash<br />
#SBATCH --account=def-youPIs<br />
#SBATCH --ntasks=8 # cpus, the nprocs defined in the input file<br />
#SBATCH --mem-per-cpu=3G # memory per cpu<br />
#SBATCH --time=00-03:00 # time (DD-HH:MM)<br />
#SBATCH --output=benzene.log # output .log file<br />
<br />
module load StdEnv/2020 gcc/9.3.0 openmpi/4.0.3<br />
module load orca/4.2.1<br />
$EBROOTORCA/orca benzene.inp<br />
}}<br />
<translate><br />
<!--T:10--><br />
Example of the input file, benzene.inp:<br />
</translate><br />
{{File<br />
|name=benzene.inp<br />
|lang="text"<br />
|contents=<br />
# Benzene RHF Opt Calculation<br />
%pal nprocs 8 end<br />
! RHF TightSCF PModel<br />
! opt<br />
<br />
* xyz 0 1<br />
C 0.000000000000 1.398696930758 0.000000000000<br />
C 0.000000000000 -1.398696930758 0.000000000000<br />
C 1.211265339156 0.699329968382 0.000000000000<br />
C 1.211265339156 -0.699329968382 0.000000000000<br />
C -1.211265339156 0.699329968382 0.000000000000<br />
C -1.211265339156 -0.699329968382 0.000000000000<br />
H 0.000000000000 2.491406946734 0.000000000000<br />
H 0.000000000000 -2.491406946734 0.000000000000<br />
H 2.157597486829 1.245660462400 0.000000000000<br />
H 2.157597486829 -1.245660462400 0.000000000000<br />
H -2.157597486829 1.245660462400 0.000000000000<br />
H -2.157597486829 -1.245660462400 0.000000000000<br />
*<br />
}}<br />
<translate><br />
<br />
===Notes=== <!--T:15--><br />
* To make sure that the program runs efficiently and makes use of all the resources or the cores asked for in your job script, please add this line <code>%pal nprocs <ncores> end</code> to your input file as shown in the above example. Replace <code><ncores></code> by the number of cores you used in your script.<br />
<br />
===(Sep. 6 2019) Temporary fix to OpenMPI version inconsistency issue=== <!--T:16--><br />
For some type of calculations (DLPNO-STEOM-CCSD in particular), one could receive unknown openmpi related fatal errors. This could be due to using an older version of openmpi (''i.e.'' 3.1.2 as suggested by 'module' for both orca/4.1.0 and 4.2.0) than recommended officially (3.1.3 for orca/4.1.0 and 3.1.4 for orca/4.2.0). To temporarily fix this issue, one can build a custom version of openmpi.<br />
<br />
<!--T:17--><br />
The following two commands prepares a custom openmpi/3.1.4 for orca/4.2.0:<br />
module load gcc/7.3.0<br />
eb OpenMPI-3.1.2-GCC-7.3.0.eb --try-software-version=3.1.4<br />
When the building is finished, one can load the custom openmpi using module:<br />
module load openmpi/3.1.4<br />
At this step, one can manually install orca/4.2.0 binaries from the official forum under the home directory after finishing the registration on the official orca forum and being granted access to the orca program on Compute Canada clusters.<br />
<br />
<br />
<!--T:18--><br />
Additional notes from the contributor:<br />
<br />
<!--T:19--><br />
This is a '''temporary''' fix prior to the official upgrade of openmpi on Compute Canada clusters. Please remember to delete the manually installed orca binaries once the official openmpi version is up to date.<br />
<br />
<!--T:20--><br />
The compiling command does not seem to apply to openmpi/2.1.x.<br />
<br />
</translate></div>Kerrachehttps://docs.alliancecan.ca/mediawiki/index.php?title=ORCA&diff=110909ORCA2022-02-01T19:11:24Z<p>Kerrache: /* ORCA 5 */</p>
<hr />
<div><languages /><br />
[[Category:Software]][[Category:ComputationalChemistry]]<br />
<br />
<translate><br />
<br />
==Introduction== <!--T:1--><br />
ORCA is a flexible, efficient and easy-to-use general purpose tool for quantum chemistry with specific emphasis on spectroscopic properties of open-shell molecules. It features a wide variety of standard quantum chemical methods ranging from semiempirical methods to DFT to single- and multireference correlated ab initio methods. It can also treat environmental and relativistic effects.<br />
<br />
== Licensing == <!--T:2--><br />
If you wish to use pre-built ORCA executables:<br />
# You have to register at https://orcaforum.kofo.mpg.de/<br />
# You will receive a first email to verify the email address and activate the account. Follow the instructions in that email.<br />
# Once the registration is complete you will get a '''second email''' stating that the "''registration for ORCA download and usage has been completed''".<br />
# [[Technical support | Contact us]] requesting access to ORCA with a copy of the '''second email''' mentioned above.<br />
<br />
== ORCA versions ==<br />
<br />
On July 2021, a version 5 of ORCA was released. This is a major upgrade of ORCA 4.<br />
<br />
== ORCA 5 ==<br />
<br />
The first released versions 5.0 and 5.0.1 have few bugs that were fixed in the later version 5.0.2. Even if the version 5.0.1 is installed on Compute Canada clusters, users should use the latest one 5.0.2.<br />
<br />
To load the version 5.0.2, use:<br />
<br />
<code><br />
module load StdEnv/2020 gcc/10.3.0 openmpi/4.1.1 orca/5.0.2<br />
</code><br />
<br />
'''Note:''' The version 5.0.1 is kept on our software stack but it may be removed any time. Users are encouraged to use the latest version 5.0.2.<br />
<br />
== ORCA 4 ==<br />
<br />
== ORCA performance ==<br />
<br />
==Using the software== <!--T:3--><br />
To see what versions of ORCA are currently available, type <code>module spider orca</code>. For detailed information about a specific version, including what other modules must be loaded first, use the module's full name. For example, <code>module spider orca/4.0.1.2</code>.<br />
<br />
<!--T:11--><br />
See [[Using modules]] for general guidance.<br />
<br />
===Job submission=== <!--T:4--><br />
For a general discussion about submitting jobs, see [[Running jobs]].<br />
<br />
<!--T:12--><br />
'''NOTE''': If you run into MPI errors with some of the ORCA executables, you can try to define the following variables:<br />
<br />
<!--T:13--><br />
export OMPI_MCA_mtl='^mxm'<br />
export OMPI_MCA_pml='^yalla'<br />
<br />
<!--T:5--><br />
The following is a job script to run ORCA using [[MPI]]:<br />
<br />
</translate><br />
{{File<br />
|name=run_orca.sh<br />
|lang="bash"<br />
|contents=<br />
#!/bin/bash<br />
#SBATCH --account=def-youPIs<br />
#SBATCH --ntasks=8 # cpus, the nprocs defined in the input file<br />
#SBATCH --mem-per-cpu=3G # memory per cpu<br />
#SBATCH --time=00-03:00 # time (DD-HH:MM)<br />
#SBATCH --output=benzene.log # output .log file<br />
<br />
module load StdEnv/2020 gcc/9.3.0 openmpi/4.0.3<br />
module load orca/4.2.1<br />
$EBROOTORCA/orca benzene.inp<br />
}}<br />
<translate><br />
<!--T:10--><br />
Example of the input file, benzene.inp:<br />
</translate><br />
{{File<br />
|name=benzene.inp<br />
|lang="text"<br />
|contents=<br />
# Benzene RHF Opt Calculation<br />
%pal nprocs 8 end<br />
! RHF TightSCF PModel<br />
! opt<br />
<br />
* xyz 0 1<br />
C 0.000000000000 1.398696930758 0.000000000000<br />
C 0.000000000000 -1.398696930758 0.000000000000<br />
C 1.211265339156 0.699329968382 0.000000000000<br />
C 1.211265339156 -0.699329968382 0.000000000000<br />
C -1.211265339156 0.699329968382 0.000000000000<br />
C -1.211265339156 -0.699329968382 0.000000000000<br />
H 0.000000000000 2.491406946734 0.000000000000<br />
H 0.000000000000 -2.491406946734 0.000000000000<br />
H 2.157597486829 1.245660462400 0.000000000000<br />
H 2.157597486829 -1.245660462400 0.000000000000<br />
H -2.157597486829 1.245660462400 0.000000000000<br />
H -2.157597486829 -1.245660462400 0.000000000000<br />
*<br />
}}<br />
<translate><br />
<br />
===Notes=== <!--T:15--><br />
* To make sure that the program runs efficiently and makes use of all the resources or the cores asked for in your job script, please add this line <code>%pal nprocs <ncores> end</code> to your input file as shown in the above example. Replace <code><ncores></code> by the number of cores you used in your script.<br />
<br />
===(Sep. 6 2019) Temporary fix to OpenMPI version inconsistency issue=== <!--T:16--><br />
For some type of calculations (DLPNO-STEOM-CCSD in particular), one could receive unknown openmpi related fatal errors. This could be due to using an older version of openmpi (''i.e.'' 3.1.2 as suggested by 'module' for both orca/4.1.0 and 4.2.0) than recommended officially (3.1.3 for orca/4.1.0 and 3.1.4 for orca/4.2.0). To temporarily fix this issue, one can build a custom version of openmpi.<br />
<br />
<!--T:17--><br />
The following two commands prepares a custom openmpi/3.1.4 for orca/4.2.0:<br />
module load gcc/7.3.0<br />
eb OpenMPI-3.1.2-GCC-7.3.0.eb --try-software-version=3.1.4<br />
When the building is finished, one can load the custom openmpi using module:<br />
module load openmpi/3.1.4<br />
At this step, one can manually install orca/4.2.0 binaries from the official forum under the home directory after finishing the registration on the official orca forum and being granted access to the orca program on Compute Canada clusters.<br />
<br />
<br />
<!--T:18--><br />
Additional notes from the contributor:<br />
<br />
<!--T:19--><br />
This is a '''temporary''' fix prior to the official upgrade of openmpi on Compute Canada clusters. Please remember to delete the manually installed orca binaries once the official openmpi version is up to date.<br />
<br />
<!--T:20--><br />
The compiling command does not seem to apply to openmpi/2.1.x.<br />
<br />
</translate></div>Kerrachehttps://docs.alliancecan.ca/mediawiki/index.php?title=ORCA&diff=110907ORCA2022-02-01T19:07:00Z<p>Kerrache: </p>
<hr />
<div><languages /><br />
[[Category:Software]][[Category:ComputationalChemistry]]<br />
<br />
<translate><br />
<br />
==Introduction== <!--T:1--><br />
ORCA is a flexible, efficient and easy-to-use general purpose tool for quantum chemistry with specific emphasis on spectroscopic properties of open-shell molecules. It features a wide variety of standard quantum chemical methods ranging from semiempirical methods to DFT to single- and multireference correlated ab initio methods. It can also treat environmental and relativistic effects.<br />
<br />
== Licensing == <!--T:2--><br />
If you wish to use pre-built ORCA executables:<br />
# You have to register at https://orcaforum.kofo.mpg.de/<br />
# You will receive a first email to verify the email address and activate the account. Follow the instructions in that email.<br />
# Once the registration is complete you will get a '''second email''' stating that the "''registration for ORCA download and usage has been completed''".<br />
# [[Technical support | Contact us]] requesting access to ORCA with a copy of the '''second email''' mentioned above.<br />
<br />
== ORCA versions ==<br />
<br />
On July 2021, a version 5 of ORCA was released. This is a major upgrade of ORCA 4.<br />
<br />
== ORCA 5 ==<br />
<br />
The first released versions 5.0 and 5.0.1 have few bugs that were fixed in the later version 5.0.2. Even if the version 5.0.1 is installed on Compute Canada clusters, users should use the latest one 5.0.2.<br />
<br />
To load the version 5.0.2, use:<br />
<br />
module load StdEnv/2020 gcc/10.3.0 openmpi/4.1.1 orca/5.0.2<br />
<br />
== ORCA 4 ==<br />
<br />
== ORCA performance ==<br />
<br />
==Using the software== <!--T:3--><br />
To see what versions of ORCA are currently available, type <code>module spider orca</code>. For detailed information about a specific version, including what other modules must be loaded first, use the module's full name. For example, <code>module spider orca/4.0.1.2</code>.<br />
<br />
<!--T:11--><br />
See [[Using modules]] for general guidance.<br />
<br />
===Job submission=== <!--T:4--><br />
For a general discussion about submitting jobs, see [[Running jobs]].<br />
<br />
<!--T:12--><br />
'''NOTE''': If you run into MPI errors with some of the ORCA executables, you can try to define the following variables:<br />
<br />
<!--T:13--><br />
export OMPI_MCA_mtl='^mxm'<br />
export OMPI_MCA_pml='^yalla'<br />
<br />
<!--T:5--><br />
The following is a job script to run ORCA using [[MPI]]:<br />
<br />
</translate><br />
{{File<br />
|name=run_orca.sh<br />
|lang="bash"<br />
|contents=<br />
#!/bin/bash<br />
#SBATCH --account=def-youPIs<br />
#SBATCH --ntasks=8 # cpus, the nprocs defined in the input file<br />
#SBATCH --mem-per-cpu=3G # memory per cpu<br />
#SBATCH --time=00-03:00 # time (DD-HH:MM)<br />
#SBATCH --output=benzene.log # output .log file<br />
<br />
module load StdEnv/2020 gcc/9.3.0 openmpi/4.0.3<br />
module load orca/4.2.1<br />
$EBROOTORCA/orca benzene.inp<br />
}}<br />
<translate><br />
<!--T:10--><br />
Example of the input file, benzene.inp:<br />
</translate><br />
{{File<br />
|name=benzene.inp<br />
|lang="text"<br />
|contents=<br />
# Benzene RHF Opt Calculation<br />
%pal nprocs 8 end<br />
! RHF TightSCF PModel<br />
! opt<br />
<br />
* xyz 0 1<br />
C 0.000000000000 1.398696930758 0.000000000000<br />
C 0.000000000000 -1.398696930758 0.000000000000<br />
C 1.211265339156 0.699329968382 0.000000000000<br />
C 1.211265339156 -0.699329968382 0.000000000000<br />
C -1.211265339156 0.699329968382 0.000000000000<br />
C -1.211265339156 -0.699329968382 0.000000000000<br />
H 0.000000000000 2.491406946734 0.000000000000<br />
H 0.000000000000 -2.491406946734 0.000000000000<br />
H 2.157597486829 1.245660462400 0.000000000000<br />
H 2.157597486829 -1.245660462400 0.000000000000<br />
H -2.157597486829 1.245660462400 0.000000000000<br />
H -2.157597486829 -1.245660462400 0.000000000000<br />
*<br />
}}<br />
<translate><br />
<br />
===Notes=== <!--T:15--><br />
* To make sure that the program runs efficiently and makes use of all the resources or the cores asked for in your job script, please add this line <code>%pal nprocs <ncores> end</code> to your input file as shown in the above example. Replace <code><ncores></code> by the number of cores you used in your script.<br />
<br />
===(Sep. 6 2019) Temporary fix to OpenMPI version inconsistency issue=== <!--T:16--><br />
For some type of calculations (DLPNO-STEOM-CCSD in particular), one could receive unknown openmpi related fatal errors. This could be due to using an older version of openmpi (''i.e.'' 3.1.2 as suggested by 'module' for both orca/4.1.0 and 4.2.0) than recommended officially (3.1.3 for orca/4.1.0 and 3.1.4 for orca/4.2.0). To temporarily fix this issue, one can build a custom version of openmpi.<br />
<br />
<!--T:17--><br />
The following two commands prepares a custom openmpi/3.1.4 for orca/4.2.0:<br />
module load gcc/7.3.0<br />
eb OpenMPI-3.1.2-GCC-7.3.0.eb --try-software-version=3.1.4<br />
When the building is finished, one can load the custom openmpi using module:<br />
module load openmpi/3.1.4<br />
At this step, one can manually install orca/4.2.0 binaries from the official forum under the home directory after finishing the registration on the official orca forum and being granted access to the orca program on Compute Canada clusters.<br />
<br />
<br />
<!--T:18--><br />
Additional notes from the contributor:<br />
<br />
<!--T:19--><br />
This is a '''temporary''' fix prior to the official upgrade of openmpi on Compute Canada clusters. Please remember to delete the manually installed orca binaries once the official openmpi version is up to date.<br />
<br />
<!--T:20--><br />
The compiling command does not seem to apply to openmpi/2.1.x.<br />
<br />
</translate></div>Kerrachehttps://docs.alliancecan.ca/mediawiki/index.php?title=ORCA&diff=110906ORCA2022-02-01T19:06:24Z<p>Kerrache: /* ORCA 5 */</p>
<hr />
<div><languages /><br />
[[Category:Software]][[Category:ComputationalChemistry]]<br />
<br />
<translate><br />
<br />
==Introduction== <!--T:1--><br />
ORCA is a flexible, efficient and easy-to-use general purpose tool for quantum chemistry with specific emphasis on spectroscopic properties of open-shell molecules. It features a wide variety of standard quantum chemical methods ranging from semiempirical methods to DFT to single- and multireference correlated ab initio methods. It can also treat environmental and relativistic effects.<br />
<br />
== Licensing == <!--T:2--><br />
If you wish to use pre-built ORCA executables:<br />
# You have to register at https://orcaforum.kofo.mpg.de/<br />
# You will receive a first email to verify the email address and activate the account. Follow the instructions in that email.<br />
# Once the registration is complete you will get a '''second email''' stating that the "''registration for ORCA download and usage has been completed''".<br />
# [[Technical support | Contact us]] requesting access to ORCA with a copy of the '''second email''' mentioned above.<br />
<br />
== ORCA versions ==<br />
<br />
On July 2021, a version 5 of ORCA was released. This is a major upgrade of ORCA 4.<br />
<br />
== ORCA 5 ==<br />
<br />
The first released versions 5.0 and 5.0.1 have few bugs that were fixed in the later version 5.0.2. Even if the version 5.0.1 is installed on Compute Canada clusters, users should use the latest one 5.0.2.<br />
<br />
To load the version 5.0.2, use:<br />
<br />
module load StdEnv/2020 gcc/10.3.0 openmpi/4.1.1 orca/5.0.2<br />
<br />
== ORCA 4 ==<br />
<br />
== ORCA performance ==<br />
<br />
==Using the software== <!--T:3--><br />
To see what versions of ORCA are currently available, type <code>module spider orca</code>. For detailed information about a specific version, including what other modules must be loaded first, use the module's full name. For example, <code>module spider orca/4.0.1.2</code>.<br />
<br />
<!--T:11--><br />
See [[Using modules]] for general guidance.<br />
<br />
===Job submission=== <!--T:4--><br />
For a general discussion about submitting jobs, see [[Running jobs]].<br />
<br />
<!--T:12--><br />
'''NOTE''': If you run into MPI errors with some of the ORCA executables, you can try to define the following variables:<br />
<br />
<!--T:13--><br />
export OMPI_MCA_mtl='^mxm'<br />
export OMPI_MCA_pml='^yalla'<br />
<br />
<!--T:5--><br />
The following is a job script to run ORCA using [[MPI]]:<br />
<br />
</translate><br />
{{File<br />
|name=run_orca.sh<br />
|lang="bash"<br />
|contents=<br />
#!/bin/bash<br />
#SBATCH --account=def-youPIs<br />
#SBATCH --ntasks=8 # cpus, the nprocs defined in the input file<br />
#SBATCH --mem-per-cpu=3G # memory per cpu<br />
#SBATCH --time=00-03:00 # time (DD-HH:MM)<br />
#SBATCH --output=benzene.log # output .log file<br />
<br />
module load StdEnv/2020 gcc/9.3.0 openmpi/4.0.3<br />
module load orca/4.2.1<br />
$EBROOTORCA/orca benzene.inp<br />
}}<br />
<translate><br />
<!--T:10--><br />
Example of the input file, benzene.inp:<br />
</translate><br />
{{File<br />
|name=benzene.inp<br />
|lang="text"<br />
|contents=<br />
# Benzene RHF Opt Calculation<br />
%pal nprocs 8 end<br />
! RHF TightSCF PModel<br />
! opt<br />
<br />
* xyz 0 1<br />
C 0.000000000000 1.398696930758 0.000000000000<br />
C 0.000000000000 -1.398696930758 0.000000000000<br />
C 1.211265339156 0.699329968382 0.000000000000<br />
C 1.211265339156 -0.699329968382 0.000000000000<br />
C -1.211265339156 0.699329968382 0.000000000000<br />
C -1.211265339156 -0.699329968382 0.000000000000<br />
H 0.000000000000 2.491406946734 0.000000000000<br />
H 0.000000000000 -2.491406946734 0.000000000000<br />
H 2.157597486829 1.245660462400 0.000000000000<br />
H 2.157597486829 -1.245660462400 0.000000000000<br />
H -2.157597486829 1.245660462400 0.000000000000<br />
H -2.157597486829 -1.245660462400 0.000000000000<br />
*<br />
}}<br />
<translate><br />
===Notes=== <!--T:15--><br />
* To make sure that the program runs efficiently and makes use of all the resources or the cores asked for in your job script, please add this line <code>%pal nprocs <ncores> end</code> to your input file as shown in the above example. Replace <code><ncores></code> by the number of cores you used in your script.<br />
<br />
===(Sep. 6 2019) Temporary fix to OpenMPI version inconsistency issue=== <!--T:16--><br />
For some type of calculations (DLPNO-STEOM-CCSD in particular), one could receive unknown openmpi related fatal errors. This could be due to using an older version of openmpi (''i.e.'' 3.1.2 as suggested by 'module' for both orca/4.1.0 and 4.2.0) than recommended officially (3.1.3 for orca/4.1.0 and 3.1.4 for orca/4.2.0). To temporarily fix this issue, one can build a custom version of openmpi.<br />
<br />
<!--T:17--><br />
The following two commands prepares a custom openmpi/3.1.4 for orca/4.2.0:<br />
module load gcc/7.3.0<br />
eb OpenMPI-3.1.2-GCC-7.3.0.eb --try-software-version=3.1.4<br />
When the building is finished, one can load the custom openmpi using module:<br />
module load openmpi/3.1.4<br />
At this step, one can manually install orca/4.2.0 binaries from the official forum under the home directory after finishing the registration on the official orca forum and being granted access to the orca program on Compute Canada clusters.<br />
<br />
<br />
<!--T:18--><br />
Additional notes from the contributor:<br />
<br />
<!--T:19--><br />
This is a '''temporary''' fix prior to the official upgrade of openmpi on Compute Canada clusters. Please remember to delete the manually installed orca binaries once the official openmpi version is up to date.<br />
<br />
<!--T:20--><br />
The compiling command does not seem to apply to openmpi/2.1.x.<br />
<br />
</translate></div>Kerrachehttps://docs.alliancecan.ca/mediawiki/index.php?title=ORCA&diff=110900ORCA2022-02-01T18:48:33Z<p>Kerrache: /* Modules */</p>
<hr />
<div><languages /><br />
[[Category:Software]][[Category:ComputationalChemistry]]<br />
<br />
<translate><br />
<br />
==Introduction== <!--T:1--><br />
ORCA is a flexible, efficient and easy-to-use general purpose tool for quantum chemistry with specific emphasis on spectroscopic properties of open-shell molecules. It features a wide variety of standard quantum chemical methods ranging from semiempirical methods to DFT to single- and multireference correlated ab initio methods. It can also treat environmental and relativistic effects.<br />
<br />
== Licensing == <!--T:2--><br />
If you wish to use pre-built ORCA executables:<br />
# You have to register at https://orcaforum.kofo.mpg.de/<br />
# You will receive a first email to verify the email address and activate the account. Follow the instructions in that email.<br />
# Once the registration is complete you will get a '''second email''' stating that the "''registration for ORCA download and usage has been completed''".<br />
# [[Technical support | Contact us]] requesting access to ORCA with a copy of the '''second email''' mentioned above.<br />
<br />
== ORCA versions ==<br />
<br />
On July 2021, a version 5 of ORCA was released. This is a major upgrade of ORCA 4.<br />
<br />
== ORCA 5 ==<br />
<br />
== ORCA 4 ==<br />
<br />
== ORCA performance ==<br />
<br />
==Using the software== <!--T:3--><br />
To see what versions of ORCA are currently available, type <code>module spider orca</code>. For detailed information about a specific version, including what other modules must be loaded first, use the module's full name. For example, <code>module spider orca/4.0.1.2</code>.<br />
<br />
<!--T:11--><br />
See [[Using modules]] for general guidance.<br />
<br />
===Job submission=== <!--T:4--><br />
For a general discussion about submitting jobs, see [[Running jobs]].<br />
<br />
<!--T:12--><br />
'''NOTE''': If you run into MPI errors with some of the ORCA executables, you can try to define the following variables:<br />
<br />
<!--T:13--><br />
export OMPI_MCA_mtl='^mxm'<br />
export OMPI_MCA_pml='^yalla'<br />
<br />
<!--T:5--><br />
The following is a job script to run ORCA using [[MPI]]:<br />
<br />
</translate><br />
{{File<br />
|name=run_orca.sh<br />
|lang="bash"<br />
|contents=<br />
#!/bin/bash<br />
#SBATCH --account=def-youPIs<br />
#SBATCH --ntasks=8 # cpus, the nprocs defined in the input file<br />
#SBATCH --mem-per-cpu=3G # memory per cpu<br />
#SBATCH --time=00-03:00 # time (DD-HH:MM)<br />
#SBATCH --output=benzene.log # output .log file<br />
<br />
module load StdEnv/2020 gcc/9.3.0 openmpi/4.0.3<br />
module load orca/4.2.1<br />
$EBROOTORCA/orca benzene.inp<br />
}}<br />
<translate><br />
<!--T:10--><br />
Example of the input file, benzene.inp:<br />
</translate><br />
{{File<br />
|name=benzene.inp<br />
|lang="text"<br />
|contents=<br />
# Benzene RHF Opt Calculation<br />
%pal nprocs 8 end<br />
! RHF TightSCF PModel<br />
! opt<br />
<br />
* xyz 0 1<br />
C 0.000000000000 1.398696930758 0.000000000000<br />
C 0.000000000000 -1.398696930758 0.000000000000<br />
C 1.211265339156 0.699329968382 0.000000000000<br />
C 1.211265339156 -0.699329968382 0.000000000000<br />
C -1.211265339156 0.699329968382 0.000000000000<br />
C -1.211265339156 -0.699329968382 0.000000000000<br />
H 0.000000000000 2.491406946734 0.000000000000<br />
H 0.000000000000 -2.491406946734 0.000000000000<br />
H 2.157597486829 1.245660462400 0.000000000000<br />
H 2.157597486829 -1.245660462400 0.000000000000<br />
H -2.157597486829 1.245660462400 0.000000000000<br />
H -2.157597486829 -1.245660462400 0.000000000000<br />
*<br />
}}<br />
<translate><br />
===Notes=== <!--T:15--><br />
* To make sure that the program runs efficiently and makes use of all the resources or the cores asked for in your job script, please add this line <code>%pal nprocs <ncores> end</code> to your input file as shown in the above example. Replace <code><ncores></code> by the number of cores you used in your script.<br />
<br />
===(Sep. 6 2019) Temporary fix to OpenMPI version inconsistency issue=== <!--T:16--><br />
For some type of calculations (DLPNO-STEOM-CCSD in particular), one could receive unknown openmpi related fatal errors. This could be due to using an older version of openmpi (''i.e.'' 3.1.2 as suggested by 'module' for both orca/4.1.0 and 4.2.0) than recommended officially (3.1.3 for orca/4.1.0 and 3.1.4 for orca/4.2.0). To temporarily fix this issue, one can build a custom version of openmpi.<br />
<br />
<!--T:17--><br />
The following two commands prepares a custom openmpi/3.1.4 for orca/4.2.0:<br />
module load gcc/7.3.0<br />
eb OpenMPI-3.1.2-GCC-7.3.0.eb --try-software-version=3.1.4<br />
When the building is finished, one can load the custom openmpi using module:<br />
module load openmpi/3.1.4<br />
At this step, one can manually install orca/4.2.0 binaries from the official forum under the home directory after finishing the registration on the official orca forum and being granted access to the orca program on Compute Canada clusters.<br />
<br />
<br />
<!--T:18--><br />
Additional notes from the contributor:<br />
<br />
<!--T:19--><br />
This is a '''temporary''' fix prior to the official upgrade of openmpi on Compute Canada clusters. Please remember to delete the manually installed orca binaries once the official openmpi version is up to date.<br />
<br />
<!--T:20--><br />
The compiling command does not seem to apply to openmpi/2.1.x.<br />
<br />
</translate></div>Kerrachehttps://docs.alliancecan.ca/mediawiki/index.php?title=ORCA&diff=110746ORCA2022-01-27T15:16:37Z<p>Kerrache: </p>
<hr />
<div><languages /><br />
[[Category:Software]][[Category:ComputationalChemistry]]<br />
<br />
<translate><br />
<br />
==Introduction== <!--T:1--><br />
ORCA is a flexible, efficient and easy-to-use general purpose tool for quantum chemistry with specific emphasis on spectroscopic properties of open-shell molecules. It features a wide variety of standard quantum chemical methods ranging from semiempirical methods to DFT to single- and multireference correlated ab initio methods. It can also treat environmental and relativistic effects.<br />
<br />
== Licensing == <!--T:2--><br />
If you wish to use pre-built ORCA executables:<br />
# You have to register at https://orcaforum.kofo.mpg.de/<br />
# You will receive a first email to verify the email address and activate the account. Follow the instructions in that email.<br />
# Once the registration is complete you will get a '''second email''' stating that the "''registration for ORCA download and usage has been completed''".<br />
# [[Technical support | Contact us]] requesting access to ORCA with a copy of the '''second email''' mentioned above.<br />
<br />
== Modules ==<br />
<br />
== ORCA 5 ==<br />
<br />
== ORCA 4 ==<br />
<br />
== ORCA performance ==<br />
<br />
==Using the software== <!--T:3--><br />
To see what versions of ORCA are currently available, type <code>module spider orca</code>. For detailed information about a specific version, including what other modules must be loaded first, use the module's full name. For example, <code>module spider orca/4.0.1.2</code>.<br />
<br />
<!--T:11--><br />
See [[Using modules]] for general guidance.<br />
<br />
===Job submission=== <!--T:4--><br />
For a general discussion about submitting jobs, see [[Running jobs]].<br />
<br />
<!--T:12--><br />
'''NOTE''': If you run into MPI errors with some of the ORCA executables, you can try to define the following variables:<br />
<br />
<!--T:13--><br />
export OMPI_MCA_mtl='^mxm'<br />
export OMPI_MCA_pml='^yalla'<br />
<br />
<!--T:5--><br />
The following is a job script to run ORCA using [[MPI]]:<br />
<br />
</translate><br />
{{File<br />
|name=run_orca.sh<br />
|lang="bash"<br />
|contents=<br />
#!/bin/bash<br />
#SBATCH --account=def-youPIs<br />
#SBATCH --ntasks=8 # cpus, the nprocs defined in the input file<br />
#SBATCH --mem-per-cpu=3G # memory per cpu<br />
#SBATCH --time=00-03:00 # time (DD-HH:MM)<br />
#SBATCH --output=benzene.log # output .log file<br />
<br />
module load StdEnv/2020 gcc/9.3.0 openmpi/4.0.3<br />
module load orca/4.2.1<br />
$EBROOTORCA/orca benzene.inp<br />
}}<br />
<translate><br />
<!--T:10--><br />
Example of the input file, benzene.inp:<br />
</translate><br />
{{File<br />
|name=benzene.inp<br />
|lang="text"<br />
|contents=<br />
# Benzene RHF Opt Calculation<br />
%pal nprocs 8 end<br />
! RHF TightSCF PModel<br />
! opt<br />
<br />
* xyz 0 1<br />
C 0.000000000000 1.398696930758 0.000000000000<br />
C 0.000000000000 -1.398696930758 0.000000000000<br />
C 1.211265339156 0.699329968382 0.000000000000<br />
C 1.211265339156 -0.699329968382 0.000000000000<br />
C -1.211265339156 0.699329968382 0.000000000000<br />
C -1.211265339156 -0.699329968382 0.000000000000<br />
H 0.000000000000 2.491406946734 0.000000000000<br />
H 0.000000000000 -2.491406946734 0.000000000000<br />
H 2.157597486829 1.245660462400 0.000000000000<br />
H 2.157597486829 -1.245660462400 0.000000000000<br />
H -2.157597486829 1.245660462400 0.000000000000<br />
H -2.157597486829 -1.245660462400 0.000000000000<br />
*<br />
}}<br />
<translate><br />
===Notes=== <!--T:15--><br />
* To make sure that the program runs efficiently and makes use of all the resources or the cores asked for in your job script, please add this line <code>%pal nprocs <ncores> end</code> to your input file as shown in the above example. Replace <code><ncores></code> by the number of cores you used in your script.<br />
<br />
===(Sep. 6 2019) Temporary fix to OpenMPI version inconsistency issue=== <!--T:16--><br />
For some type of calculations (DLPNO-STEOM-CCSD in particular), one could receive unknown openmpi related fatal errors. This could be due to using an older version of openmpi (''i.e.'' 3.1.2 as suggested by 'module' for both orca/4.1.0 and 4.2.0) than recommended officially (3.1.3 for orca/4.1.0 and 3.1.4 for orca/4.2.0). To temporarily fix this issue, one can build a custom version of openmpi.<br />
<br />
<!--T:17--><br />
The following two commands prepares a custom openmpi/3.1.4 for orca/4.2.0:<br />
module load gcc/7.3.0<br />
eb OpenMPI-3.1.2-GCC-7.3.0.eb --try-software-version=3.1.4<br />
When the building is finished, one can load the custom openmpi using module:<br />
module load openmpi/3.1.4<br />
At this step, one can manually install orca/4.2.0 binaries from the official forum under the home directory after finishing the registration on the official orca forum and being granted access to the orca program on Compute Canada clusters.<br />
<br />
<br />
<!--T:18--><br />
Additional notes from the contributor:<br />
<br />
<!--T:19--><br />
This is a '''temporary''' fix prior to the official upgrade of openmpi on Compute Canada clusters. Please remember to delete the manually installed orca binaries once the official openmpi version is up to date.<br />
<br />
<!--T:20--><br />
The compiling command does not seem to apply to openmpi/2.1.x.<br />
<br />
</translate></div>Kerrachehttps://docs.alliancecan.ca/mediawiki/index.php?title=ORCA&diff=110745ORCA2022-01-27T15:13:29Z<p>Kerrache: </p>
<hr />
<div><languages /><br />
[[Category:Software]][[Category:ComputationalChemistry]]<br />
<br />
<translate><br />
<br />
==Introduction== <!--T:1--><br />
ORCA is a flexible, efficient and easy-to-use general purpose tool for quantum chemistry with specific emphasis on spectroscopic properties of open-shell molecules. It features a wide variety of standard quantum chemical methods ranging from semiempirical methods to DFT to single- and multireference correlated ab initio methods. It can also treat environmental and relativistic effects.<br />
<br />
== Licensing == <!--T:2--><br />
If you wish to use pre-built ORCA executables:<br />
# You have to register at https://orcaforum.kofo.mpg.de/<br />
# You will receive a first email to verify the email address and activate the account. Follow the instructions in that email.<br />
# Once the registration is complete you will get a '''second email''' stating that the "''registration for ORCA download and usage has been completed''".<br />
# [[Technical support | Contact us]] requesting access to ORCA with a copy of the '''second email''' mentioned above.<br />
<br />
== ORCA 5 ==<br />
<br />
== ORCA 4 ==<br />
<br />
== ORCA performance ==<br />
<br />
==Using the software== <!--T:3--><br />
To see what versions of ORCA are currently available, type <code>module spider orca</code>. For detailed information about a specific version, including what other modules must be loaded first, use the module's full name. For example, <code>module spider orca/4.0.1.2</code>.<br />
<br />
<!--T:11--><br />
See [[Using modules]] for general guidance.<br />
<br />
===Job submission=== <!--T:4--><br />
For a general discussion about submitting jobs, see [[Running jobs]].<br />
<br />
<!--T:12--><br />
'''NOTE''': If you run into MPI errors with some of the ORCA executables, you can try to define the following variables:<br />
<br />
<!--T:13--><br />
export OMPI_MCA_mtl='^mxm'<br />
export OMPI_MCA_pml='^yalla'<br />
<br />
<!--T:5--><br />
The following is a job script to run ORCA using [[MPI]]:<br />
<br />
</translate><br />
{{File<br />
|name=run_orca.sh<br />
|lang="bash"<br />
|contents=<br />
#!/bin/bash<br />
#SBATCH --account=def-youPIs<br />
#SBATCH --ntasks=8 # cpus, the nprocs defined in the input file<br />
#SBATCH --mem-per-cpu=3G # memory per cpu<br />
#SBATCH --time=00-03:00 # time (DD-HH:MM)<br />
#SBATCH --output=benzene.log # output .log file<br />
<br />
module load StdEnv/2020 gcc/9.3.0 openmpi/4.0.3<br />
module load orca/4.2.1<br />
$EBROOTORCA/orca benzene.inp<br />
}}<br />
<translate><br />
<!--T:10--><br />
Example of the input file, benzene.inp:<br />
</translate><br />
{{File<br />
|name=benzene.inp<br />
|lang="text"<br />
|contents=<br />
# Benzene RHF Opt Calculation<br />
%pal nprocs 8 end<br />
! RHF TightSCF PModel<br />
! opt<br />
<br />
* xyz 0 1<br />
C 0.000000000000 1.398696930758 0.000000000000<br />
C 0.000000000000 -1.398696930758 0.000000000000<br />
C 1.211265339156 0.699329968382 0.000000000000<br />
C 1.211265339156 -0.699329968382 0.000000000000<br />
C -1.211265339156 0.699329968382 0.000000000000<br />
C -1.211265339156 -0.699329968382 0.000000000000<br />
H 0.000000000000 2.491406946734 0.000000000000<br />
H 0.000000000000 -2.491406946734 0.000000000000<br />
H 2.157597486829 1.245660462400 0.000000000000<br />
H 2.157597486829 -1.245660462400 0.000000000000<br />
H -2.157597486829 1.245660462400 0.000000000000<br />
H -2.157597486829 -1.245660462400 0.000000000000<br />
*<br />
}}<br />
<translate><br />
===Notes=== <!--T:15--><br />
* To make sure that the program runs efficiently and makes use of all the resources or the cores asked for in your job script, please add this line <code>%pal nprocs <ncores> end</code> to your input file as shown in the above example. Replace <code><ncores></code> by the number of cores you used in your script.<br />
<br />
===(Sep. 6 2019) Temporary fix to OpenMPI version inconsistency issue=== <!--T:16--><br />
For some type of calculations (DLPNO-STEOM-CCSD in particular), one could receive unknown openmpi related fatal errors. This could be due to using an older version of openmpi (''i.e.'' 3.1.2 as suggested by 'module' for both orca/4.1.0 and 4.2.0) than recommended officially (3.1.3 for orca/4.1.0 and 3.1.4 for orca/4.2.0). To temporarily fix this issue, one can build a custom version of openmpi.<br />
<br />
<!--T:17--><br />
The following two commands prepares a custom openmpi/3.1.4 for orca/4.2.0:<br />
module load gcc/7.3.0<br />
eb OpenMPI-3.1.2-GCC-7.3.0.eb --try-software-version=3.1.4<br />
When the building is finished, one can load the custom openmpi using module:<br />
module load openmpi/3.1.4<br />
At this step, one can manually install orca/4.2.0 binaries from the official forum under the home directory after finishing the registration on the official orca forum and being granted access to the orca program on Compute Canada clusters.<br />
<br />
<br />
<!--T:18--><br />
Additional notes from the contributor:<br />
<br />
<!--T:19--><br />
This is a '''temporary''' fix prior to the official upgrade of openmpi on Compute Canada clusters. Please remember to delete the manually installed orca binaries once the official openmpi version is up to date.<br />
<br />
<!--T:20--><br />
The compiling command does not seem to apply to openmpi/2.1.x.<br />
<br />
</translate></div>Kerrachehttps://docs.alliancecan.ca/mediawiki/index.php?title=QIIME&diff=101716QIIME2021-07-07T13:53:12Z<p>Kerrache: Removing the easybuild part since conda is not supported anymore</p>
<hr />
<div><languages /><br />
<translate><br />
<!--T:1--><br />
'''QIIME''' (pronounced ''chime'') stands for ''Quantitative Insights Into [https://en.wikipedia.org/wiki/Microbial_ecology Microbial Ecology]'', is an open-source [https://en.wikipedia.org/wiki/Bioinformatics bioinformatics] pipeline for performing [https://en.wikipedia.org/wiki/Microbiota microbiome] analysis from raw DNA sequencing data. QIIME is designed to take users from raw sequencing data generated on [https://www.illumina.com/ Illumina] or other platforms to publication-quality graphics and statistics. This includes demultiplexing and quality filtering, [https://en.wikipedia.org/wiki/Operational_taxonomic_unit OTU] picking, taxonomic assignment, phylogenetic reconstruction, diversity analyses and visualizations. QIIME has been applied to studies based on billions of sequences from tens of thousands of samples.<br />
<br />
<!--T:2--><br />
'''Note''': QIIME 2 has replaced QIIME 1 as of January 1, 2018; version 1 is no longer supported.<br />
<br />
<!--T:20--><br />
'''Note''': As of February, 2020, due to various issues generated by Conda environments in our HPC systems, '''installation using Anaconda or Miniconda is no longer supported'''.<br />
<br />
==Installation == <!--T:4--><br />
QIIME2 can be installed using [[Singularity]] or using EasyBuild. Singularity is strongly preferred since it does not generate many thousands of files in your home directory, potentially causing you to exceed the disk quota limit on the number of files.<br />
<br />
=== Using Singularity === <!--T:14--><br />
<br />
<!--T:15--><br />
The QIIME2 developers publish images on [https://hub.docker.com/u/qiime2 Docker Hub]. In order to use one of these images on our systems you must first [[Singularity/en#Creating_an_image_using_Docker_Hub|build a Singularity image]]:<br />
<br />
<!--T:16--><br />
{{Commands<br />
|module load singularity<br />
|singularity build qiime2-2019.10.sif docker://qiime2/core:2019.10<br />
}}<br />
<br />
<!--T:21--><br />
This build step may take over an hour, but you only need do this once. Save the image file (<code>qiime2-2019.10.sif</code> in this example)<br />
for later re-use. <br />
<br />
<!--T:17--><br />
Then run your code as described at [[Singularity]]. Typically you will run each QIIME command in a <code>singularity exec</code> statement:<br />
<br />
<!--T:22--><br />
{{Commands<br />
|singularity exec qiime2-2019.10.sif <your QIIME command><br />
}}<br />
<br />
<!--T:23--><br />
So your [[Running jobs|SBATCH]] script might look something like this:<br />
<br />
<!--T:24--><br />
<pre><br />
#!/bin/bash<br />
#SBATCH --time=15:00:00<br />
#SBATCH --account=def-someuser<br />
<br />
<!--T:25--><br />
singularity exec -B $PWD:/home -B /scratch/someuser:/outputs \<br />
-B /projects/someuser/path/to/inputs:/inputs qiime2-2019.10.sif \<br />
qiime tools import --type 'FeatureData[Sequence]' \<br />
--input-path /inputs/some_fastafile.fa \<br />
--output-path /outputs/some_output_feature.qza<br />
<br />
<!--T:26--><br />
singularity exec -B $PWD:/home -B /scratch/someuser:/outputs \<br />
-B /projects/someuser/path/to/inputs:/inputs qiime2-2019.10.sif \<br />
qiime tools import \<br />
--type 'FeatureData[Taxonomy]' \<br />
--input-format HeaderlessTSVTaxonomyFormat \<br />
--input-path /inputs/some_taxonomy_file.tax \<br />
--output-path /outputs/some_output_ref-taxonomy.qza<br />
<br />
<!--T:27--><br />
singularity exec -B $PWD:/home -B /scratch/someuser:/outputs \<br />
-B /projects/someuser/path/to/inputs:/inputs qiime2-2019.10.sif \<br />
qiime feature-classifier fit-classifier-naive-bayes \<br />
--i-reference-reads /outputs/some_output_feature.qza \<br />
--i-reference-taxonomy /outputs/some_output_ref-taxonomy.qza \<br />
--o-classifier /outputs/some_output_classifier.qza<br />
</pre><br />
<br />
<!--T:28--><br />
Note that it is important to use the [[Singularity#Bind_mounts|bind]] option (<tt>-B</tt>) with each folder you want to work with when you run programs in your container. For more information about Singularity, you can watch the recorded [https://www.youtube.com/watch?v=kYb0aXS5DEE Singularity webinar].<br />
<br />
<!--T:18--><br />
On first importing data into QIIME format you may receive an error ending with a message like this:<br />
<pre><br />
Timezone offset does not match system offset: 0 != -18000. Please, check your config files.<br />
</pre><br />
This can be worked around by setting a time zone before invoking Singularity:<br />
<br />
<!--T:19--><br />
{{Commands<br />
|export TZ{{=}}'UTC'<br />
|singularity exec qiime2-2019.10.sif qiime tools import ...<br />
}}<br />
<br />
=References = <!--T:11--><br />
<br />
<!--T:13--><br />
[http://qiime.org/ QIIME homepage]<br><br />
<!--[https://docs.conda.io/projects/conda/en/latest/user-guide/getting-started.html Getting started with Conda]<br>--><br />
</translate><br />
[[Category:Bioinformatics]]<br />
[[Category:User Installed Software]]</div>Kerrachehttps://docs.alliancecan.ca/mediawiki/index.php?title=CPMD&diff=98538CPMD2021-03-31T20:15:27Z<p>Kerrache: /* License limitations */</p>
<hr />
<div><languages /><br />
<translate><br />
<!--T:1--><br />
[[Category:Software]]<br />
<br />
[https://www.cpmd.org/wordpress/ CPMD] is a plane wave/pseudopotential DFT code for ab initio molecular dynamics simulations.<br />
<br />
<!--T:2--><br />
= License limitations =<br />
<br />
Before you can start using [http://cpmd.org CPMD], you have to register and accept the [http://cpmd.org/download/cpmd-download-1/accept-license CPMD license terms] and then [[Technical_support | send us a support request]] stating that you have in fact registered and accepted CPMD license terms. Afterwards, we will get in touch with the CPMD admins to confirm your registration, and then will grant you access to the CPMD software.<br />
<!--T:3--><br />
<br />
= Module =<br />
<br />
You can access CPMD by loading a [[Utiliser des modules/en|module]]. <br />
<br />
<source lang="bash"><br />
module load intel/2020.1.217 openmpi/4.0.3 cpmd/4.3<br />
</source><br />
<br />
= Local installation of CPMD =<br />
<br />
It has been our experience, however, lately that a response from CPMD admins can take weeks or even months. If you are a registered CPMD user, you will have access to the CPMD source files, and therefore can build the software yourself in your home directory using our software build environment called EasyBuild with the exact same recipe that we would use for a central installation.<br />
<br />
<!--T:4--><br />
Below are instructions on how to build CPMD 4.3 under your account on a cluster of your choice:<br />
<br />
<!--T:5--><br />
Create a local directory, first, like so<br />
$ mkdir -p ~/.local/easybuild/sources/c/CPMD<br />
<br />
<!--T:6--><br />
Place all the CPMD source tarballs and patches into that directory<br />
<pre><br />
$ ls -al ~/.local/easybuild/sources/c/CPMD<br />
cpmd2cube.tar.gz<br />
cpmd2xyz-scripts.tar.gz<br />
cpmd-v4.3.tar.gz<br />
fourier.tar.gz<br />
patch.to.4612<br />
patch.to.4615<br />
patch.to.4616<br />
patch.to.4621<br />
patch.to.4624<br />
patch.to.4627<br />
</pre><br />
<br />
<!--T:7--><br />
Then run the EasyBuild command<br />
$ eb CPMD-4.3-iomkl-2020a.eb --rebuild<br />
<br />
<!--T:8--><br />
The <code>--rebuild</code> option forces EasyBuild to ignore CPMD 4.3 installed in a central location and proceed instead with the installation into your home directory.<br />
<br />
<!--T:9--><br />
Once the software is installed, log out and log back in.<br />
<br />
<!--T:10--><br />
Now, when you type <code>module load cpmd</code>, the software installed into your home directory will get picked up.<br />
<br />
<!--T:11--><br />
<pre><br />
$ module load cpmd<br />
$ which cpmd.x<br />
~/.local/easybuild/software/2020/avx2/MPI/intel2020/openmpi4/cpmd/4.3/bin/cpmd.x<br />
</pre><br />
<br />
<!--T:12--><br />
You can use it now as usual in your submission script.<br />
<br />
=Example of job script =<br />
<br />
To run a job, you will need to set an input file and access to [https://www.cpmd.org/wordpress/index.php/documentation/pseudo-potentials/ pseudo-potential] that are available for download from CPMD website after authentication {only for registered users}.<br />
<br />
If the input file and the pseudo-potentials are in the same directory, the command to run the program in parallel is:<br />
<br />
<code>srun cpmd.x <input files> > <output file></code> (as in the script 1)<br />
<br />
It is also possible to put the pseudo potential on another directory and run the code as follow:<br />
<br />
<code>srun cpmd.x <input files> <path to pseudo potentials location> > <output file></code> (as in the script 2)<br />
<br />
<tabs><br />
<tab name="INPUT"><br />
{{File<br />
|name=1-h2-wave.inp<br />
|lang="txt"<br />
|contents=<br />
&INFO<br />
isolated hydrogen molecule.<br />
single point calculation.<br />
&END<br />
<br />
&CPMD<br />
OPTIMIZE WAVEFUNCTION<br />
CONVERGENCE ORBITALS<br />
1.0d-7<br />
CENTER MOLECULE ON<br />
PRINT FORCES ON<br />
&END<br />
<br />
&SYSTEM<br />
SYMMETRY<br />
1<br />
ANGSTROM<br />
CELL<br />
8.00 1.0 1.0 0.0 0.0 0.0<br />
CUTOFF<br />
70.0<br />
&END <br />
<br />
&DFT<br />
FUNCTIONAL LDA<br />
&END <br />
<br />
&ATOMS<br />
*H_MT_LDA.psp<br />
LMAX=S<br />
2<br />
4.371 4.000 4.000<br />
3.629 4.000 4.000<br />
&END <br />
}}<br />
</tab><br />
<br />
<tab name="Script 1"><br />
{{File<br />
|name=run-cpmd.sh<br />
|lang="bash"<br />
|contents=<br />
#!/bin/bash<br />
<br />
#SBATCH --account=def-someacct<br />
#SBATCH --nodes=1<br />
#SBATCH --ntasks-per-node=4<br />
#SBATCH --mem-per-cpu=2500M<br />
#SBATCH --time=0-1:00<br />
<br />
# Load the modules:<br />
<br />
module load intel/2020.1.217 openmpi/4.0.3 cpmd/4.3<br />
<br />
echo "Starting run at: `date`"<br />
<br />
CPMD_INPUT="1-h2-wave.inp"<br />
CPMD_OUTPUT="1-h2-wave_output.txt"<br />
<br />
srun cpmd.x ${CPMD_INPUT} > ${CPMD_OUTPUT}<br />
<br />
echo "Program finished with exit code $? at: `date`"<br />
}}<br />
</tab><br />
<br />
<tab name="Script 2"><br />
{{File<br />
|name=run-cpmd.sh<br />
|lang="bash"<br />
|contents=<br />
#!/bin/bash<br />
<br />
#SBATCH --account=def-someacct<br />
#SBATCH --nodes=1<br />
#SBATCH --ntasks-per-node=4<br />
#SBATCH --mem-per-cpu=2500M<br />
#SBATCH --time=0-1:00<br />
<br />
# Load the modules:<br />
<br />
module load intel/2020.1.217 openmpi/4.0.3 cpmd/4.3<br />
<br />
echo "Starting run at: `date`"<br />
<br />
CPMD_INPUT="1-h2-wave.inp"<br />
CPMD_OUTPUT="1-h2-wave_output.txt"<br />
PP_PATH=<path to the location of pseudo-potentials><br />
<br />
srun cpmd.x ${CPMD_INPUT} ${PP_PATH} > ${CPMD_OUTPUT}<br />
<br />
echo "Program finished with exit code $? at: `date`"<br />
}}<br />
</tab><br />
</tabs><br />
<br />
=Related limks =<br />
<br />
* CPMD [https://www.cpmd.org/wordpress/ home page].<br />
* CPMD [https://www.cpmd.org/wordpress/index.php/documentation/ manual].<br />
* CPMD [https://www.cpmd.org/wordpress/index.php/documentation/pseudo-potentials/ pseudo-potentials].<br />
* CPMD [https://www.cpmd.org/wordpress/index.php/documentation/faqs/ FAQs].<br />
</translate></div>Kerrache