rsnt_translations
53,464
edits
Running jobs: Difference between revisions
Jump to navigation
Jump to search
no edit summary
No edit summary |
No edit summary |
||
| Line 216: | Line 216: | ||
Though batch submission is the most common and most efficient way to take advantage of our clusters, interactive jobs are also supported. These can be useful for things like: | Though batch submission is the most common and most efficient way to take advantage of our clusters, interactive jobs are also supported. These can be useful for things like: | ||
* Data exploration at the command line | * Data exploration at the command line | ||
* Interactive | * Interactive console tools like R and iPython | ||
* Significant software development, debugging, or compiling | * Significant software development, debugging, or compiling | ||
| Line 228: | Line 228: | ||
<!--T:129--> | <!--T:129--> | ||
It is also possible to run graphical programs interactively on a compute node by adding the | It is also possible to run graphical programs interactively on a compute node by adding the <b>--x11</b> flag to your <code<salloc</code> command. In order for this to work, you must first connect to the cluster with X11 forwarding enabled (see the [[SSH]] page for instructions on how to do that). Note that an interactive job with a duration of three hours or less will likely start very soon after submission as we have dedicated test nodes for jobs of this duration. Interactive jobs that request more than three hours run on the cluster's regular set of nodes and may wait for many hours or even days before starting, at an unpredictable (and possibly inconvenient) hour. | ||
== Monitoring jobs == <!--T:31--> | == Monitoring jobs == <!--T:31--> | ||
| Line 262: | Line 262: | ||
<!--T:169--> | <!--T:169--> | ||
Output from a non-interactive Slurm job is normally | Output from a non-interactive Slurm job is normally <i>buffered</i>, which means that there is usually a delay between when data is written by the job and when you can see the output on a login node. Depending on the application you are running and the load on the filesystem, this delay can range from less than a second to many minutes, or until the job completes. | ||
<!--T:170--> | <!--T:170--> | ||
There are methods to reduce or eliminate the buffering, but we do not recommend using them because buffering is vital to preserving the overall performance of the filesystem. If you need to monitor the output from a job in | There are methods to reduce or eliminate the buffering, but we do not recommend using them because buffering is vital to preserving the overall performance of the filesystem. If you need to monitor the output from a job in <i>real time</i>, we recommend you run an [[#Interactive_jobs|interactive job]] as described above. | ||
=== Completed jobs === <!--T:150--> | === Completed jobs === <!--T:150--> | ||
| Line 465: | Line 465: | ||
== Automating job submission == <!--T:174--> | == Automating job submission == <!--T:174--> | ||
As described earlier, [[#Array job|array jobs]] can be used to automate job submission. We provide a few other (more advanced) tools designed to facilitate running a large number of related serial, parallel, or GPU calculations. This practice is sometimes called | As described earlier, [[#Array job|array jobs]] can be used to automate job submission. We provide a few other (more advanced) tools designed to facilitate running a large number of related serial, parallel, or GPU calculations. This practice is sometimes called <i>farming</i>, <i>serial farming</i>, or <i>task farming</i>. In addition to automating the workflow, these tools can also improve computational efficiency by bundling up many short computations into fewer tasks of longer duration. | ||
<!--T:175--> | <!--T:175--> | ||