cc_staff
213
edits
Graham: Difference between revisions
Jump to navigation
Jump to search
no edit summary
No edit summary |
No edit summary |
||
| Line 159: | Line 159: | ||
==Volta GPU nodes on Graham== <!--T:46--> | ==Volta GPU nodes on Graham== <!--T:46--> | ||
Additional, new Volta GPU nodes were added, as described in the table above. There are 9 Volta nodes in total. | |||
In 7 of these, four GPUs are connected to each CPU socket (except for one node, which is only populated with 6 GPUs, three per socket). | |||
<!--T:50--> | <!--T:50--> | ||
| Line 169: | Line 169: | ||
<!--T:52--> | <!--T:52--> | ||
'''Important''': You should scale the number of CPUs requested, keeping the ratio of CPUs to GPUs at 3.5 or less. For example, if you want to run a job using 4 GPUs, you should request '''at most 14 CPU cores'''. For a job with 1 GPU, you should request '''at most 3 CPU cores'''. Users are allowed to run a few short test jobs (shorter than 1 hour) that break this rule to see how your code performs. | '''Important''': You should scale the number of CPUs requested, keeping the ratio of CPUs to GPUs at 3.5 or less on 28 core nodes. For example, if you want to run a job using 4 GPUs, you should request '''at most 14 CPU cores'''. For a job with 1 GPU, you should request '''at most 3 CPU cores'''. Users are allowed to run a few short test jobs (shorter than 1 hour) that break this rule to see how your code performs. | ||
The two newest Volta nodes have 40 cores so above core requests can be adjusted accordingly. They also have NVLINK, which can provide huge benefits for situation where memory bandwidth between GPUs is the bottleneck. To use one of these NVLINK nodes, it should be requested directly, by adding the option '''--nodelist=gra1337''' or '''--nodelist=gra1338''' to the job submission script. | |||
<!--T:53--> | <!--T:53--> | ||