cc_staff
353
edits
AI and Machine Learning: Difference between revisions
Jump to navigation
Jump to search
Marked this version for translation
(Add troubleshooting section) |
(Marked this version for translation) |
||
| Line 97: | Line 97: | ||
Note that Comet is not currently available on Graham. | Note that Comet is not currently available on Graham. | ||
== Troubleshooting == | == Troubleshooting == <!--T:31--> | ||
=== Determinism with RNN using CUDA === | === Determinism with RNN using CUDA === <!--T:32--> | ||
<!--T:33--> | |||
RNN and multi-head attention API calls may exhibit non-deterministic behavior when the cuDNN library is built with CUDA Toolkit 10.2 or higher. The user can eliminate the non-deterministic behavior of cuDNN RNN and multi-head attention APIs by setting a single buffer size in the CUBLAS_WORKSPACE_CONFIG environmental variable, for example, :16:8 or :4096:2, which instructs cuBLAS to allocate eight buffers of 16 KB each in GPU memory or two buffers of 4 MB each. | RNN and multi-head attention API calls may exhibit non-deterministic behavior when the cuDNN library is built with CUDA Toolkit 10.2 or higher. The user can eliminate the non-deterministic behavior of cuDNN RNN and multi-head attention APIs by setting a single buffer size in the CUBLAS_WORKSPACE_CONFIG environmental variable, for example, :16:8 or :4096:2, which instructs cuBLAS to allocate eight buffers of 16 KB each in GPU memory or two buffers of 4 MB each. | ||
</translate> | </translate> | ||