TensorFlow is "an open-source software library for Machine Intelligence".
These instructions install Tensorflow into your home directory using Compute Canada's pre-built Python wheels. Custom Python wheels are stored in
/cvmfs/soft.computecanada.ca/custom/python/wheelhouse/. To install TensorFlow's wheel we will use the
pip command and install it into a Python virtual environment. The below instructions install for Python 3.5.2 but you can also install for Python 3.5.Y or 2.7.X by loading a different Python module.
Load modules required by TensorFlow:
[name@server $] module load python/3.5.2
Create a new Python virtual environment:
[name@server $] virtualenv tensorflow
Activate your newly created Python virtual environment:
[name@server $] source tensorflow/bin/activate
Install TensorFlow into your newly created virtual environment using the command from either one of the two following subsections.
(tensorflow) [name@server $] pip install tensorflow-cpu
(tensorflow) [name@server $] pip install tensorflow-gpu
Submitting a TensorFlow job with a GPU
Once you have the above setup completed you can submit a TensorFlow job as
[name@server $] sbatch tensorflow-test.sh
The job submission script has the content
#!/bin/bash #SBATCH --gres=gpu:1 # request GPU "generic resource" #SBATCH --cpus-per-task=6 # maximum CPU cores per GPU request: 6 on Cedar, 16 on Graham. #SBATCH --mem=32000M # memory per node #SBATCH --time=0-03:00 # time (DD-HH:MM) #SBATCH --output=%N-%j.out # %N for node name, %j for jobID module load cuda cudnn python/3.5.2 source tensorflow/bin/activate python ./tensorflow-test.py
while the Python script has the form,
import tensorflow as tf node1 = tf.constant(3.0, dtype=tf.float32) node2 = tf.constant(4.0) # also tf.float32 implicitly print(node1, node2) sess = tf.Session() print(sess.run([node1, node2]))
Once the above job has completed (should take less than a minute) you should see an output file called something like cdr116-122907.out with contents similar to the following example,
2017-07-10 12:35:19.489458: I tensorflow/core/common_runtime/gpu/gpu_device.cc:940] Found device 0 with properties: name: Tesla P100-PCIE-12GB major: 6 minor: 0 memoryClockRate (GHz) 1.3285 pciBusID 0000:82:00.0 Total memory: 11.91GiB Free memory: 11.63GiB 2017-07-10 12:35:19.491097: I tensorflow/core/common_runtime/gpu/gpu_device.cc:961] DMA: 0 2017-07-10 12:35:19.491156: I tensorflow/core/common_runtime/gpu/gpu_device.cc:971] 0: Y 2017-07-10 12:35:19.520737: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1030] Creating TensorFlow device (/gpu:0) -> (device: 0, name: Tesla P100-PCIE-12GB, pci bus id: 0000:82:00.0) Tensor("Const:0", shape=(), dtype=float32) Tensor("Const_1:0", shape=(), dtype=float32) [3.0, 4.0]
TensorFlow can run on all GPU node types. Cedar's GPU large node type, which is equipped with 4 x P100-PCIE-16GB with GPUDirect P2P enabled between each pair, is highly recommended for large scale Deep Learning or Machine Learning research. See Using GPUs with SLURM for more information.