2. Environment setup
This chapter describes the set up steps to use TensorFlow Serving.
The Graphcore distributions of TensorFlow 1 and TensorFlow Serving are included in the Poplar SDK.
Note
From Poplar SDK 3.1, TensorFlow 1 will only be supported in CentOS 7. In addition, Examples and Tutorials for TensorFlow 1 are only available up to version 3.0 of the SDK. There has been limited testing of the 3.0 versions of the TensorFlow 1 tutorials and examples with later versions of the Poplar SDK.
You need to download and install the Poplar SDK as described in the getting started guide for your system before you can complete the steps in this chapter:
2.1. Enable Poplar SDK
You need to enable the Poplar SDK before you can use the Graphcore distributions of TensorFlow 1 and TensorFlow Serving.
$ source [path-to-sdk]/enable
where [path-to-sdk] is the path to the Poplar SDK.
You can verify that Poplar has been successfully set up by running:
$ popc --version
This will display the version of the installed software.
2.2. Create and enable a Python virtual environment
It is recommended that you work in a Python virtual environment. You can create and activate a virtual environment as follows:
$ python[py_ver] -m venv ~/[base_dir]/[venv_name]
$ source ~/[base_dir]/[venv_name]/bin/activate
where [base_dir] is a location of your choice and [venv_name] is the name of the directory that will be created for the virtual environment. [py_ver] is the version of Python you are using and it depends on your OS.
You can get more information about the versions of Python and other tools supported in the Poplar SDK for different operating systems in the Poplar SDK 3.3.0 Release Notes. You can check which OS you are running with lsb_release -a.
2.3. Install the TensorFlow 1 for IPU wheels and validate (optional)
Note
Although you can use the TensorFlow Serving API with the standard TensorFlow 1 distribution, we recommend using the Graphcore distribution of TensorFlow 1 for the best performance.
This section describes how to install the Graphcore distributions of TensorFlow and an add-ons package.
There are two TensorFlow 1 wheels included in the Poplar SDK, one for AMD processors and one for Intel processors. Check which processor is used on your system by running:
$ lscpu | grep name
Install the wheel files needed to run TensorFlow 1 on the IPU.
$ python -m pip install ${POPLAR_SDK_ENABLED?}/../tensorflow-1.*+amd_*.whl
$ python -m pip install ${POPLAR_SDK_ENABLED?}/../ipu_tensorflow_addons-1.*.whl
$ python3 -c "from tensorflow.python import ipu"
$ python -m pip install ${POPLAR_SDK_ENABLED?}/../tensorflow-1.*+intel_*.whl
$ python -m pip install ${POPLAR_SDK_ENABLED?}/../ipu_tensorflow_addons-1.*.whl
$ python3 -c "from tensorflow.python import ipu"
where POPLAR_SDK_ENABLED is the location of the Poplar SDK defined when the SDK was enabled. The ? ensures that an error message is displayed if Poplar has not been enabled. In this case, please enable the SDK first.
2.4. Install TensorFlow Serving API
To send gRPC requests you need to install the TensorFlow Serving API package, which is not included in the Poplar SDK.
You need to use pip and install the TensorFlow Serving API package from the official distribution. It is crucial to pass the --no-deps argument to pip so it does not reinstall the TensorFlow package.
Note
The major and minor versions of the TensorFlow Serving API should match the version of TensorFlow you use. You can check the TensorFlow version with:
$ python -m pip show tensorflow
For example, if you use TensorFlow version 1.15.0, the package can be installed by running:
$ python -m pip install --no-deps tensorflow-serving-api==1.15.*