2. Environment setup

This chapter describes the set up steps to use TensorFlow Serving.

The Graphcore distributions of TensorFlow 2 and TensorFlow Serving are included in the Poplar SDK.

You need to download and install the Poplar SDK as described in the getting started guide for your system before you can complete the steps in this chapter:

Enable Poplar SDK
Create and enable a Python virtual environment
Install the TensorFlow 2 for IPU wheels and validate (optional)
Install TensorFlow Serving API

2.1. Enable Poplar SDK

You need to enable the Poplar SDK before you can use the Graphcore distributions of TensorFlow 2 and TensorFlow Serving.

$ source [path-to-sdk]/enable

where [path-to-sdk] is the path to the Poplar SDK.

You can verify that Poplar has been successfully set up by running:

$ popc --version

This will display the version of the installed software.

2.2. Create and enable a Python virtual environment

It is recommended that you work in a Python virtual environment. You can create and activate a virtual environment as follows:

$ python[py_ver] -m venv ~/[base_dir]/[venv_name]
$ source ~/[base_dir]/[venv_name]/bin/activate

where [base_dir] is a location of your choice and [venv_name] is the name of the directory that will be created for the virtual environment. [py_ver] is the version of Python you are using and it depends on your OS.

You can get more information about the versions of Python and other tools supported in the Poplar SDK for different operating systems in the Poplar SDK 3.3.0 Release Notes. You can check which OS you are running with lsb_release -a.

2.3. Install the TensorFlow 2 for IPU wheels and validate (optional)

Note

Although you can use the TensorFlow Serving API with the standard TensorFlow 2 distribution, we recommend using the Graphcore distribution of TensorFlow 2 for the best performance.

This section describes how to install the Graphcore distributions of TensorFlow and Keras and an add-ons package.

There are two TensorFlow 2 wheels included in the Poplar SDK, one for AMD processors and one for Intel processors. Check which processor is used on your system by running:

$ lscpu | grep name

Install the wheel files needed to run TensorFlow 2 on the IPU.

$ python -m pip install ${POPLAR_SDK_ENABLED?}/../tensorflow-2.*+amd_*.whl
$ python -m pip install --force-reinstall --no-deps ${POPLAR_SDK_ENABLED?}/../keras-2.*.whl
$ python -m pip install ${POPLAR_SDK_ENABLED?}/../ipu_tensorflow_addons-2.*.whl
$ python3 -c "from tensorflow.python import ipu"
$ python3 -c "import keras"
$ python3 -c "from ipu_tensorflow_addons.keras import layers"

$ python -m pip install ${POPLAR_SDK_ENABLED?}/../tensorflow-2.*+intel_*.whl
$ python -m pip install --force-reinstall --no-deps ${POPLAR_SDK_ENABLED?}/../keras-2.*.whl
$ python -m pip install ${POPLAR_SDK_ENABLED?}/../ipu_tensorflow_addons-2.*.whl
$ python3 -c "from tensorflow.python import ipu"
$ python3 -c "import keras"
$ python3 -c "from ipu_tensorflow_addons.keras import layers"

where POPLAR_SDK_ENABLED is the location of the Poplar SDK defined when the SDK was enabled. The ? ensures that an error message is displayed if Poplar has not been enabled. In this case, please enable the SDK first.

2.4. Install TensorFlow Serving API

To send gRPC requests you need to install the TensorFlow Serving API package, which is not included in the Poplar SDK.

You need to use pip and install the TensorFlow Serving API package from the official distribution. It is crucial to pass the --no-deps argument to pip so it does not reinstall the TensorFlow package.

Note

The major and minor versions of the TensorFlow Serving API should match the version of TensorFlow you use. You can check the TensorFlow version with:

$ python -m pip show tensorflow

For example, if you use TensorFlow version 2.6.3, the package can be installed by running:

$ python -m pip install --no-deps tensorflow-serving-api==2.6.*

Search help