1. Introduction

This document introduces the Graphcore distribution of TensorFlow Serving from the perspective of exporting models from TensorFlow and running them on IPUs using TensorFlow Serving. It mostly focuses on IPU-specific functionalities of TensorFlow Serving, thus it assumes you have some knowledge of TensorFlow and the original TensorFlow Serving. We recommend reading the original TensorFlow Serving documentation before reading this document.

Note

This document is for the distribution of TensorFlow Serving for TensorFlow 2. For information on TensorFlow Serving for TensorFlow 1 refer to IPU TensorFlow Serving 1 User Guide.

In this document, we show how to set up the environment, export a basic model from TensorFlow and use it with TensorFlow Serving. We cover exporting both Keras and non-Keras models, analysing the exported SavedModel and sending inference requests to TensorFlow Serving using both REST and gRPC APIs.