1. Introduction

This document introduces the Graphcore distribution of TensorFlow Serving from the perspective of exporting models from TensorFlow and running them on IPUs using TensorFlow Serving. It mostly focuses on IPU-specific functionalities of TensorFlow Serving, thus it assumes you have some knowledge of TensorFlow and the original TensorFlow Serving. We recommend reading the original TensorFlow Serving documentation before reading this document.

Note

This document is for the distribution of TensorFlow Serving for TensorFlow 1. For information on TensorFlow Serving for TensorFlow 2 refer to IPU TensorFlow Serving 2 User Guide.

See the getting started guide for your IPU system in the Graphcore documentation portal for information on installing the Poplar SDK and the Graphcore distributions of TensorFlow and TensorFlow Serving.

In this document we show how to export a basic model from TensorFlow and use it with TensorFlow Serving. We cover exporting the model, analyzing the exported SavedModel and sending inference requests to TensorFlow Serving using both REST and gRPC APIs.