4. Model handling
Each framework is capable of exporting and importing programs designed for the IPU architecture. PopEF was created to unify and facilitate this process. This chapter describes the general model export and import workflows and links to the relevant information for each framework.
The Poplar SDK supports three high-level frameworks that allow you to create, export, import and execute a model. These are PyTorch for the IPU, TensorFlow for the IPU and PopART.
4.1. Exporting models
Each framework must be able to save its state after the model compilation is complete, so that it can be restored at a later time. To make this possible, it is necessary to save such elements as:
a serialised Poplar executable
its associated metadata
tensor data blobs if model parameters have not been frozen (which means are not constant and saved inside the PopEF file)
a framework-specific opaque blob to store information only relevant to that framework.
The general export process is shown in Fig. 4.1.
4.1.2. How to export
Exported models are stored as PopEF files. Each framework supports pre-compilation of executables for the IPU on a machine that doesn’t necessarily have access to IPUs.
Instructions for exporting models in the frameworks supported by the Poplar SDK are:
Enable caching allows you to implicitly export a model to the specified directory after the compilation process.
Precompilation allows you to to compile the model and run it later. This is useful for when you don’t have immediate access to a machine with IPUs.
Explicit API functions for exporting a model:
Implicit export of a model to the specified directory with
popart::SessionOptionsis possible using two options:
Explicit export is possible using:
The same operations are also available in the PopART Python API
4.1.3. Poplar executable
In this case, you must prepare the
Metadata structure yourself and any other data for the program, such as tensor data blobs, to make the PopEF data complete.
4.2. Importing models
As with exporting models, all frameworks also provide model import functionality.
Instructions for importing models into the frameworks supported by the Poplar SDK are:
You can also import and run precompiled models using:
4.2.1. Model management
To facilitate the management of imported models, PopEF provides two classes:
ModelBuilder. They let you
bind blobs from PopEF files to models. Based on
the data from the
Model class, it is possible to run the model using the
To do this, you must first deserialize the contents of the blob the model contains.
This is possible with the
function. The return value is an object of type
that can be used to create a
Using the remaining data from
Model and the
poplar::Engine object, you can run the model yourself.