12. PopART Python API

This chapter describes the PopART Python API. Many classes are wrappers around the equivalent C++ class, for example popart.builder.Builder wraps the C++ Builder class (renamed BuilderCore in Python). There are more detailed descriptions of some functions in Section 13, PopART C++ API.

12.1. Sessions

12.1.1. Training session

class popart_core._TrainingSessionCore
broadcastWeights(self: popart_core._TrainingSessionCore, rootRank: int = 0) None

Broadcasts the weight from the PopRun instance with index rootRank to all other instances.

Parameters

rootRank – The index of the PopRun instance from which the weights should be broadcasted.

compileAndExport(self: popart_core._TrainingSessionCore, filename: str, err: popart_core.OutOfMemoryError) None
connectStreamToCallback(self: popart_core._TrainingSessionCore, arg0: str, arg1: Callable[[capsule], None], arg2: int) None

Connect a Poplar stream with a callback.

This method will be called whenever the stream will be read or was written to by the device. The memory location will only be valid for reading or writing for the duration of the callback.

Parameters
  • streamHandle – The name of the stream to connect to.

  • callback – The callback to be called whenever the stream is to be read or was written to by the device.

  • index – The replica index to connect to, when using replicated graphs. Default=0.

getCycleCount(self: popart_core._TrainingSessionCore, id: str = '') int

Copy the cycle count tensor from the device to the host.

Parameters

id – The identifier of the cycle count tensor.

getInfo(self: popart_core._TrainingSessionCore, arg0: str) popart_internal_ir.TensorInfo

Get the tensor information for a tensor.

Parameters

TensorId – The identifier of the tensor to get the tensor information for.

Returns

The tensor information for the tensor.

getIr(self: popart_core._TrainingSessionCore) popart_internal_ir.Ir
getRNGState(self: popart_core._TrainingSessionCore) List[int]
getRandomSeed(self: popart_core._TrainingSessionCore) int

Get the value of the random number generator seed.

Calling setRandomSeed() with this value (at a later stage) reinstates the random state logic that seeds random operations.

Returns

The value used to seed current random operations.

getReport(self: popart_core._TrainingSessionCore) pva::Report

Retrieve the graph report from the poplar::Engine.

The options which were passed to the Session constructor will influence the information in the report.

This method may only be called after prepareDevice() has been called.

Returns

The PopVision Analysis report object.

getSerializedGraph(self: popart_core._TrainingSessionCore) bytes

Retrieve the serialized graph from the poplar::Engine.

A JSON format report is produced.

This method may only be called after prepareDevice() has been called.

Returns

A string containing the serialized graph.

getSummaryReport(self: popart_core._TrainingSessionCore, resetProfile: bool = True) str

Retrieve the summary from from the ``poplar::Engine.`:code:`

The options which were passed to the Session constructor will influence the information in the report.

This method may only be called after prepareDevice() has been called.

Parameters

resetProfile – If true:code:, resets the execution profile. Default = true.

Returns

A string containing the report.

getTensorIds(self: popart_core._TrainingSessionCore) Set[str]

Returns the ids of all tensors in the model.

pre prepareDevice() must have been called.

loadEngineAndConnectStreams(self: popart_core._TrainingSessionCore) None

Load the engine on the device and connect the streams.

This will set up the poplar::Streams.

Note: This call is optional. The engine will implicitly be loaded on the device when required.

loadExecutable(self: popart_core._TrainingSessionCore, filename: str) None

Load the compiled executable and metadata from a file.

The file must have been created with compileAndExport(const std::string).

Parameters

filename – The name of the file to load the executable and metadata from.

modelToHost(self: popart_core._TrainingSessionCore, arg0: str) None

Write the current model to an ONNX file.

Parameters

fn – The path to file. The path can be absolute or relative. If you plan to run your program in multiple processes simultaneously, you should avoid possible race conditions by writing to different files, for example by using temporary files.

prepareDevice(self: popart_core._TrainingSessionCore, loadEngine: bool = True, err: popart_core.OutOfMemoryError) None

Prepare the network for execution.

This will create the snap::Graph and ``poplar::Engine.`:code:`

Parameters

loadEngine – If true, load the engine and connect the streams once the device is ready.

readWeights(self: popart_core._TrainingSessionCore, arg0: popart_core.IWeightsIO) None

Read the weights from the host stream memory and write to the host.

This method may only be called after weightsToHost() has been called.

Parameters

weightsIo – The weight data that is read from the host stream memory is written to the addresses in p weightsIo.out.

resetHostWeights(self: popart_core._TrainingSessionCore, modelProtoOrFilename: str, ignoreWeightsInModelWithoutCorrespondingHostWeight: bool = False) None

Reset weights with weights in an ONNX model.

Note that the only differences between the ONNX model and the current model must be the weights. No other differences are allowed.

This method only updates the weights on the host. weightsFromHost() must be called after this method to update the weights on the device.

Parameters
  • model – An ONNX model protobuf, or the name of a file containing an ONNX model protobuf.

  • ignoreWeightsInModelWithoutCorrespondingHostWeight – If true, do not throw an error if there are initializers in the ONNX model without corresponding initializer tensor(s) in the session’s IR.

run(*args, **kwargs)

Overloaded function.

  1. run(self: popart_core._TrainingSessionCore, stepio: popart_core.IStepIO, debugName: str = ‘’) -> None

Run one step.

Read input data from address in p stepIO.in.

Write the output data to addresses in p stepIO.out.

Parameters
  • stepIO – The input and output data.

  • debugName – A debug string to identify this run in logs.

  1. run(self: popart_core._TrainingSessionCore, programHandle: str, stepio: popart_core.IStepIO, debugName: str = ‘’) -> None

Run one step.

Read input data from address in p stepIO.in.

Write the output data to addresses in p stepIO.out.

Parameters
  • stepIO – The input and output data.

  • debugName – A debug string to identify this run in logs.

saveExecutable(self: popart_core._TrainingSessionCore, path: str, savePopartMetadata: bool = True, saveVariables: bool = True) None

Save a compiled graph with additional data to a file.

PopART is able to save its state after the model compilation is complete, so that it can be restored at a later time. To make this possible, it is necessary to save such elements as:

  • a serialised Poplar executable,

  • its associated metadata,

  • tensor data blobs if model parameters have not been frozen (refer to the SessionOptions::constantWeights for more information),

  • a PopART-specific opaque blob to store information only relevant to PopART. This is needed to restore PopART state.

The file will be in the PopEF format. This means that the file can be used to restore the state of the PopART program without recompiling the graph, or run inference using the Triton Inference Server with the Graphcore Triton backend. See the Poplar Triton Backend User Guide for more information. If you want to analyze file structure saved by the function please refer to the PopEF dump tool.

pre prepareDevice() must have been called.

Parameters
  • path – The name of the file or directory where the compiled executable, metadata and variables will be saved. If you specified a path to the directory, the function will write the data to the file: “<path>/executable.popef”. If the file exists, the function will overwrite the old data with the new ones.

  • savePopartMetadata – If you do not need the option to restore the PopART state later, you can set the flag to false to reduce disk space taken up by the file.

  • saveVariables – If you don’t need to save variables (tensors) state, you can set the flag to false if you want to save them later or in a different location. The function will save data consistent with the variables contained within the model.

saveVariables(self: popart_core._TrainingSessionCore, path: str) None

Save all variables to a file.

The function will save data consistent with the variables contained within the model.

The file will be in the PopEF format. If you want to analyze tensors saved by the function please refer to the PopEF dump tool.

pre prepareDevice() must have been called.

Parameters

path – The name of the file or directory where the compiled variables will be saved. If you specified a path to the directory, the function will write the data to the file: “<path>/variables.popef”. If the file exists, the function will overwrite the old data with the new ones.

setRNGState(self: popart_core._TrainingSessionCore, rngValue: List[int]) None
setRandomSeed(self: popart_core._TrainingSessionCore, seedValue: int) None

Set the value of the random number generator seed.

This method explicitly seeds all random operations. Additionally, this method derives a new state for the random number generator (RNG) from the seed and sets it on the device. This RNG state is used to resolve stochastic rounding. Note that to deterministically store and restore the combined random state for a session, do the following:

C++: ``` // Store random state (session s0). auto seed = s0.getRandomSeed(); auto rngState = s0.getRNGState();

// Restore random state (session s1). s1.setRandomSeed(seed); // <– affects RNG state, order important s1.setRNGState(rngState); ```

Python: ``` # Store random state (session s0). seed = s0.getRandomSeed() rngState = s0.getRNGState()

# Restore random state (session s1). s1.setRandomSeed(seed) // <– affects RNG state, order important s1.setRNGState(rngState) ```

Parameters

seedValue – The value of the seed.

updateEngineCache(self: popart_core._TrainingSessionCore) None

Update cacheEntries from engine cache directory and update ir::hashMatched_ with the updated cacheEntries

updateExternallySavedTensorLocations(self: popart_core._TrainingSessionCore, arg0: str, arg1: str) None

Update the tensor locations of tensors in the session’s ONNX model.

A new file will be created at this point, and written to when the ONNX model is saved with a subsequent call to modelToHost().

Parameters
  • fromLocation – All externally saved tensors with location p fromLocation will have their location updated to p toLocation.

  • toLocation – The updated tensor locations. This must not already exist.

updateOptimizerFromHost(self: popart_core._TrainingSessionCore, arg0: popart_core.Optimizer) None

Update the optimizer from the host.

This method updates the optimizer and the associated hyperparameters but not the optimizer state tensors.

NOTE: The optimizer parameter has to be compatible with the optimizer passed to the TrainingSession constructor. For example, you cannot call this function with an SDG1 optimizer if you created the session with an SDG0 optimizer. This is because it is not possible to change the IR after a session has been constructed.

Parameters

optimizer – A pointer to a popart::Optimizer.

weightsFromHost(self: popart_core._TrainingSessionCore) None

Copy weights from the host to the device.

weightsToHost(self: popart_core._TrainingSessionCore) None

Copy the weights from the device to the host steam memory.

writeWeights(self: popart_core._TrainingSessionCore, arg0: popart_core.IWeightsIO) None

Write the weights from the host to the IR tensor memory.

This method may only be called after weightsFromHost() has been called.

Parameters

weightsIo – The weight data is written to the addresses in p weightsIo.out.

12.1.2. Inference session

class popart_core._InferenceSessionCore
areHostWeightsInSync(self: popart_core._InferenceSessionCore) bool

Are all the weights in sync with the ipu?

checkInplacingAmbiguity(self: popart_core._InferenceSessionCore) None
compileAndExport(self: popart_core._InferenceSessionCore, filename: str, err: popart_core.OutOfMemoryError) None
copyDeviceWeightsToHost(self: popart_core._InferenceSessionCore) None

Copy data from the device, to the host buffers, to the tensor.tensorData() buffers. Will not run a WeightsToHost program if weights already in sync with ipu. After WeightsToHost, marks the weights as in sync with the ipu.

getAllTensorIds(self: popart_core._InferenceSessionCore) Set[str]

Returns the ids of all tensors in the model.

pre prepareDevice() must have been called.

getCycleCount(self: popart_core._InferenceSessionCore, id: str = '') int

Copy the cycle count tensor from the device to the host.

Parameters

id – The identifier of the cycle count tensor.

getInfo(self: popart_core._InferenceSessionCore, arg0: str) popart_internal_ir.TensorInfo

Get the tensor information for a tensor.

Parameters

TensorId – The identifier of the tensor to get the tensor information for.

Returns

The tensor information for the tensor.

getIr(self: popart_core._InferenceSessionCore) popart_internal_ir.Ir
getRNGState(self: popart_core._InferenceSessionCore) List[int]
getRandomSeed(self: popart_core._InferenceSessionCore) int

Get the value of the random number generator seed.

Calling setRandomSeed() with this value (at a later stage) reinstates the random state logic that seeds random operations.

Returns

The value used to seed current random operations.

getReport(self: popart_core._InferenceSessionCore) pva::Report

Retrieve the graph report from the poplar::Engine.

The options which were passed to the Session constructor will influence the information in the report.

This method may only be called after prepareDevice() has been called.

Returns

The PopVision Analysis report object.

getSerializedGraph(self: popart_core._InferenceSessionCore) bytes
getSummaryReport(self: popart_core._InferenceSessionCore, resetProfile: bool = True) str

Retrieve the summary from from the ``poplar::Engine.`:code:`

The options which were passed to the Session constructor will influence the information in the report.

This method may only be called after prepareDevice() has been called.

Parameters

resetProfile – If true:code:, resets the execution profile. Default = true.

Returns

A string containing the report.

loadEngineAndConnectStreams(self: popart_core._InferenceSessionCore) None

Load the engine on the device and connect the streams.

This will set up the poplar::Streams.

Note: This call is optional. The engine will implicitly be loaded on the device when required.

loadExecutable(self: popart_core._InferenceSessionCore, filename: str) None

Load the compiled executable and metadata from a file.

The file must have been created with compileAndExport(const std::string).

Parameters

filename – The name of the file to load the executable and metadata from.

markHostWeightsInSync(self: popart_core._InferenceSessionCore) None

Mark the d2hWeightBuffers as in sync with the ipu.

markHostWeightsOutOfSync(self: popart_core._InferenceSessionCore) None

Mark the d2hWeightBuffers as out of sync with the ipu.

modelToHost(self: popart_core._InferenceSessionCore, arg0: str) None

Write the current model to an ONNX file.

Parameters

fn – The path to file. The path can be absolute or relative. If you plan to run your program in multiple processes simultaneously, you should avoid possible race conditions by writing to different files, for example by using temporary files.

prepareDevice(self: popart_core._InferenceSessionCore, loadEngine: bool = True, err: popart_core.OutOfMemoryError) None

Prepare the network for execution.

This will create the snap::Graph and ``poplar::Engine.`:code:`

Parameters

loadEngine – If true, load the engine and connect the streams once the device is ready.

readWeights(self: popart_core._InferenceSessionCore, arg0: popart_core.IWeightsIO) None

Read the weights from the host stream memory and write to the host.

This method may only be called after weightsToHost() has been called.

Parameters

weightsIo – The weight data that is read from the host stream memory is written to the addresses in p weightsIo.out.

resetHostWeights(self: popart_core._InferenceSessionCore, modelProtoOrFilename: str, ignoreWeightsInModelWithoutCorrespondingHostWeight: bool = False) None

Reset weights with weights in an ONNX model.

Note that the only differences between the ONNX model and the current model must be the weights. No other differences are allowed.

This method only updates the weights on the host. weightsFromHost() must be called after this method to update the weights on the device.

Parameters
  • model – An ONNX model protobuf, or the name of a file containing an ONNX model protobuf.

  • ignoreWeightsInModelWithoutCorrespondingHostWeight – If true, do not throw an error if there are initializers in the ONNX model without corresponding initializer tensor(s) in the session’s IR.

run(*args, **kwargs)

Overloaded function.

  1. run(self: popart_core._InferenceSessionCore, stepio: popart_core.IStepIO, debugName: str = ‘’) -> None

Run one step.

Read input data from address in p stepIO.in.

Write the output data to addresses in p stepIO.out.

Parameters
  • stepIO – The input and output data.

  • debugName – A debug string to identify this run in logs.

  1. run(self: popart_core._InferenceSessionCore, programHandle: str, stepio: popart_core.IStepIO, debugName: str = ‘’) -> None

Run one step.

Read input data from address in p stepIO.in.

Write the output data to addresses in p stepIO.out.

Parameters
  • stepIO – The input and output data.

  • debugName – A debug string to identify this run in logs.

saveExecutable(self: popart_core._InferenceSessionCore, path: str, savePopartMetadata: bool = True, saveVariables: bool = True) None

Save a compiled graph with additional data to a file.

PopART is able to save its state after the model compilation is complete, so that it can be restored at a later time. To make this possible, it is necessary to save such elements as:

  • a serialised Poplar executable,

  • its associated metadata,

  • tensor data blobs if model parameters have not been frozen (refer to the SessionOptions::constantWeights for more information),

  • a PopART-specific opaque blob to store information only relevant to PopART. This is needed to restore PopART state.

The file will be in the PopEF format. This means that the file can be used to restore the state of the PopART program without recompiling the graph, or run inference using the Triton Inference Server with the Graphcore Triton backend. See the Poplar Triton Backend User Guide for more information. If you want to analyze file structure saved by the function please refer to the PopEF dump tool.

pre prepareDevice() must have been called.

Parameters
  • path – The name of the file or directory where the compiled executable, metadata and variables will be saved. If you specified a path to the directory, the function will write the data to the file: “<path>/executable.popef”. If the file exists, the function will overwrite the old data with the new ones.

  • savePopartMetadata – If you do not need the option to restore the PopART state later, you can set the flag to false to reduce disk space taken up by the file.

  • saveVariables – If you don’t need to save variables (tensors) state, you can set the flag to false if you want to save them later or in a different location. The function will save data consistent with the variables contained within the model.

saveVariables(self: popart_core._InferenceSessionCore, path: str) None

Save all variables to a file.

The function will save data consistent with the variables contained within the model.

The file will be in the PopEF format. If you want to analyze tensors saved by the function please refer to the PopEF dump tool.

pre prepareDevice() must have been called.

Parameters

path – The name of the file or directory where the compiled variables will be saved. If you specified a path to the directory, the function will write the data to the file: “<path>/variables.popef”. If the file exists, the function will overwrite the old data with the new ones.

setEngineIsLoaded(self: popart_core._InferenceSessionCore, isLoaded: bool) None
setRNGState(self: popart_core._InferenceSessionCore, rngValue: List[int]) None
setRandomSeed(self: popart_core._InferenceSessionCore, seedValue: int) None

Set the value of the random number generator seed.

This method explicitly seeds all random operations. Additionally, this method derives a new state for the random number generator (RNG) from the seed and sets it on the device. This RNG state is used to resolve stochastic rounding. Note that to deterministically store and restore the combined random state for a session, do the following:

C++: ``` // Store random state (session s0). auto seed = s0.getRandomSeed(); auto rngState = s0.getRNGState();

// Restore random state (session s1). s1.setRandomSeed(seed); // <– affects RNG state, order important s1.setRNGState(rngState); ```

Python: ``` # Store random state (session s0). seed = s0.getRandomSeed() rngState = s0.getRNGState()

# Restore random state (session s1). s1.setRandomSeed(seed) // <– affects RNG state, order important s1.setRNGState(rngState) ```

Parameters

seedValue – The value of the seed.

updateExternallySavedTensorLocations(self: popart_core._InferenceSessionCore, arg0: str, arg1: str) None

Update the tensor locations of tensors in the session’s ONNX model.

A new file will be created at this point, and written to when the ONNX model is saved with a subsequent call to modelToHost().

Parameters
  • fromLocation – All externally saved tensors with location p fromLocation will have their location updated to p toLocation.

  • toLocation – The updated tensor locations. This must not already exist.

weightsFromHost(self: popart_core._InferenceSessionCore) None

Copy weights from the host to the device.

weightsToHost(self: popart_core._InferenceSessionCore) None

Copy the weights from the device to the host steam memory.

writeWeights(self: popart_core._InferenceSessionCore, arg0: popart_core.IWeightsIO) None

Write the weights from the host to the IR tensor memory.

This method may only be called after weightsFromHost() has been called.

Parameters

weightsIo – The weight data is written to the addresses in p weightsIo.out.

12.1.3. Session Options

12.2. Data input and output

Note

The base class for data input and output in PopART is popart::IStepIO. The way in which this class is used is detailed in Section 13.2, Data input and output (IStepIO).

12.3. Tensors

12.4. Optimizers

12.4.1. SGD

12.4.2. ConstSGD

12.4.3. Adam

12.4.4. AdaDelta, RMSProp & AdaGrad

12.5. Builder

12.5.1. AiGraphcoreOpset1

12.6. Data flow

12.7. Device manager

12.8. Ops

12.8.1. Op definition for PopART IR

12.9. Patterns

12.10. Utility classes

12.10.1. Writer

12.10.2. Error handling

12.10.3. Debug context

12.10.4. Input shape information

12.10.5. Type definitions

12.10.6. Enums