5. PopEF C++ API reference

This chapter describes the PopEF C++ API.

5.1. Metadata

enum class popef::BlobType

Values:

enumerator POPLAR_EXECUTABLE
enumerator METADATA
enumerator DATA_TENSOR
enumerator DATA_FEED
enumerator OPAQUE
enum class popef::DataType

Values:

enumerator BOOL
enumerator F8
enumerator F16
enumerator F32
enumerator F64
enumerator S8
enumerator U8
enumerator S16
enumerator U16
enumerator S32
enumerator U32
enumerator S64
enumerator U64
enumerator F8143
enumerator F8152
enum class popef::TensorType

Values:

enumerator UNKNOWN
enumerator INPUT
enumerator OUTPUT
class Option

Capnp-backed structure encapsulating a key-value pair.

Used in the engine and device options maps in the metadata.

Unnamed Group

Option() = default
Option(const std::string &name, const std::string &value)
const std::string &name() const
void setName(const std::string &value)
const std::string &value() const
void setValue(const std::string &value)
class ProgramFlow

Capnp-backed structure encapsulating lists of program indexes to be run during the load, main and save phases.

Unnamed Group

using ProgramIndexType = std::uint32_t
std::vector<ProgramIndexType> &load()
std::vector<ProgramIndexType> &main()
std::vector<ProgramIndexType> &save()
const std::vector<uint32_t> &load() const
void setLoad(const std::vector<uint32_t> &value)
const std::vector<uint32_t> &main() const
void setMain(const std::vector<uint32_t> &value)
const std::vector<uint32_t> &save() const
void setSave(const std::vector<uint32_t> &value)
class TensorInfo

Capnp-backed structure encapsulating the shape and data type of a tensor.

Note that you can create several types of tensors:

  • matrices: shape: [2, 3, 4], dt: F32 (Please be aware that [1, 1, 1] is a matrix)

  • vectors: shape: [20], dt: S32 (Please be aware [1] is a vector)

  • scalars: shape: [], dt: U32 ([] means that shape is empty)

Unnamed Group

using ShapeDimType = std::int64_t
bool operator==(const TensorInfo &rhs) const
bool operator!=(const TensorInfo &rhs) const
std::vector<int64_t> &shape()
int64_t sizeInBytes() const
const std::vector<int64_t> &shape() const
void setShape(const std::vector<int64_t> &value)
DataType dataType() const
void setDataType(DataType value)
class TensorDataInfo

Capnp-backed structure encapsulating metadata about a tensor.

Unnamed Group

bool operator==(const TensorDataInfo &rhs) const
bool operator!=(const TensorDataInfo &rhs) const
TensorInfo &tensorInfo()
const std::string &name() const
void setName(const std::string &value)
const TensorInfo &tensorInfo() const
void setTensorInfo(const TensorInfo &value)
class FeedDataInfo

Capnp-backed structure encapsulating metadata about a data feed.

Unnamed Group

using NumTensorsType = std::uint32_t
FeedDataInfo()
bool operator==(const FeedDataInfo &rhs) const
bool operator!=(const FeedDataInfo &rhs) const
TensorInfo &tensorInfo()
const std::string &name() const
void setName(const std::string &value)
const TensorInfo &tensorInfo() const
void setTensorInfo(const TensorInfo &value)
uint32_t numTensors() const
void setNumTensors(uint32_t value)
class Anchor

Capnp-backed structure mapping input/output data to a named program entry/exit point.

Unnamed Group

using ProgramsType = std::uint32_t
Anchor()
bool operator==(const Anchor &rhs) const
bool operator!=(const Anchor &rhs) const
TensorInfo &tensorInfo()
const std::string &name() const
void setName(const std::string &value)
const std::vector<uint32_t> &programs() const
void setPrograms(const std::vector<uint32_t> &value)
const std::string &handle() const
void setHandle(const std::string &value)
const TensorInfo &tensorInfo() const
void setTensorInfo(const TensorInfo &value)
TensorType type() const
void setType(TensorType value)
bool isPerReplica() const
void setIsPerReplica(bool value)
bool useRemoteBuffers() const
void setUseRemoteBuffers(bool value)
uint32_t repeats() const
void setRepeats(uint32_t value)
class Metadata

Capnp-backed class encapsulating all metadata for a specific executable.

Unnamed Group

using ReplicationFactorType = std::uint32_t
using NumIpusType = std::uint32_t
using NumProcessesType = std::uint32_t
using IpuVersionType = std::uint32_t
std::string version

The version string applied to this metadata.

Metadata()
std::vector<Anchor> &anchors()
std::vector<Option> &engineOptions()
std::vector<Option> &deviceOptions()
ProgramFlow &programFlow()
uint32_t replicationFactor() const
void setReplicationFactor(uint32_t value)
uint32_t numIpus() const
void setNumIpus(uint32_t value)
const std::string &seedHandle() const
void setSeedHandle(const std::string &value)
const std::vector<Anchor> &anchors() const
void setAnchors(const std::vector<Anchor> &value)
const std::string &executable() const
void setExecutable(const std::string &value)
uint32_t numProcesses() const
void setNumProcesses(uint32_t value)
uint32_t ipuVersion() const
void setIpuVersion(uint32_t value)
bool isPOD() const
void setIsPOD(bool value)
bool isInference() const
void setIsInference(bool value)
const std::vector<Option> &engineOptions() const
void setEngineOptions(const std::vector<Option> &value)
const std::vector<Option> &deviceOptions() const
void setDeviceOptions(const std::vector<Option> &value)
const ProgramFlow &programFlow() const
void setProgramFlow(const ProgramFlow &value)
const std::unordered_map<uint32_t, std::string> &programsMap() const
void setProgramsMap(const std::unordered_map<uint32_t, std::string> &value)
uint32_t deviceIterations() const
void setDeviceIterations(uint32_t value)
uint32_t deviceIteration() const
void setDeviceIteration(uint32_t value)

5.2. Classes for writing PopEF streams

enum class popef::FileWriter::Mode

Specifies available file writing modes.

Values:

enumerator OVERWRITE

Allows to overwrite an existing file or create a new one.

enumerator APPEND

Allows to add additional blobs to an existing popef file.

class ExclusiveWriter

Abstract base class for blob writers.

Only one ExclusiveWriter can be active at a time.

Subclassed by popef::BlobWriter, popef::MultiFeedWriter

Public Functions

virtual ~ExclusiveWriter()
virtual void close() = 0

Close the writer, ready for the next one to be created.

Should leave the stream pointing to the end of the blob, so that the next one can be written.

class BlobWriter : public popef::ExclusiveWriter

Object to write the data associated to a blob.

Write the data associated to the object just created using BlobWriter::write().

Attention

You should not instantiate this class directly, it should always be created by a Writer.

Public Functions

explicit BlobWriter(std::unique_ptr<BlobWriterImpl> impl)

Create a BlobWriter using the given implementation.

Attention

You should not instantiate this class directly, it should always be created by a Writer.

~BlobWriter() override
virtual void close() override

Close the BlobWriter.

Public Members

std::ostream stream

Writing data to a blob is possible through the ostream object.

class MultiFeedWriter : public popef::ExclusiveWriter

Object to write to multiple feeds simultaneously, allowing interleaved writes.

Call MultiFeedWriter::addFeed() to add a feed, then write to it using MultiFeedWriter::write(), passing the zero-based index of the feed. You can interleave writes to different feeds in any way you please.

Public Functions

MultiFeedWriter(std::shared_ptr<OutputStream> stream, const std::string &version)

Construct a MultiFeedWriter.

Parameters
  • stream[in] output stream to write to.

  • version[in] version string for the feed blobs.

~MultiFeedWriter() override
void addFeed(const FeedDataInfo &fdi)

Add a new feed.

Warning

Note that the FeedDataInfo structure passed to addFeed must contain a valid (>0) number of tensors.

Parameters

fdi – The FeedDataInfo structure which applies to the feed to be added.

void write(size_t feed, const char *data, size_t size)

Write data to a feed.

Parameters
  • feed – The zero-based index of the feed to write to.

  • data – The data buffer.

  • size – The number of bytes to write.

void write(const std::string &feed, const char *data, size_t size)

Write data to a feed.

Parameters
  • feed – The feed name to write to.

  • data – The data buffer.

  • size – The number of bytes to write.

virtual void close() override

Close the MultiFeedWriter.

class Writer

Write blobs to a given stream.

A writer is a wrapper around a stream which is used to serialise PopEF blobs to the underlying stream.

The exception to this is the MultiFeedWriter, which can handle writing to multiple feeds in an interleaved fashion. See MultiFeedWriter for details.

Warning

Because the size of most blobs cannot be predicted in advance, only one blob writer can be active at any time. The creation of a new blob writer implicitly closes the previous one.

Subclassed by popef::FileWriter

Public Functions

explicit Writer(std::ostream &out)

Instantiate a writer which will output blobs to the given stream.

Parameters

out[out] Destination stream.

void setVersion(const std::string &version)
Parameters

version[in] Version string that will be used in all the future headers created by this writer.

std::shared_ptr<BlobWriter> createExecutable(const std::string &name, bool compress = false)

Append a PoplarExecutable blob to the stream.

Parameters
  • name[in] Name of the executable.

  • compress[in] Automatically compress the executable as it is being written.

Returns

The blob writer to use to serialise the binary.

std::shared_ptr<BlobWriter> createOpaqueBlob(const std::string &name, const std::string &executable)

Append an Opaque blob to the stream.

Parameters
  • name[in] Name of the opaque blob. (Usually the name of the framework owning it.)

  • executable[in] Name of the executable the blob is associated with.

Returns

The blob writer to use to serialise the data.

std::shared_ptr<BlobWriter> createFeedData(const FeedDataInfo &obj)

Append a single feed to the stream.

If you want to add multiple feeds and write to them in an interleaved fashion, see createMultiFeedData().

Note

If the number of tensors in obj is set to 0, then the writer will update the number of tensors in the structure when it is written to the stream.

Parameters

obj[in] Information about the feed

Returns

The feed writer to use to serialise the tensors. All the tensors must be written contiguously and without gaps. The number of bytes written must match exactly the data provided in the feed’s information.

std::shared_ptr<MultiFeedWriter> createMultiFeedData()

Append multiple, interleaved feeds to the stream.

Use this method if you want to write to multiple feeds in an interleaved fashion. If you just want to write to a single feed, use createFeedData(). If using createMultiFeedData(), you must know the number of tensors in each feed in advance.

Returns

The writer used to create and write to a group of feeds.

std::shared_ptr<BlobWriter> createTensorData(const TensorDataInfo &obj)

Append a tensor to the stream.

Parameters

obj[in] Information about the tensor.

Returns

The tensor writer to use to serialise the tensor. The number of bytes written must match exactly the data provided in the feed’s information.

void write(const Metadata &obj)

Append a metadata blob to the stream.

Parameters

obj[in] Metadata object to serialise.

void close()

Finalise the writer.

After this call the stream that was passed to the constructor will no longer be used and an exception will be thrown if any object created by the Writer tries to access the stream.

virtual ~Writer()

Protected Functions

Writer()

Allows subclasses to create streams on construction before initialising the Writer using initialiseImpl().

void initialiseImpl(std::ostream &out)

Allows subclasses to create streams to initialise the Writer after construction, allowing the creation of streams at construction time.

class FileWriter : public popef::Writer

Write blobs to a given file.

It creates a file if it does not exist. If the file exists, it allows to append blobs to it or override existing content.

A writer is a wrapper around a stream which is used to serialise PopEF blobs to the underlying stream.

The exception to this is the MultiFeedWriter, which can handle writing to multiple feeds in an interleaved fashion. See MultiFeedWriter for details.

Warning

Because the size of most blobs cannot be predicted in advance, only one blob writer can be active at any time. The creation of a new blob writer implicitly closes the previous one.

Public Functions

explicit FileWriter(const std::string &filename, Mode mode = Mode::OVERWRITE)

Instantiate a Writer which will write to a file, and will manage the lifetime of the stream correctly with respect to the lifetime of the underlying Writer.

5.3. Classes for reading PopEF streams

class SubReader

Base class for subreaders (classes that read specific blobs from a stream).

Attention

You should not create SubReaders; they are created by a Reader.

Subclassed by popef::ExecutableReader, popef::FeedReader, popef::OpaqueReader, popef::TensorReader

Unnamed Group

SubReader(const SubReader &src)
SubReader(SubReader &&src)
~SubReader()
SubReader &operator=(const SubReader &src)
SubReader &operator=(SubReader &&src)

Public Functions

SubReader(std::string version_in, std::unique_ptr<InputStream> stream_in)

Create a SubReader.

Attention

SubReaders, including ExecutableReader, should not be created by users; they are created by a Reader.

Parameters
  • version_in[in] The version string for this blob.

  • stream_in[in] The wrapped stream to read the blob from.

std::streamsize getAvailableReadSize() const

Returns the number of bytes available for reading from the current position to the end of the sub-stream (blob).

Public Members

std::string version

The version string for this blob.

Protected Attributes

std::unique_ptr<InputStream> stream

InputStream to read blob data from.

class TensorReader : public popef::SubReader

Loads a tensor’s data from a stream.

Attention

SubReaders, including TensorReader, should not be created by users; they are created by a Reader.

Unnamed Group

TensorReader(const TensorReader &src)
TensorReader(TensorReader &&src)
TensorReader &operator=(const TensorReader &src)
TensorReader &operator=(TensorReader &&src)

Public Functions

TensorReader(std::string tnsr_version, std::unique_ptr<InputStream> stream)

Create a TensorReader.

Attention

SubReaders, including TensorReader, should not be created by users; they are created by a Reader.

Parameters
  • tnsr_version[in] The version string for this tensor.

  • stream[in] The wrapped stream to read the tensor from.

std::unique_ptr<std::istream> getStandaloneDataStream() const

Returns a std::istream unique_ptr which reads the tensor data (just as TensorReader::data does), but maintains the liveness chain independently of the existence of the parent TensorReader.

Public Members

TensorDataInfo info

Metadata of the serialised tensor.

std::istream data

Stream pointing to the tensor’s data.

class FeedReader : public popef::SubReader

Loads a feed’s data from a stream.

Attention

SubReaders, including FeedReader, should not be created by users; they are created by a Reader.

Unnamed Group

FeedReader(const FeedReader &src)
FeedReader(FeedReader &&src)
FeedReader &operator=(const FeedReader &src)
FeedReader &operator=(FeedReader &&src)

Public Functions

FeedReader(std::string feed_version, std::unique_ptr<InputStream> stream)

Create a FeedReader.

Attention

SubReaders, including FeedReader, should not be created by users; they are created by a Reader.

Parameters
  • feed_version[in] The version string for this data feed.

  • stream[in] The wrapped stream to read the data from.

std::unique_ptr<std::istream> getStandaloneDataStream() const

Returns a std::istream unique_ptr which reads the feed data (just as FeedReader::data does), but maintains the liveness chain independently of the existence of the parent FeedReader.

Public Members

FeedDataInfo info

Metadata of the serialised feed.

std::istream data

Stream pointing to the feed’s data.

class OpaqueReader : public popef::SubReader

Loads an opaque blob from a stream.

Attention

SubReaders, including OpaqueReader, should not be created by users; they are created by a Reader.

Unnamed Group

OpaqueReader(const OpaqueReader &src)
OpaqueReader(OpaqueReader &&src)
OpaqueReader &operator=(const OpaqueReader &src)
OpaqueReader &operator=(OpaqueReader &&src)

Public Functions

OpaqueReader(std::string opq_version, std::unique_ptr<InputStream> stream)

Create an OpaqueReader.

Attention

SubReaders, including OpaqueReader, should not be created by users; they are created by a Reader.

Parameters
  • opq_version[in] The version string for this opaque blob.

  • stream[in] The wrapped stream to read the blob from.

std::unique_ptr<std::istream> getStandaloneDataStream() const

Returns a std::istream unique_ptr which reads the blob data (just as OpaqueReader::data does), but maintains the liveness chain independently of the existence of the parent OpaqueReader.

Public Members

std::string name

Name of the opaque blob (usually the name of the framework which created it).

std::string executable

Name of the executable the opaque blob is related to.

std::istream data

Stream pointing to the blob’s data.

class ExecutableReader : public popef::SubReader

Loads a Poplar executable from a stream.

Attention

SubReaders, including ExecutableReader, should not be created by users; they are created by a Reader.

Unnamed Group

ExecutableReader(const ExecutableReader &src)
ExecutableReader(ExecutableReader &&src)
ExecutableReader &operator=(const ExecutableReader &src)
ExecutableReader &operator=(ExecutableReader &&src)

Public Functions

ExecutableReader(std::string exe_version, std::unique_ptr<InputStream> stream)

Create an ExecutableReader.

Attention

SubReaders, including ExecutableReader, should not be created by users; they are created by a Reader.

Parameters
  • exe_version[in] The version string for this executable.

  • stream[in] The wrapped stream to read the executable from.

std::unique_ptr<std::istream> getStandaloneExecutableStream() const

Returns a std::istream unique_ptr which reads the serialized executable (just as ExecutableReader::executable does), but maintains the liveness chain independently of the existence of the parent ExecutableReader.

Public Members

std::string name

The name of the executable.

bool is_compressed

If true the executable is stored compressed.

It will be decompressed on the fly as it’s being read so it doesn’t require any special handling.

Important: Seeking through a compressed executable is expensive.

std::istream executable

Stream pointing to the serialised Poplar executable.

class Reader

Reads blobs from a given stream.

A reader build a list of PopEF blobs from one or more streams.

Note

The reader only parses the metadata from each blob into memory. Binary blobs are kept as input streams, ready to be read.

Public Functions

void parseStream(const std::shared_ptr<std::istream> &in, std::streamoff size = 0)

Load PopEF blobs from the given stream.

Parameters
  • in[inout] The stream to parse.

  • size[in] How many bytes to read from the stream. If 0, parse until the end of the stream is encountered.

void parseFile(const std::string &filename)

Helper function to parse a file.

const std::vector<ExecutableReader> &executables()

Combined list of executables.

const std::vector<OpaqueReader> &opaqueBlobs()

Combined list of opaque blobs.

const std::vector<TensorReader> &tensors()

Combined list of tensors.

const std::vector<FeedReader> &feeds()

Combined list of feeds.

const std::vector<Metadata> &metadata()

Combined list of metadata.

5.4. Model representation classes

enum class popef::FilterStrategy

Strategy to use to filter objects of the same type.

Values:

enumerator USE_LAST

If there are several objects of a given type, use the last one.

enumerator ASSERT_ONE

Raise an error if there is more than one object of a given type.

enum class popef::UnknownDataStrategy

Strategy to use when encountering data sources for tensors which don’t exist in the metadata.

Values:

enumerator KEEP_ALL

Keep the readers even though the data is not used by this executable.

enumerator REMOVE_EXTRA

Remove data which isn’t used by this executable.

enumerator ASSERT_ON_EXTRA

Raise an error if the reader contains data not used by the executable.

enum class popef::VersionCheck

Strategy to use when validating the headers’ version field.

Values:

enumerator NO_CHECK

Don’t perform any validation on versions.

enumerator EXACT_MATCH

All the versions must be identical.

class Model

A Model is a combination of blobs representing a single model.

A model is defined by an executable and its associated metadata and data.

Models can be created by hand or by a ModelBuilder

Public Functions

explicit Model(ExecutableReader exec, Metadata meta)

Create a Model object.

Parameters
  • exec[in] The executable blob reader.

  • meta[in] The associated metadata.

void assertAllInputsProvided() const

Throw an exception if all the input tensors don’t have either a TensorReader or FeedReader associated with them.

Public Members

const Metadata metadata

The metadata associated with the executable used by this model.

const ExecutableReader executable

The executable used by this model.

std::map<std::string, const FeedReader> input_feeds

Map feed names to their data source.

std::map<std::string, const TensorReader> input_tensors

Map tensor names to their data source.

class ModelBuilder

Filters the list of blobs of a Reader to obtain a Model.

The builder is a state machine. Set all the filters to configure it and call createModel() to filter the blobs and instantiate the model.

Public Functions

explicit ModelBuilder(std::shared_ptr<Reader> reader)

Create a ModelBuilder object.

Parameters

reader[in] The Reader allows obtaining blobs for creating models.

std::vector<std::string> listExecutables() const

List the names of the executables contained in the reader.

void useExecutable(const std::string &name)

Set the name of the executable to use.

Note

The executable filter will be used if several executables have the same name.

Parameters

name[in] If name is empty then all the executables will be considered a match and the executable filter will be used to determine which one to use.

void setExecutableFilter(FilterStrategy filter)

In case several executables have the same name.

void setMetadataFilter(FilterStrategy filter)

After only the metadata matching the selected executable are kept, this filter will be used to determine which one to use.

void setDataFilter(FilterStrategy filter)

Filter to use for data objects (tensors / infeeds).

void setUnknownDataStrategy(UnknownDataStrategy filter)

Strategy to use when encountering data sources for tensors which don’t exist in the metadata.

void setVersionCheck(VersionCheck check)

Strategy to use to validate versions.

std::unique_ptr<Model> createModel()

Apply the filters and instantiate a model.

5.5. Error classes

class Error : public runtime_error

Exception class for PopEF.

Subclassed by popef::logging::InternalError

Public Functions

explicit Error(const char *s)

Creates an error with the specified message.

Parameters

s[in] Error message.

class InternalError : public popef::logging::Error

Exception class specific to internal errors.

This should be used as an assert, for states which should be unreachable.

Public Functions

explicit Error(const char *s)

Creates an error with the specified message.

Parameters

s[in] Error message.