5. PopEF C++ API reference

This chapter describes the PopEF C++ API.

5.1. Metadata

enum class popef::BlobType

Values:

enumerator POPLAR_EXECUTABLE

enumerator METADATA

enumerator DATA_TENSOR

enumerator DATA_FEED

enumerator OPAQUE

enum class popef::DataType

Values:

enumerator BOOL

enumerator F8

enumerator F16

enumerator F32

enumerator F64

enumerator S8

enumerator U8

enumerator S16

enumerator U16

enumerator S32

enumerator U32

enumerator S64

enumerator U64

enumerator F8143

enumerator F8152

enum class popef::TensorType

Values:

enumerator UNKNOWN

enumerator INPUT

enumerator OUTPUT

class Option

Capnp-backed structure encapsulating a key-value pair.

Used in the engine and device options maps in the metadata.

Unnamed Group

Option() = default

Option(const std::string &name, const std::string &value)

const std::string &name() const

void setName(const std::string &value)

const std::string &value() const

void setValue(const std::string &value)

class ProgramFlow

Capnp-backed structure encapsulating lists of program indexes to be run during the load, main and save phases.

Unnamed Group

using ProgramIndexType = std::uint32_t

std::vector<ProgramIndexType> &load()

std::vector<ProgramIndexType> &main()

std::vector<ProgramIndexType> &save()

const std::vector<uint32_t> &load() const

void setLoad(const std::vector<uint32_t> &value)

const std::vector<uint32_t> &main() const

void setMain(const std::vector<uint32_t> &value)

const std::vector<uint32_t> &save() const

void setSave(const std::vector<uint32_t> &value)

class TensorInfo

Capnp-backed structure encapsulating the shape and data type of a tensor.

Note that you can create several types of tensors:

matrices: shape: [2, 3, 4], dt: F32 (Please be aware that [1, 1, 1] is a matrix)
vectors: shape: [20], dt: S32 (Please be aware [1] is a vector)
scalars: shape: [], dt: U32 ([] means that shape is empty)

Unnamed Group

using ShapeDimType = std::int64_t

bool operator==(const TensorInfo &rhs) const

bool operator!=(const TensorInfo &rhs) const

std::vector<int64_t> &shape()

int64_t sizeInBytes() const

const std::vector<int64_t> &shape() const

void setShape(const std::vector<int64_t> &value)

DataType dataType() const

void setDataType(DataType value)

class TensorDataInfo

Capnp-backed structure encapsulating metadata about a tensor.

Unnamed Group

bool operator==(const TensorDataInfo &rhs) const

bool operator!=(const TensorDataInfo &rhs) const

TensorInfo &tensorInfo()

const std::string &name() const

void setName(const std::string &value)

const TensorInfo &tensorInfo() const

void setTensorInfo(const TensorInfo &value)

class FeedDataInfo

Capnp-backed structure encapsulating metadata about a data feed.

Unnamed Group

using NumTensorsType = std::uint32_t

FeedDataInfo()

bool operator==(const FeedDataInfo &rhs) const

bool operator!=(const FeedDataInfo &rhs) const

TensorInfo &tensorInfo()

const std::string &name() const

void setName(const std::string &value)

const TensorInfo &tensorInfo() const

void setTensorInfo(const TensorInfo &value)

uint32_t numTensors() const

void setNumTensors(uint32_t value)

class Anchor

Capnp-backed structure mapping input/output data to a named program entry/exit point.

Unnamed Group

using ProgramsType = std::uint32_t

Anchor()

bool operator==(const Anchor &rhs) const

bool operator!=(const Anchor &rhs) const

TensorInfo &tensorInfo()

const std::string &name() const

void setName(const std::string &value)

const std::vector<uint32_t> &programs() const

void setPrograms(const std::vector<uint32_t> &value)

const std::string &handle() const

void setHandle(const std::string &value)

const TensorInfo &tensorInfo() const

void setTensorInfo(const TensorInfo &value)

TensorType type() const

void setType(TensorType value)

bool isPerReplica() const

void setIsPerReplica(bool value)

bool useRemoteBuffers() const

void setUseRemoteBuffers(bool value)

uint32_t repeats() const

void setRepeats(uint32_t value)

class Metadata

Capnp-backed class encapsulating all metadata for a specific executable.

Unnamed Group

using ReplicationFactorType = std::uint32_t

using NumIpusType = std::uint32_t

using NumProcessesType = std::uint32_t

using IpuVersionType = std::uint32_t

std::string version: The version string applied to this metadata.

Metadata()

std::vector<Anchor> &anchors()

std::vector<Option> &engineOptions()

std::vector<Option> &deviceOptions()

ProgramFlow &programFlow()

uint32_t replicationFactor() const

void setReplicationFactor(uint32_t value)

uint32_t numIpus() const

void setNumIpus(uint32_t value)

const std::string &seedHandle() const

void setSeedHandle(const std::string &value)

const std::vector<Anchor> &anchors() const

void setAnchors(const std::vector<Anchor> &value)

const std::string &executable() const

void setExecutable(const std::string &value)

uint32_t numProcesses() const

void setNumProcesses(uint32_t value)

uint32_t ipuVersion() const

void setIpuVersion(uint32_t value)

bool isPOD() const

void setIsPOD(bool value)

bool isInference() const

void setIsInference(bool value)

const std::vector<Option> &engineOptions() const

void setEngineOptions(const std::vector<Option> &value)

const std::vector<Option> &deviceOptions() const

void setDeviceOptions(const std::vector<Option> &value)

const ProgramFlow &programFlow() const

void setProgramFlow(const ProgramFlow &value)

const std::unordered_map<uint32_t, std::string> &programsMap() const

void setProgramsMap(const std::unordered_map<uint32_t, std::string> &value)

uint32_t deviceIterations() const

void setDeviceIterations(uint32_t value)

uint32_t deviceIteration() const

void setDeviceIteration(uint32_t value)

5.2. Classes for writing PopEF streams

enum class popef::FileWriter::Mode

Specifies available file writing modes.

Values:

enumerator OVERWRITE: Allows to overwrite an existing file or create a new one.

enumerator APPEND: Allows to add additional blobs to an existing popef file.

class ExclusiveWriter

Abstract base class for blob writers.

Only one ExclusiveWriter can be active at a time.

Subclassed by popef::BlobWriter, popef::MultiFeedWriter

Public Functions

virtual ~ExclusiveWriter()

virtual void close() = 0

Close the writer, ready for the next one to be created.

Should leave the stream pointing to the end of the blob, so that the next one can be written.

class BlobWriter : public popef::ExclusiveWriter 

Object to write the data associated to a blob.

Write the data associated to the object just created using BlobWriter::write().

Attention: You should not instantiate this class directly, it should always be created by a Writer.

Public Functions

explicit BlobWriter(std::unique_ptr<BlobWriterImpl> impl)

Create a BlobWriter using the given implementation.

Attention: You should not instantiate this class directly, it should always be created by a Writer.

~BlobWriter() override

virtual void close() override: Close the BlobWriter.

Public Members

std::ostream stream: Writing data to a blob is possible through the ostream object.

class MultiFeedWriter : public popef::ExclusiveWriter 

Object to write to multiple feeds simultaneously, allowing interleaved writes.

Call MultiFeedWriter::addFeed() to add a feed, then write to it using MultiFeedWriter::write(), passing the zero-based index of the feed. You can interleave writes to different feeds in any way you please.

Public Functions

MultiFeedWriter(std::shared_ptr<OutputStream> stream, const std::string &version)

Construct a MultiFeedWriter.

Parameters

stream – [in] output stream to write to.
version – [in] version string for the feed blobs.

~MultiFeedWriter() override

void addFeed(const FeedDataInfo &fdi)

Add a new feed.

Warning

Note that the FeedDataInfo structure passed to addFeed must contain a valid (>0) number of tensors.

Parameters: fdi – The FeedDataInfo structure which applies to the feed to be added.

void write(size_t feed, const char *data, size_t size)

Write data to a feed.

Parameters

feed – The zero-based index of the feed to write to.
data – The data buffer.
size – The number of bytes to write.

void write(const std::string &feed, const char *data, size_t size)

Write data to a feed.

Parameters

feed – The feed name to write to.
data – The data buffer.
size – The number of bytes to write.

virtual void close() override: Close the MultiFeedWriter.

class Writer

Write blobs to a given stream.

A writer is a wrapper around a stream which is used to serialise PopEF blobs to the underlying stream.

The exception to this is the MultiFeedWriter, which can handle writing to multiple feeds in an interleaved fashion. See MultiFeedWriter for details.

Warning

Because the size of most blobs cannot be predicted in advance, only one blob writer can be active at any time. The creation of a new blob writer implicitly closes the previous one.

Subclassed by popef::FileWriter

Public Functions

explicit Writer(std::ostream &out)

Instantiate a writer which will output blobs to the given stream.

Parameters: out – [out] Destination stream.

void setVersion(const std::string &version)

Parameters: version – [in] Version string that will be used in all the future headers created by this writer.

std::shared_ptr<BlobWriter> createExecutable(const std::string &name, bool compress = false)

Append a PoplarExecutable blob to the stream.

Parameters

name – [in] Name of the executable.
compress – [in] Automatically compress the executable as it is being written.

Returns

The blob writer to use to serialise the binary.

std::shared_ptr<BlobWriter> createOpaqueBlob(const std::string &name, const std::string &executable)

Append an Opaque blob to the stream.

Parameters

name – [in] Name of the opaque blob. (Usually the name of the framework owning it.)
executable – [in] Name of the executable the blob is associated with.

Returns

The blob writer to use to serialise the data.

std::shared_ptr<BlobWriter> createFeedData(const FeedDataInfo &obj)

Append a single feed to the stream.

If you want to add multiple feeds and write to them in an interleaved fashion, see createMultiFeedData().

Note

If the number of tensors in obj is set to 0, then the writer will update the number of tensors in the structure when it is written to the stream.

Parameters: obj – [in] Information about the feed
Returns: The feed writer to use to serialise the tensors. All the tensors must be written contiguously and without gaps. The number of bytes written must match exactly the data provided in the feed’s information.

std::shared_ptr<MultiFeedWriter> createMultiFeedData()

Append multiple, interleaved feeds to the stream.

Use this method if you want to write to multiple feeds in an interleaved fashion. If you just want to write to a single feed, use createFeedData(). If using createMultiFeedData(), you must know the number of tensors in each feed in advance.

Returns: The writer used to create and write to a group of feeds.

std::shared_ptr<BlobWriter> createTensorData(const TensorDataInfo &obj)

Append a tensor to the stream.

Parameters: obj – [in] Information about the tensor.
Returns: The tensor writer to use to serialise the tensor. The number of bytes written must match exactly the data provided in the feed’s information.

void write(const Metadata &obj)

Append a metadata blob to the stream.

Parameters: obj – [in] Metadata object to serialise.

void close()

Finalise the writer.

After this call the stream that was passed to the constructor will no longer be used and an exception will be thrown if any object created by the Writer tries to access the stream.

virtual ~Writer()

Protected Functions

Writer(): Allows subclasses to create streams on construction before initialising the Writer using initialiseImpl().

void initialiseImpl(std::ostream &out): Allows subclasses to create streams to initialise the Writer after construction, allowing the creation of streams at construction time.

class FileWriter : public popef::Writer 

Write blobs to a given file.

It creates a file if it does not exist. If the file exists, it allows to append blobs to it or override existing content.

A writer is a wrapper around a stream which is used to serialise PopEF blobs to the underlying stream.

The exception to this is the MultiFeedWriter, which can handle writing to multiple feeds in an interleaved fashion. See MultiFeedWriter for details.

Warning

Because the size of most blobs cannot be predicted in advance, only one blob writer can be active at any time. The creation of a new blob writer implicitly closes the previous one.

Public Functions

explicit FileWriter(const std::string &filename, Mode mode = Mode::OVERWRITE): Instantiate a Writer which will write to a file, and will manage the lifetime of the stream correctly with respect to the lifetime of the underlying Writer.

5.3. Classes for reading PopEF streams

class SubReader

Base class for subreaders (classes that read specific blobs from a stream).

Attention: You should not create SubReaders; they are created by a Reader.

Subclassed by popef::ExecutableReader, popef::FeedReader, popef::OpaqueReader, popef::TensorReader

Unnamed Group

SubReader(const SubReader &src)

SubReader(SubReader &&src)

~SubReader()

SubReader &operator=(const SubReader &src)

SubReader &operator=(SubReader &&src)

Public Functions

SubReader(std::string version_in, std::unique_ptr<InputStream> stream_in)

Create a SubReader.

Attention: SubReaders, including ExecutableReader, should not be created by users; they are created by a Reader.

Parameters

version_in – [in] The version string for this blob.
stream_in – [in] The wrapped stream to read the blob from.

std::streamsize getAvailableReadSize() const: Returns the number of bytes available for reading from the current position to the end of the sub-stream (blob).

Public Members

std::string version: The version string for this blob.

Protected Attributes

std::unique_ptr<InputStream> stream: InputStream to read blob data from.

class TensorReader : public popef::SubReader 

Loads a tensor’s data from a stream.

Attention: SubReaders, including TensorReader, should not be created by users; they are created by a Reader.

Unnamed Group

TensorReader(const TensorReader &src)

TensorReader(TensorReader &&src)

TensorReader &operator=(const TensorReader &src)

TensorReader &operator=(TensorReader &&src)

Public Functions

TensorReader(std::string tnsr_version, std::unique_ptr<InputStream> stream)

Create a TensorReader.

Attention: SubReaders, including TensorReader, should not be created by users; they are created by a Reader.

Parameters

tnsr_version – [in] The version string for this tensor.
stream – [in] The wrapped stream to read the tensor from.

std::unique_ptr<std::istream> getStandaloneDataStream() const: Returns a std::istream unique_ptr which reads the tensor data (just as TensorReader::data does), but maintains the liveness chain independently of the existence of the parent TensorReader.

Public Members

TensorDataInfo info: Metadata of the serialised tensor.

std::istream data: Stream pointing to the tensor’s data.

class FeedReader : public popef::SubReader 

Loads a feed’s data from a stream.

Attention: SubReaders, including FeedReader, should not be created by users; they are created by a Reader.

Unnamed Group

FeedReader(const FeedReader &src)

FeedReader(FeedReader &&src)

FeedReader &operator=(const FeedReader &src)

FeedReader &operator=(FeedReader &&src)

Public Functions

FeedReader(std::string feed_version, std::unique_ptr<InputStream> stream)

Create a FeedReader.

Attention: SubReaders, including FeedReader, should not be created by users; they are created by a Reader.

Parameters

feed_version – [in] The version string for this data feed.
stream – [in] The wrapped stream to read the data from.

std::unique_ptr<std::istream> getStandaloneDataStream() const: Returns a std::istream unique_ptr which reads the feed data (just as FeedReader::data does), but maintains the liveness chain independently of the existence of the parent FeedReader.

Public Members

FeedDataInfo info: Metadata of the serialised feed.

std::istream data: Stream pointing to the feed’s data.

class OpaqueReader : public popef::SubReader 

Loads an opaque blob from a stream.

Attention: SubReaders, including OpaqueReader, should not be created by users; they are created by a Reader.

Unnamed Group

OpaqueReader(const OpaqueReader &src)

OpaqueReader(OpaqueReader &&src)

OpaqueReader &operator=(const OpaqueReader &src)

OpaqueReader &operator=(OpaqueReader &&src)

Public Functions

OpaqueReader(std::string opq_version, std::unique_ptr<InputStream> stream)

Create an OpaqueReader.

Attention: SubReaders, including OpaqueReader, should not be created by users; they are created by a Reader.

Parameters

opq_version – [in] The version string for this opaque blob.
stream – [in] The wrapped stream to read the blob from.

std::unique_ptr<std::istream> getStandaloneDataStream() const: Returns a std::istream unique_ptr which reads the blob data (just as OpaqueReader::data does), but maintains the liveness chain independently of the existence of the parent OpaqueReader.

Public Members

std::string name: Name of the opaque blob (usually the name of the framework which created it).

std::string executable: Name of the executable the opaque blob is related to.

std::istream data: Stream pointing to the blob’s data.

class ExecutableReader : public popef::SubReader 

Loads a Poplar executable from a stream.

Attention: SubReaders, including ExecutableReader, should not be created by users; they are created by a Reader.

Unnamed Group

ExecutableReader(const ExecutableReader &src)

ExecutableReader(ExecutableReader &&src)

ExecutableReader &operator=(const ExecutableReader &src)

ExecutableReader &operator=(ExecutableReader &&src)

Public Functions

ExecutableReader(std::string exe_version, std::unique_ptr<InputStream> stream)

Create an ExecutableReader.

Attention: SubReaders, including ExecutableReader, should not be created by users; they are created by a Reader.

Parameters

exe_version – [in] The version string for this executable.
stream – [in] The wrapped stream to read the executable from.

std::unique_ptr<std::istream> getStandaloneExecutableStream() const: Returns a std::istream unique_ptr which reads the serialized executable (just as ExecutableReader::executable does), but maintains the liveness chain independently of the existence of the parent ExecutableReader.

Public Members

std::string name: The name of the executable.

bool is_compressed

If true the executable is stored compressed.

It will be decompressed on the fly as it’s being read so it doesn’t require any special handling.

Important: Seeking through a compressed executable is expensive.

std::istream executable: Stream pointing to the serialised Poplar executable.

class Reader

Reads blobs from a given stream.

A reader build a list of PopEF blobs from one or more streams.

Note

The reader only parses the metadata from each blob into memory. Binary blobs are kept as input streams, ready to be read.

Public Functions

void parseStream(const std::shared_ptr<std::istream> &in, std::streamoff size = 0)

Load PopEF blobs from the given stream.

Parameters

in – [inout] The stream to parse.
size – [in] How many bytes to read from the stream. If 0, parse until the end of the stream is encountered.

void parseFile(const std::string &filename): Helper function to parse a file.

const std::vector<ExecutableReader> &executables(): Combined list of executables.

const std::vector<OpaqueReader> &opaqueBlobs(): Combined list of opaque blobs.

const std::vector<TensorReader> &tensors(): Combined list of tensors.

const std::vector<FeedReader> &feeds(): Combined list of feeds.

const std::vector<Metadata> &metadata(): Combined list of metadata.

5.4. Model representation classes

enum class popef::FilterStrategy

Strategy to use to filter objects of the same type.

Values:

enumerator USE_LAST: If there are several objects of a given type, use the last one.

enumerator ASSERT_ONE: Raise an error if there is more than one object of a given type.

enum class popef::UnknownDataStrategy

Strategy to use when encountering data sources for tensors which don’t exist in the metadata.

Values:

enumerator KEEP_ALL: Keep the readers even though the data is not used by this executable.

enumerator REMOVE_EXTRA: Remove data which isn’t used by this executable.

enumerator ASSERT_ON_EXTRA: Raise an error if the reader contains data not used by the executable.

enum class popef::VersionCheck

Strategy to use when validating the headers’ version field.

Values:

enumerator NO_CHECK: Don’t perform any validation on versions.

enumerator EXACT_MATCH: All the versions must be identical.

class Model

A Model is a combination of blobs representing a single model.

A model is defined by an executable and its associated metadata and data.

Models can be created by hand or by a ModelBuilder

Public Functions

explicit Model(ExecutableReader exec, Metadata meta)

Create a Model object.

Parameters

exec – [in] The executable blob reader.
meta – [in] The associated metadata.

void assertAllInputsProvided() const: Throw an exception if all the input tensors don’t have either a TensorReader or FeedReader associated with them.

Public Members

const Metadata metadata: The metadata associated with the executable used by this model.

const ExecutableReader executable: The executable used by this model.

std::map<std::string, const FeedReader> input_feeds: Map feed names to their data source.

std::map<std::string, const TensorReader> input_tensors: Map tensor names to their data source.

class ModelBuilder

Filters the list of blobs of a Reader to obtain a Model.

The builder is a state machine. Set all the filters to configure it and call createModel() to filter the blobs and instantiate the model.

Public Functions

explicit ModelBuilder(std::shared_ptr<Reader> reader)

Create a ModelBuilder object.

Parameters: reader – [in] The Reader allows obtaining blobs for creating models.

std::vector<std::string> listExecutables() const: List the names of the executables contained in the reader.

void useExecutable(const std::string &name)

Set the name of the executable to use.

Note

The executable filter will be used if several executables have the same name.

Parameters: name – [in] If name is empty then all the executables will be considered a match and the executable filter will be used to determine which one to use.

void setExecutableFilter(FilterStrategy filter): In case several executables have the same name.

void setMetadataFilter(FilterStrategy filter): After only the metadata matching the selected executable are kept, this filter will be used to determine which one to use.

void setDataFilter(FilterStrategy filter): Filter to use for data objects (tensors / infeeds).

void setUnknownDataStrategy(UnknownDataStrategy filter): Strategy to use when encountering data sources for tensors which don’t exist in the metadata.

void setVersionCheck(VersionCheck check): Strategy to use to validate versions.

std::unique_ptr<Model> createModel(): Apply the filters and instantiate a model.

5.5. Error classes

class Error : public runtime_error

Exception class for PopEF.

Subclassed by popef::logging::InternalError

Public Functions

explicit Error(const char *s)

Creates an error with the specified message.

Parameters: s – [in] Error message.

class InternalError : public popef::logging::Error 

Exception class specific to internal errors.

This should be used as an assert, for states which should be unreachable.

Public Functions

explicit Error(const char *s)

Creates an error with the specified message.

Parameters: s – [in] Error message.

Search help

5. PopEF C++ API reference

5.1. Metadata

5.2. Classes for writing PopEF streams

5.3. Classes for reading PopEF streams

5.4. Model representation classes

5.5. Error classes