11. `popart.ir` User Guide (experimental)

Warning

The popart.ir Python module is currently experimental and may be subject to change in future releases in ways that are backwards incompatible without deprecation warnings.

Warning

Due to the experimental nature of popart.ir the documentation provided in this section is incomplete.

As an alternative to using the ONNX builder to create models, popart.ir is an experimental PopART Python module which you can use to create (and, to a limited degree, manipulate) PopART models directly.

PopART models are represented using an intermediate representation (IR). The popart.ir package allows you to manipulate these IRs.

11.1. Concepts

11.1.1. Building blocks

`popart.ir` building blocks — Fig. 11.1 An IR contains a main graph (MG) and multiple other graphs (G). Graphs can contain ops, intermediate tensors (T) and constant tensors (C). The main graph can also contain intermediate, constant and variable tensors (V).

The building blocks of popart.ir are IRs, graphs, tensors and ops. This section summaries these concepts, further information on each topic can be found later in the user guide.

IRs

An IR is an executable program that can be run using a PopART session and a Python process can initialise multiple IRs. An IR contains one main graph, created on IR initialisation, and multiple other subgraphs that you create.

Graphs

`popart.ir` calling a subgraph. — Fig. 11.2 In this example the IR’s main graph (MG) calls subgraph 1 (G1) which in turn calls subgraph 2 (G2). This creates a call tree which is depicted on the right. Op nodes are green, intermediate tensors are red and constants are yellow.

A graphs describes a computational directed acyclic graph (DAG) which contains two types of nodes: tensors and ops. There are two types of graphs: the main graph and subgraphs.

The main graph is a special graph and only one exists per IR. It is the entry point of a IR (like the main function in many programming languages). The main graph can contain intermediate, constant and variable tensors.
Subgraphs have input and output tensors. Subgraphs can be called by other graphs using the call or repeat op. If a subgraph has multiple call sites, the subgraph is outlined during lowering, leading to code reuse and reduced memory usage. A subgraph can only contain intermediate or constant tensors and not variables. Subgraphs have intermediate tensors which are marked as inputs or outputs. When a subgraph is called the inputs must be provided by the calling graph. The input data can be either be passed by reference or value, and this is determined by the user at the call site.

Tensors

Tensors have a shape and datatype, and sometimes initialisation data. A tensor will be produced by an op known as the producer and can have multiple consumer ops. There are three types of tensors: intermediate, variable and constant. Variables and constants are initialised with data.

Constants contain data that cannot change.
Variables contain data that are always live and hence is never freed. Typically model weights are kept on device between runs and are therefore defined as variable tensors.
Intermediates are not initialised with data and are live from the time they are produced until their final consumer.

Ops

An op represents an operation in the computational graph and can have input and output tensors.

11.2. Simple example

To use popart.ir you first need to import it as a Python package:

import popart.ir as pir

Note that we typically import popart.ir as pir for brevity.

As explained previously, the main purpose of this package is creating and manipulating IRs, which are represented by the class pir.Ir. See below for a basic example of how to construct such an object.

import popart.ir as pir
import popart.ir.ops as ops

# Creating a model with popart.ir
ir = pir.Ir()
main = ir.main_graph()
with main:
    # host load
    input0 = pir.h2d_stream([1], pir.float32, name="input0_stream")
    a = ops.host_load(input0, "a")
    input1 = pir.h2d_stream([1], pir.float32, name="input1_stream")
    b = ops.host_load(input1, "b")

    # addition
    o = ops.add(a, b)

    # host store
    o_d2h = pir.d2h_stream(o.shape, o.dtype, name="output_stream")
    ops.host_store(o_d2h, o)

files/simple_addition_popart_ir.py

In popart.ir an IR is essentially a collection of pir.Graph objects. Each such graph contains a number of operations. Each IR has a main graph that is constructed by default. This main graph serves as the entry point for your model. A main graph is obtained via ir.main_graph() in the example above.

By adding operations within a with main context, the operations are automatically added to the main graph. In this example, three operations added: host_load, add and host_store.

In this model we created two device-to-host streams input0 and input1 and one host-to-device stream output. The host_load operations are used to stream data from the host to the device populating tensors a and b, respectively. Another operation, add, then sums these two tensors. Finally, the host_store streams the result data back from the device to the host.

11.3. Data types

Currently, popart.ir supports the data types listed in ir_datatypes_table. These data types are defined in popart.ir directly and will be converted to their IPU-compatible data type. Note that the int64 and uint64 will be downcast to int32 and uint32 respectively if the session option enableSupportedDataTypeCasting is set to True.

Table 11.1 Data types in `popart.ir`
popart.ir dtype	int	floating point	signed	np.dtype	Python dtype	alias
`bool`	False	False	False	np.bool	builtins.bool	N/A
`int8`	True	False	True	np.int8	None	N/A
`int32`	True	False	True	np.int32	None	N/A
`uint8`	True	False	False	np.uint8	None	N/A
`uint32`	True	False	False	np.uint32	None	N/A
`float16`	False	True	True	np.float16	None	`half`
`float32`	False	True	True	np.float32	builtins.float	`float`
`float64`	False	True	True	np.float64	None	`double`

11.4. Tensors

You define a tensor with shape, data type and optional initialisation data. A tensor has zero or more consumer operations and up to one producer operation.

There are three types of tensors in popart.ir:

Constant
Variable
Intermediate

An intermediate tensor is the output of an operation. Variables and constants are initialised with data. For instance, in the example tensor_addition_code, a is a variable tensor, b is a constant tensor, and o is an intermediate tensor.

with main:
    a = pir.variable(3, dtype=pir.int8, name="variable_a")
    b = pir.constant(1, dtype=pir.int8, name="constant_b")

    # addition
    o = a + b

files/tensor_addition_popart_ir.py

11.4.1. Constant

A constant tensor is initialised with data during graph creation. This tensor cannot change during the runtime of a model. You can also use python numeric literals in popart.ir. These literals are implicitly converted to constant tensors. That is, these lines

can also be writen as

o = a + 1

11.4.2. Variable

A variable tensor represents trainable parameters in a model or non-trainable optimizer states. You create and initialize variable tensors in the main graph scope. You can add a variable to the main graph using pir.variable.

To enable flexible interaction, you can read or write variables on the IPU at runtime using the session.readWeights() and session.writeWeights() methods respectively. Therefore, variables are always live in IPU memory and don’t get freed during execution.

Note that, you have to copy the initial value of a variable to IPU device from the host before running the graph by using session.weightsFromHost().

11.4.3. Intermediate

An intermediate tensor is produced by an operation, which means it is not initialised with data. It stays live in IPU memory from when it is produced until the last time it is consumed.

11. popart.ir User Guide (experimental)