12. IPU-optimised operations

Several custom versions of operators are provided to target functions available in PopLibs. See the TensorFlow Python API for more details.

12.1. Image operations

Our architecture is well-suited to efficiently handle convolutions over four-channel tensors, however it is common for images to be represented with three channels. In order to obtain better IPU performance, both from a latency and memory standpoint, we advise that when dealing with three-channel inputs, you pad the fourth channel dimension.

See tensorflow.python.ipu.image_ops.normalise_image() for the op that can perform this padding, in addition to normalising and casting if needed. Note that this padding will be performed on-device, after the data has been transferred to the IPU.

An example of its use can be found in the fused_normalise_image() function in the CNN training application example in Graphcore’s examples repository on GitHub.

12.2. Matmul serialisation

You have the option to serialise matrix multiplications along a particular dimension, in order to reduce the code size of the multiplication and the temporary memory requirements of the matmul, at the expense of extra computation.

See tensorflow.python.ipu.math_ops.serialized_matmul() for details of the op.

An example of its use can be found in the mlm_head() function in the BERT application example in Graphcore’s examples repository on GitHub.

12.3. Dropout

The PopLibs version of dropout does not need to store the dropout mask between the forward and backward parts of the graph, saving memory.

See tensorflow.python.ipu.rand_ops.dropout().

12.4. Embedding lookup

This is a version of embedding lookup that has been optimised for the IPU. It allows the embedding lookup to be serialised into smaller lookups, which can reduce the maximum memory at the cost of extra computation when the embedding tensors are used by multiple operations.

See tensorflow.python.ipu.embedding_ops.embedding_lookup().