DynamicSlice

#include <popops/DynamicSlice.hpp>

Support for dynamic slices.

namespace poplar: Poplar classes and functions.

namespace popops

Common functions, such as elementwise and reductions.

Functions

bool operator<(const SlicePlan &a, const SlicePlan &b) noexcept

bool operator==(const SlicePlan &a, const SlicePlan &b) noexcept

bool operator!=(const SlicePlan &a, const SlicePlan &b) noexcept

poplar::Tensor createSliceableTensor(poplar::Graph &graph, const poplar::Type &type, const std::vector<size_t> &shape, const std::vector<size_t> &dims, const std::vector<size_t> &sizes, std::size_t minGrainSize = 0, const poplar::DebugContext &debugContext = {})

Create and map a tensor to be sliced/updated efficiently.

The returned tensor will be spread over as many tiles as possible while respecting the minimum number of elements per tile (minGrainSize) and still being in a form that can be sliced/updated efficiently.

Parameters

graph – The Poplar graph.
type – The type of the elements.
shape – The shape of the tensor to be slice/updated.
dims – The dimensions of the tensor that will be slice/updated.
sizes – The size of the slice in each of the dimensions.
minGrainSize – The minimum elements per slice mapped to each tile
debugContext – Optional debug information.

Returns

A tensor shape shape that is suitably mapped

poplar::Tensor createSliceableTensor(poplar::Graph &graph, const poplar::Type &type, const std::vector<size_t> &shape, const std::vector<size_t> &dims, const std::vector<size_t> &sizes, const SlicePlan &plan, const poplar::OptionFlags &options, const poplar::DebugContext &debugContext = {})

Create and map a tensor to be sliced/updated efficiently.

The returned tensor will be laid out according to the plan.

Parameters

graph – The Poplar graph.
type – The type of the elements.
shape – The shape of the tensor to be slice/updated.
dims – The dimensions of the tensor that will be slice/updated.
sizes – The size of the slice in each of the dimensions.
plan – Plan describing how the slicing/updating operation will be implemented.
options – Flags controlling how the operation will be implemented.
debugContext – Optional debug information.

Returns

A tensor shape shape that is suitably mapped.

poplar::Tensor createGroupedSliceableTensor(poplar::Graph &graph, const poplar::Type &type, const std::size_t groupSize, const std::vector<std::size_t> &shape, const std::vector<std::size_t> &dims, const std::vector<std::size_t> &sizes, const SlicePlan &plan, const poplar::OptionFlags &options, const poplar::DebugContext &debugContext = {})

Create and map a tensor with a group dimension to be sliced/updated efficiently.

Groups allow multiple independent slice/update operations of the same size to be done in parallel.

The shape, dims, and sizes parameters are defined as in createSliceableTensor() and are used for every group.

A valid plan must be provided. The returned tensor will be laid out according to the plan.

Parameters

graph – The Poplar graph.
type – The type of the elements.
groupSize – The group size.
shape – The shape of the tensor to be slice/updated.
dims – The dimensions of the tensor that will be slice/updated.
sizes – The size of the slice in each of the dimensions.
plan – Plan describing how the slicing/updating operation will be implemented.
options – Flags controlling how the operation will be implemented.
debugContext – Optional debug information.

Returns

A tensor suitably mapped with groupSize as the first dimension and the remaining dimensions as in shape.

poplar::Tensor createSliceTensor(poplar::Graph &graph, const poplar::Tensor &t, const std::vector<size_t> &dims, const std::vector<size_t> &sizes, std::size_t numIndices, const poplar::DebugContext &debugContext = {})

Create and map a tensor that is used as a result of slicing, or as an input to an update.

Introspection on the tensor t is used to lay out the created tensor such that it can be used to efficiently update t.

Parameters

graph – The Poplar graph.
t – The tensor to be updated.
dims – The dimensions of the tensor that will be sliced/updated.
sizes – The number of elements of each dimension in dims that will be sliced/updated.
numIndices – The number of slices this tensor should contain.
plan – Plan describing how the slicing/updating operation will be implemented.
options – Flags controlling how the operation will be implemented.
debugContext – Optional debug information.

Returns

A tensor with shape [numIndices, shape…] mapped appropriately to be sliced into/updated from.

poplar::Tensor createSliceTensor(poplar::Graph &graph, const poplar::Type &type, const std::vector<std::size_t> &shape, const std::vector<std::size_t> &dims, const std::vector<std::size_t> &sizes, std::size_t numIndices, const SlicePlan &plan, const poplar::OptionFlags &options, const poplar::DebugContext &debugContext = {})

Create and map a tensor that is used as a result of slicing, or as an input to an update.

The returned tensor is laid out according to the plan for the slice/update operation.

Parameters

graph – The Poplar graph.
type – The type of the elements.
shape – The shape of the tensor to be slice/updated.
dims – The dimensions of the tensor that will be sliced/updated.
sizes – The number of elements of each dimension in dims that will be sliced/updated.
numIndices – The number of slices this tensor should contain.
plan – Plan describing how the slicing/updating operation will be implemented.
options – Flags controlling how the operation will be implemented.
debugContext – Optional debug information.

Returns

A tensor with shape [numIndices, shape…]

poplar::Tensor createGroupedSliceTensor(poplar::Graph &graph, const poplar::Type &type, const std::size_t groupSize, const std::vector<std::size_t> &shape, const std::vector<std::size_t> &dims, const std::vector<std::size_t> &sizes, const std::size_t numIndices, const SlicePlan &plan, const poplar::OptionFlags &options, const poplar::DebugContext &debugContext = {})

Create and map a tensor with a group dimension that is used as a result of slicing, or as an input to an update.

The group dimension is the first dimension of the output tensor.

Groups allow multiple independent slice/update operations of the same size to be done in parallel.

The shape, dims, and sizes parameters are defined as in createGroupedSliceTensor and are used for every group.

The returned tensor is laid out according to the plan for the slice/update operation. It is an error to call this function without a valid plan.

Parameters

graph – The Poplar graph.
type – The type of the elements.
groupSize – The group size
shape – The shape of the tensor to be slice/updated.
dims – The dimensions of the tensor that will be sliced/updated.
sizes – The number of elements of each dimension in dims that will be sliced/updated.
numIndices – The number of slices this tensor should contain.
plan – Plan describing how the slicing/updating operation will be implemented.
options – Flags controlling how the operation will be implemented.
debugContext – Optional debug information.

Returns

A tensor with shape [groupSize, numIndices, shape…]

poplar::Tensor createIndicesTensor(poplar::Graph &graph, const std::vector<std::size_t> &dims, std::size_t numIndices, const SlicePlan &plan, const poplar::OptionFlags &options, const poplar::DebugContext &debugContext = {})

Create and map a tensor to contain indices for slicing or updating a tensor efficiently.

Parameters

graph – The Poplar graph.
dims – The dimensions of a tensor to be sliced/updated that will be sliced/updated using these indices.
numIndices – The number of indices this tensor should contain
plan – Plan describing how the slicing/updating operation will be implemented.
options – Flags controlling how the operation will be implemented.
debugContext – Optional debug information.

Returns

A tensor of shape [numIndices, dims.size()] mapped appropriately to be used as the indices for a slice/update operation. Element type is always UNSIGNED_INT.

poplar::Tensor createGroupedIndicesTensor(poplar::Graph &graph, const std::size_t groupSize, const std::vector<std::size_t> &dims, const std::size_t numIndices, const SlicePlan &plan, const poplar::OptionFlags&, const poplar::DebugContext &debugContext = {})

Create and map a tensor with a group dimension to contain indices for slicing or updating a tensor efficiently.

Groups allow multiple independent slice/update operations of the same size to be done in parallel. Indices have the same bounds for every element in the group.

dims is defined as in the non-grouped version. They do not include a group dimension.

It is an error to call this function without a valid plan and groupSize must match the group size with which the operation is planned in plan.

Parameters

graph – The Poplar graph.
groupSize – The size of the group
dims – The dimensions of a tensor to be sliced/updated that will be sliced/updated using these indices.
numIndices – The number of indices this tensor should contain
plan – Plan describing how the slicing/updating operation will be implemented.
options – Flags controlling how the operation will be implemented.
debugContext – Optional debug information.

Returns

A tensor of shape [groupSize, numIndices, dims.size()] mapped appropriately to be used as the indices for a slice/update operation. Element type is always UNSIGNED_INT.

poplar::Tensor createSliceableTensorFromSlice(poplar::Graph &graph, const poplar::Tensor &s, const std::vector<std::size_t> &dims, const std::vector<std::size_t> &numSlices, const poplar::DebugContext &debugContext = {})

Create and map a tensor to be sliced/updated.

The tensor is mapped in a way that can be efficiently sliced and updated to/from the given slice tensor. It will be distributed across as many tiles as the given slice and with the same contiguous regions on each tile. The tensor’s shape and mapping are derived from the reference slice tensor.

Parameters

graph – The Poplar graph.
s – The reference slice.
dims – The dimensions of the returned tensor that will be sliced.
numSlices – The number of independent slices in each sliced dimension.
debugContext – Optional debug information.

Returns

A tensor to be sliced/updated.

poplar::Tensor dynamicSlice(poplar::Graph &graph, const poplar::Tensor &t, const poplar::Tensor &offset, const std::vector<std::size_t> &dims, const std::vector<std::size_t> &sizes, poplar::program::Sequence &prog, const poplar::DebugContext &debugContext = {}, const poplar::OptionFlags &options = {})

Slice a tensor based on offsets specified by a tensor.

dims gives the dimensions to slice, sizes defines the size of the slice in those dimensions and offset gives the base offsets on each execution.

offset[0], dims and sizes must have the same size. offset may have a second dimension with an element per tile, which can eliminate exchange.

** Dynamic slice options **

remapOutOfBoundIndices (true, false) [=false] Out of bounds indices are mapped to index 0.
paddingIndexUsed (true, false) [=false] Padding index equal to the size of the slice dimension of tensor t may be used in the indices. The actual padding values returned for a padding index are zeros.

Note

If the elements of a single slice cannot be well spread across available tiles (for example, because it has fewer elements than the available number of tiles) then the operation as a whole will also have poor tile balance and consequently poor tile memory balance.

Parameters

graph – The Poplar graph.
t – The source tensor.
offset – A tensor of offsets at which the output is extracted.
dims – The dimensions of t to slice.
sizes – The size of the slice in each of the dimensions in dims.
prog – The program to be extended
options – Option flags
debugContext – Optional debug information.

Returns

The specified subtensor

poplar::Tensor createDynamicSliceOutput(poplar::Graph &graph, const poplar::Tensor &t, const poplar::Tensor &offset, const std::vector<std::size_t> &dims, const std::vector<std::size_t> &sizes, const poplar::DebugContext &debugContext = {}, const poplar::OptionFlags &options = {})

Create a tensor for use with dynamicSliceWithOutput.

Parameters, allowed options and the return values are the same as dynamicSlice except there is no prog parameter.

Parameters

graph – The Poplar graph.
t – The source tensor.
offset – A tensor of offsets at which the output is extracted.
dims – The dimensions of t to slice.
sizes – The size of the slice in each of the dimensions in dims.
options – Option flags
debugContext – Optional debug information.

Returns

The specified subtensor

void dynamicSliceWithOutput(poplar::Graph &graph, const poplar::Tensor &output, const poplar::Tensor &t, const poplar::Tensor &offset, const std::vector<std::size_t> &dims, const std::vector<std::size_t> &sizes, poplar::program::Sequence &prog, const poplar::DebugContext &debugContext = {}, const poplar::OptionFlags &options = {})

Slice a tensor based on offsets specified by a tensor.

dims gives the dimensions to slice, sizes defines the size of the slice in those dimensions and offset gives the base offsets on each execution.

offset[0], dims and sizes must have the same size. offset may have a second dimension with an element per tile, which can eliminate exchange.

see dynamicSlice for information on options

Parameters

graph – The Poplar graph.
output – The output tensor, This should ideally be created with createSliceTensor to maximise efficiency,
t – The source tensor.
offset – A tensor of offsets at which the output is extracted.
dims – The dimensions of t to slice.
sizes – The size of the slice in each of the dimensions in dims.
prog – The program to be extended
debugContext – Optional debug information.
debugContext – Optional debug information
options – Option to control remapping of indices and padding

poplar::Graph::TileToTensorMapping getSliceMapping(poplar::Graph &graph, const poplar::Tensor &t, const std::vector<std::size_t> &dims, const std::vector<std::size_t> &sizes)

Get the tile mapping for a slice of a tensor.

dims gives the dimensions to slice, sizes defines the size of the slice in those dimensions.

Parameters

graph – The Poplar graph.
t – The source tensor.
dims – The dimensions of t to slice.
sizes – The size of the slice in each of the dimensions in dims.

void dynamicUpdate(poplar::Graph &graph, const poplar::Tensor &t, const poplar::Tensor &s, const poplar::Tensor &offset, const std::vector<std::size_t> &dims, const std::vector<std::size_t> &sizes, poplar::program::Sequence &prog, const poplar::DebugContext &debugContext = {}, const poplar::OptionFlags &options = {})

Update a subtensor at offsets read from a tensor.

dims gives the dimensions that are partially updated, by sizes elements, at offsets offset. Unspecified dimensions are copied in full with zero offset.

offset[0], dims and sizes must have the same size. offset may have a second dimension with an element per tile, which can eliminate exchange.

see dynamicSlice for information on options

Parameters

graph – The Poplar graph.
t – The tensor to update.
s – The updates.
offset – The offset within t to be updated.
dims – The dimensions to be dynamically updated.
sizes – The size of the update in each of the dimensions in dims.
prog – The program to be extended.
debugContext – Optional debug information.
options – Option flags

poplar::Tensor multiSlice(poplar::Graph &graph, const poplar::Tensor &t, const poplar::Tensor &offsets, const std::vector<std::size_t> &dims, const std::vector<std::size_t> &sizes, poplar::program::Sequence &prog, const SlicePlan &plan, const poplar::OptionFlags &options, const poplar::DebugContext &debugContext = {})

Take multiple slices from a base tensor.

The returned tensor will have a rank one greater than t. Its outer dimension will be offsets.dim(0). Note that dims refers to the dimensions of t. t can be created using createSliceableTensor() to ensure efficient mapping.

dims and sizes must be size 1.

** multiSlice options **

remapOutOfBoundIndices (true, false) [=false] Out of bounds indices are mapped to index 0.
paddingIndexUsed (true, false) [=false] Padding index equal to the size of the slice dimension of tensor t may be used in the indices. The actual padding values returned for a padding index are zeros. Padding index is excluded from index validation if it is enabled.

Parameters

graph – The Poplar graph.
t – The tensor being sliced.
offsets – The offsets within t to be sliced.
dims – The dimensions of t to be sliced.
sizes – The size of the update in each of the dimensions in dims.
prog – The program to be extended.
plan – Plan describing how the operation will be implemented.
options – Flags controlling how the operation will be implemented.
debugContext – Optional debug information.

poplar::Tensor groupedMultiSlice(poplar::Graph &graph, const poplar::Tensor &t, const poplar::Tensor &offsets, const std::vector<std::size_t> &dims, const std::vector<std::size_t> &sizes, poplar::program::Sequence &prog, const SlicePlan &plan, const poplar::OptionFlags &options, const poplar::DebugContext &debugContext = {})

Take multiple slices from a base tensor where the base tensor and the offsets tensor have the group dimension as the first dimension.

The indices given in the offsets tensor should be in the range [0, base.dim(1)]. Indices outside of this range are allowed but return undefined values.

The returned tensor will have a rank one greater than t. Its outer dimension is the group size and the second dimension will be offsets.dim(1). dims indexes the dimensions of each group in t. This makes it consistent with grouped variants of functions to create tensors. t can be created using createGroupedSliceableTensor() to ensure efficient mapping.

dims and sizes must be size 1.

see multiSlice for informtion on options passed to this function in addition to those passed for plan.

Parameters

graph – The Poplar graph.
t – The tensor being sliced.
offsets – The offsets within t to be sliced.
dims – The dimensions of t for each group.
sizes – The size of the update in each of the dimensions in dims.
prog – The program to be extended.
plan – Plan describing how the operation will be implemented.
options – Flags controlling how the operation will be implemented.
debugContext – Optional debug information.

poplar::Tensor multiSlice(poplar::Graph &graph, const poplar::Tensor &t, poplar::ArrayRef<unsigned> offsets, std::size_t dim, poplar::program::Sequence &prog, const poplar::DebugContext &debugContext = {}, const poplar::OptionFlags &optionFlags = {})

Take multiple slices from a base tensor.

The returned tensor will have a rank one greater than t. Its outer dimension will be offsets.size(). Note that dim refers to a dimension of t. Any entry in offset greater than equal to the size of dims in t returns a slice filled with zeros.

see other overloads of multiSlice for information on optionFlags.

Parameters

graph – The Poplar graph.
t – The tensor being sliced.
offsets – The offsets within t to be sliced.
dim – The dimension of t to be sliced.
prog – The program to be extended.
debugContext – Optional debug information.
optionFlags – Option flags.

void multiUpdate(poplar::Graph &graph, const poplar::Tensor &t, const poplar::Tensor &s, const poplar::Tensor &offsets, const std::vector<std::size_t> &dims, const std::vector<std::size_t> &sizes, poplar::program::Sequence &prog, const SlicePlan &plan, const poplar::OptionFlags &options, const poplar::DebugContext &debugContext = {})

Update multiple slices in a tensor.

See overloads of multiSlice for information on options.

dims and sizes must be size 1.

Parameters

graph – The Poplar graph.
t – The tensor being updated.
s – The slices to insert.
offsets – The offsets within t to be updated.
dims – The dimensions of t to be updated.
sizes – The size of the update in each of the dimensions in dims.
prog – The program to be extended.
plan – Plan describing how the operation will be implemented.
options – Flags controlling how the operation will be implemented.
debugContext – Optional debug information.

void groupedMultiUpdate(poplar::Graph &graph, const poplar::Tensor &t, const poplar::Tensor &s, const poplar::Tensor &offsets, const std::vector<std::size_t> &dims, const std::vector<std::size_t> &sizes, poplar::program::Sequence &prog, const SlicePlan &plan, const poplar::OptionFlags &options, const poplar::DebugContext &debugContext = {})

Update multiple slices in a tensor with a group dimension.

The tensors t s , and offsets have group dimension as the first dimension.

dims and sizes must be size 1.

See overloads of multiSlice for information on options.

Parameters

graph – The Poplar graph.
t – The tensor being updated.
s – The slices to insert.
offsets – The offsets within t to be updated.
dims – The dimensions of each group of t to be updated.
sizes – The size of the update in each of the dimensions in dims.
prog – The program to be extended.
plan – Plan describing how the operation will be implemented.
options – Flags controlling how the operation will be implemented.
debugContext – Optional debug information.

void multiUpdateAdd(poplar::Graph &graph, const poplar::Tensor &t, const poplar::Tensor &s, const poplar::Tensor &offsets, const poplar::Tensor &scale, const std::vector<std::size_t> &dims, const std::vector<std::size_t> &sizes, poplar::program::Sequence &prog, const SlicePlan &plan, const poplar::OptionFlags &options, const poplar::DebugContext &debugContext = {})

Accumulate multiple slices in a tensor.

dims and sizes must be size 1.

See overloads of multiSlice for information on options.

Parameters

graph – The Poplar graph.
t – The tensor being updated (must be rank 2).
s – The slices to accumulate.
offsets – The offsets within t to be accumulated.
scale – The scaling to apply to the update. The type of the tensor should be the same as that of t and s except for the case when t and s are of type HALF. In which case scale can be of type FLOAT or HALF.
dims – The dimensions of t to be accumulated (must be rank 1).
sizes – The size of the accumulate in each of the dimensions in dims.
prog – The program to be extended.
plan – Plan describing how the operation will be implemented.
options – Flags controlling how the operation will be implemented.
debugContext – Optional debug information.

void groupedMultiUpdateAdd(poplar::Graph &graph, const poplar::Tensor &t, const poplar::Tensor &s, const poplar::Tensor &offsets, const poplar::Tensor &scale, const std::vector<std::size_t> &dims, const std::vector<std::size_t> &sizes, poplar::program::Sequence &prog, const SlicePlan &plan, const poplar::OptionFlags &options, const poplar::DebugContext &debugContext = {})

Accumulate multiple slices in a tensor where the first dimension of the tensors t, s and offsets is the group dimension.

t, s must be of the same type

dims and sizes must be size 1.

See overloads of multiSlice for information on options.

Parameters

graph – The Poplar graph.
t – The tensor being updated (must be rank 3).
s – The slices to accumulate.
offsets – The offsets within t to be accumulated.
scale – The scaling to apply to the update. The type of the tensor should be the same as that of t and s except for the case when t and s are of type HALF. In which case scale can be of type FLOAT or HALF.
dims – The dimensions of of each group to be accumulated.
sizes – The size of the accumulate in each of the dimensions in dims.
prog – The program to be extended.
plan – Plan describing how the operation will be implemented.
options – Flags controlling how the operation will be implemented.
debugContext – Optional debug information.

void multiUpdateAdd(poplar::Graph &graph, const poplar::Tensor &t, const poplar::Tensor &s, poplar::ArrayRef<unsigned> offsets, const poplar::Tensor &scale, std::size_t dim, poplar::program::Sequence &prog, const poplar::DebugContext &debugContext = {}, const poplar::OptionFlags &optionFlags = {})

Accumulate multiple slices in a tensor.

t, s must be of the same type

See overloads of multiSlice for information on optionsFlags.

Parameters

graph – The Poplar graph.
t – The tensor being updated.
s – The slices to accumulate.
offsets – The offsets within t to be accumulated.
scale – The scaling to apply to the update. The type of the tensor should be the same as that of t and s except for the case when t and s are of type HALF. In which case scale can be of type FLOAT or HALF.
dim – The dimension of t to be accumulated.
prog – The program to be extended.
debugContext – Optional debug information.
optionFlags – Option flags.

void multiUpdateMax(poplar::Graph &graph, const poplar::Tensor &t, const poplar::Tensor &s, const poplar::Tensor &offsets, const std::vector<std::size_t> &dims, const std::vector<std::size_t> &sizes, poplar::program::Sequence &prog, const SlicePlan &plan, const poplar::OptionFlags &options, const poplar::DebugContext &debugContext = {})

Find maximum over multiple slices in a tensor.

t, s must have the same element type offsets[i] >= t.dim(0) are ignored.

dims and sizes must be size 1.

See overloads of multiSlice for information on options.

Parameters

graph – The Poplar graph.
t – The tensor being updated (must be rank 2).
s – The slices to find maximum over.
offsets – The offsets within t to find maximum over.
dims – The dimensions of t to find maximum over (must be rank 1).
sizes – The size of the update in each of the dimensions in dims.
prog – The program to be extended.
plan – Plan describing how the operation will be implemented.
options – Flags controlling how the operation will be implemented.
debugContext – Optional debug information.

void groupedMultiUpdateMax(poplar::Graph &graph, const poplar::Tensor &t, const poplar::Tensor &s, const poplar::Tensor &offsets, const std::vector<std::size_t> &dims, const std::vector<std::size_t> &sizes, poplar::program::Sequence &prog, const SlicePlan &plan, const poplar::OptionFlags &options, const poplar::DebugContext &debugContext = {})

Find maximum over multiple slices in a tensor with a group dimension.

The tensors t, s, offsets have groups as their first dimension. The offsets tensor contains indices per group which update elements of the corresponding group.

dims and sizes must be size 1.

See overloads of multiSlice for information on options.

Parameters

graph – The Poplar graph.
t – The tensor being updated (must be rank 2).
s – The slices to find maximum over.
offsets – The offsets within t to find maximum over.
dims – The dimensions of each group of t to find maximum over (must be rank 1).
sizes – The size of the update in each of the dimensions in dims.
prog – The program to be extended.
plan – Plan describing how the operation will be implemented.
options – Flags controlling how the operation will be implemented.
debugContext – Optional debug information.

void multiUpdateMul(poplar::Graph &graph, const poplar::Tensor &t, const poplar::Tensor &s, const poplar::Tensor &offsets, const std::vector<std::size_t> &dims, const std::vector<std::size_t> &sizes, poplar::program::Sequence &prog, const SlicePlan &plan, const poplar::OptionFlags &options, const poplar::DebugContext &debugContext = {})

Perform tensor product over multiple slices in a tensor.

t, s must have the same element type offsets[i] >= t.dim(0) are ignored.

dims and sizes must be size 1.

See overloads of multiSlice for information on options.

Parameters

graph – The Poplar graph.
t – The tensor being updated (must be rank 2).
s – The slices to perform product over.
offsets – The offsets within t to compute product over.
dims – The dimensions of t to compute product over (must be rank 1).
sizes – The size of the update in each of the dimensions in dims.
prog – The program to be extended.
plan – Plan describing how the operation will be implemented.
options – Flags controlling how the operation will be implemented.
debugContext – Optional debug information.

void groupedMultiUpdateMul(poplar::Graph &graph, const poplar::Tensor &t, const poplar::Tensor &s, const poplar::Tensor &offsets, const std::vector<std::size_t> &dims, const std::vector<std::size_t> &sizes, poplar::program::Sequence &prog, const SlicePlan &plan, const poplar::OptionFlags &options, const poplar::DebugContext &debugContext = {})

Perform tensor product over multiple slices in a tensor with a group dimension.

The tensors t, s, offsets have groups as their first dimension. The offsets tensor contains indices per group which update elements of the corresponding group.

dims and sizes must be size 1.

See overloads of multiSlice for information on options.

Parameters

graph – The Poplar graph.
t – The tensor being updated (must be rank 2).
s – The slices to perform product over.
offsets – The offsets within t to compute product over.
dims – The dimensions of each group of t to compute product over (must be rank 1).
sizes – The size of the update in each of the dimensions in dims.
prog – The program to be extended.
plan – Plan describing how the operation will be implemented.
options – Flags controlling how the operation will be implemented.
debugContext – Optional debug information.

class SlicePlan

#include <DynamicSlice.hpp>

An object representing a plan that describes how to implement a slice or update.

This can be used as a parameter to a function that will slice or update a tensor.

Public Functions

SlicePlan()

~SlicePlan()

SlicePlan(const SlicePlan &other)

SlicePlan(SlicePlan &&other)

SlicePlan &operator=(const SlicePlan &other)

SlicePlan &operator=(SlicePlan &&other)

SlicePlan(std::unique_ptr<SlicePlanInternal> internal)

inline SlicePlanInternal &getImpl() const

Private Members

std::unique_ptr<SlicePlanInternal> internal

Friends

friend std::ostream &operator<<(std::ostream &o, const SlicePlan &p)

friend bool operator<(const SlicePlan &a, const SlicePlan &b) noexcept

friend bool operator==(const SlicePlan &a, const SlicePlan &b) noexcept

friend poplar::ProfileValue toProfileValue(const SlicePlan &p)

namespace embedding

Functions

SlicePlan plan(const poplar::Graph &graph, const poplar::Type &dataType, const std::size_t numEntries, const std::size_t outputSize, const std::vector<std::size_t> &numLookups, const poplar::OptionFlags &options)

Create a plan for implementing a set of operations on an embedding matrix.

** Embedding plan options **

usedForSlice (true, false) [=true]

If true, you intend to use this embedding plan for both a multiSlice operation. An error is thrown if set to false and usedForUpdate is set to false.
usedForUpdate (true, false) [=true]

If true, you intend to use this embedding plan for both a multiUpdate operation. An error is thrown if set to false and usedForSlice is set to false.
operationForUpdate (“none”, “add”, “max”, “mul”) [=”add”]

Only applicable when usedForUpdate = true. Is the type of operation used in multi-update. Set to “none” for multiUpdate “add” for multiUpdateAdd “max” for multiUpdateMax “mul” for multiUpdateMul
availableMemoryProportion Decimal between 0 and 1 (inclusive) [=0.6]

If set, gives the proportion of tile memory made available for temporary variables (variables that become live and die during the operation) for this operation. If not set, the operation has the freedom to use unlimited temporary memory.

See also

createWeights()

See also

Optimising Temporary Memory Usage for Convolutions and Matmuls on the IPU technical note for some practical examples of using availableMemoryProportion
indicesDistribution (uniform, onePoint) [=uniform]

A description of the statistical distribution of the indices that will be sliced/updated over the input size (numEntries) of the operation. This is used to when estimating the runtime of the multiSlice and multiUpdate operation.
- uniform Indices are assumed to be uniformly distributed over the input size of the embedding.
- onePoint Indices are assumed to all be equal.
indicesAreSorted (true, false) [=false]

Plan assuming indices used in MultiUpdate/MultiUpdateOp are sorted in increasing order. The same option must then be used along with the plan when calling MultiUpdate with and without an operation.
validateIndices (true, false) [=false]

Check that all indices are valid at runtime. If any is invalid execution is aborted. Ignored if remapOutOfBoundIndices = true.
partialType (half, float) If not provided, defaults to using the same as the data type. Partials type should always be the same or higher precision than the data type of the embedding matrix. Is applicable only for the case where operationForUpdate is add. It is ignored for all other operations and multi-slice.

Parameters

graph – The graph the operation will be added to.
dataType – The data type of the entries in the embedding matrix and the resulting lookups from the matrix.
numEntries – Input size of embedding matrix.
outputSize – Output size of embedding matrix lookup.
numLookups – Vector of numbers of indices which will be looked up in the embedding matrix.
options – Set of option flags controlling how the operation will be implemented.

Returns

A plan which describes how the embedding matrix lookup/update operations should be implemented.

SlicePlan plan(const poplar::Graph &graph, const poplar::Type &dataType, const std::size_t groupSize, const std::size_t numEntries, const std::size_t outputSize, const std::vector<std::size_t> &numLookups, const poplar::OptionFlags &options): Overload of plan with group size.

std::vector<SlicePlan> planMultiple(const std::vector<SlicePlanningParameters> &spds)

Create multiple plans for implementing a set of operations on an embedding matrix.

The plans are computed in parallel.

Parameters: spds – A vector of parameters to compute slice plans for. For more information about the individual parameters see SlicePlanningParameters and plan.
Returns: A vector of slice plans for each SlicePlanningParameters object passed as input.

struct SlicePlanningParameters

#include <DynamicSlice.hpp>

A complete collection of the parameters required to compute a plan for slice/update operations.

Public Functions

inline SlicePlanningParameters(const poplar::Graph &graph, const poplar::Type &dataType, const std::size_t groupSize, const std::size_t numEntries, const std::size_t outputSize, const std::vector<std::size_t> &numLookups, const poplar::OptionFlags &optionFlags)

inline SlicePlanningParameters(const poplar::Graph &graph, const poplar::Type &dataType, const std::size_t numEntries, const std::size_t outputSize, const std::vector<std::size_t> &numLookups, const poplar::OptionFlags &optionFlags)

Public Members

const poplar::Graph &graph: The graph the operation will be added to.

const poplar::Type dataType: The data type of the entries in the embedding matrix and the resulting lookups from the matrix.

const std::size_t groupSize = 1: The group size.

const std::size_t numEntries: Input size of embedding matrix.

const std::size_t outputSize: Output size of embedding matrix lookup.

const std::vector<std::size_t> numLookups: Vector of numbers of indices which will be looked up in the embedding matrix.

const poplar::OptionFlags optionFlags: Set of option flags controlling how the operation will be implemented.

Search help

DynamicSlice