DynamicSlice
#include <popops/DynamicSlice.hpp>
Support for dynamic slices.

namespace poplar
Poplar classes and functions.

namespace popops
Common functions, such as elementwise and reductions.
map
Map an expression across tensors.
Elementwise Options
enableGenerateCodelet
(true, false) [=true]When true and the following conditions are met, poplar will generate a codelet to execute the map operation. Otherwise, it will sequence poplibs codelets to create the expression.
All of the inputs are of the same size
Inputs do not alias
Multiple operations are being performed
mapInPlace
Update the input tensors with the result of map().
mapWithOutput
Write the result of map() to the given output tensor.
checkTypes
Check that the host compiletime type
constType
is compatible with the runtime IPU typeelementType
. param elementType
The runtime IPU type.
 param constant
Unused.
 tparam constType
The host compiletime type.
 throws std::runtime_error
If the types are not compatible.
varianceToInvStdDev
Convert variance to inverse standard deviation.
Each element in the output tensor is the result of
1 / sqrt(variance_value + epsilon)
, wherevariance_value
is the corresponding element invariance
.Warning
If
variance_value + epsilon
is zero then the result will be invalid and this operation could generate a dividebyzero floatingpoint exception (if enabled). param graph
The graph to update.
 param variance
A tensor of variance values.
 param epsilon
A (typically small) scalar to add to the variance values, to avoid numerical issues (for example, divide by zero).
 param prog
The sequence of programs to append this conversion operation to.
 param debugContext
Optional debug information.
 return
A tensor where each element is the inverse standard deviation.
invStdDevToVariance
Convert inverse standard deviation to variance.
Each element in the output tensor is the result of
1 / (invStdDev_value + epsilon)^2
, whereinvStdDev_value
is the corresponding element ininvStdDev
.Warning
If
invStdDev_value + epsilon
is zero then the result will be invalid and this operation could generate a dividebyzero floatingpoint exception (if enabled). param graph
The graph to update.
 param invStdDev
A tensor of inverse standard deviation values.
 param epsilon
A (typically small) scalar to add to the variance values, to avoid numerical issues (for example, divide by zero).
 param prog
The sequence of programs to append this conversion operation to.
 param debugContext
Optional debug information.
 param options
A list of flags to pass to the expression evaluator.
 return
A tensor where each element is the variance.
add
Add each element in
A
to the corresponding element inB
. param graph
The graph to update.
 param A
A tensor of elements.
 param B
A tensor of elements.
 param prog
The sequence to extend with the execution of the expression evaluation.
 param debugContext
Optional debug information
 param options
Elementwise options. See map().
 return
A tensor where each element is the result of
a + b
, wherea
andb
are the corresponding elements ofA
andB
tensors respectively.
addInPlace
Update the tensor
A
with the result of add().See add() for parameter descriptions.
addWithOutput
Write the result of add() to the given output tensor,
out
.See add() for the remaining parameter descriptions.
 param out
The tensor to write the results to.
sub
Subtract the elements of
B
fromA
and return the result in a new tensor. param graph
The graph to update.
 param A
The tensor of elements which will be subtracted from.
 param B
The tensor of elements to subtract from
A
. param prog
The sequence to extend with the execution of the expression evaluation.
 param debugContext
Optional debug information.
 param options
Elementwise options. See map().
 return
A tensor where each element is equal to a  b, where
a
andb
are the corresponding elements ofA
andB
tensors respectively.
subInPlace
Update the tensor
A
with the result of sub().See sub() for parameter descriptions.
subWithOutput
Write the result of sub() to the given output tensor,
out
.See sub() for the remaining parameter descriptions.
 param out
The tensor to write the results to.
mul
Multiply each element in
A
by the corresponding element inB
. param graph
The graph to update.
 param A
A tensor of elements.
 param B
A tensor of elements.
 param prog
The sequence to extend with the execution of the expression evaluation.
 param debugContext
Optional debug information.
 param options
Elementwise options. See map().
 return
A tensor where each element is the result of
a * b
, wherea
andb
are the corresponding elements ofA
andB
tensors respectively.
mulInPlace
Update the tensor
A
with the result of mul().See mul() for parameter descriptions.
mulWithOutput
Write the result of mul() to the given output tensor,
out
.See mul() for the remaining parameter descriptions.
 param out
The tensor to write the results to.
div
Divide each element in
A
by the corresponding element inB
. param graph
The graph to update.
 param A
The tensor of dividends.
 param B
The tensor of divisors.
 param prog
The sequence to extend with the execution of the expression evaluation.
 param debugContext
Optional debug information
 param options
Elementwise options. See map().
 return
A tensor where each element is the result of
a / b
, wherea
andb
are the corresponding elements ofA
andB
tensors respectively.
divInPlace
Update the tensor
A
with the result of div().See div() for parameter descriptions.
divWithOutput
Write the result of div() to the given output tensor,
out
.See div() for the remaining parameter descriptions.
 param out
The tensor to write the results to.
pow
Compute each element in
A
to the power of the corresponding element inB
. param graph
The graph to update.
 param A
The tensor of bases.
 param B
The tensor of exponents.
 param prog
The sequence to extend with the execution of the expression evaluation.
 param debugContext
Optional debug information.
 param options
Elementwise options. See map().
 return
A tensor where each element is equal to
pow(a, b)
, wherea
andb
are the corresponding elements ofA
andB
tensors respectively.
powInPlace
Update the tensor
A
with the result of pow().See pow() for parameter descriptions.
powWithOutput
Write the result of pow() to the given output tensor,
out
.See pow() for the remaining parameter descriptions.
 param out
The tensor to write the results to.
rem
Compute the remainder of each element in
A
divided by the corresponding element inB
. param graph
The graph to update.
 param A
The tensor of dividends.
 param B
The tensor of divisors.
 param prog
The sequence to extend with the execution of the expression evaluation.
 param debugContext
Optional debug information.
 param options
Elementwise options. See map().
 return
A tensor where each element is equal to a % b, where
a
andb
are the corresponding elements ofA
andB
tensors respectively.
remInPlace
Update the tensor
A
with the result of rem().See rem() for parameter descriptions.
remWithOutput
Write the result of rem() to the given output tensor,
out
.See rem() for the remaining parameter descriptions.
 param out
The tensor to write the results to.
bitwiseAnd
Compute the bitwise AND of each element in
A
with the corresponding element inB
. param graph
The graph to update.
 param A
A tensor of elements.
 param B
A tensor of elements.
 param prog
The sequence to extend with the execution of the expression evaluation.
 param debugContext
Optional debug information
 param options
Elementwise options. See map().
 return
A tensor where each element is the result of
a & b
, wherea
andbare
the corresponding elements ofA
andB
tensors respectively.
bitwiseAndInPlace
Update the tensor
A
with the result of bitwiseAnd().See bitwiseAnd() for parameter descriptions.
bitwiseAndWithOutput
Write the result of bitwiseAnd() to the given output tensor,
out
.See bitwiseAnd() for the remaining parameter descriptions.
 param out
The tensor to write the results to.
bitwiseOr
Compute the bitwise OR of each element in
A
with the corresponding element inB
. param graph
The graph to update.
 param A
A tensor of elements.
 param B
A tensor of elements.
 param prog
The sequence to extend with the execution of the expression evaluation.
 param debugContext
Optional debug information
 param options
Elementwise options. See map().
 return
A tensor where each element is the result of
a  b
, wherea
andb
are the corresponding elements ofA
andB
tensors respectively.
bitwiseOrInPlace
Update the tensor
A
with the result of bitwiseOr().See bitwiseOr() for parameter descriptions.
bitwiseOrWithOutput
Write the result of bitwiseOr() to the given output tensor,
out
.See bitwiseOr() for the remaining parameter descriptions.
 param out
The tensor to write the results to.
bitwiseXor
Compute the bitwise XOR of each element in
A
with the corresponding element inB
. param graph
The graph to update.
 param A
A tensor of elements.
 param B
A tensor of elements.
 param prog
The sequence to extend with the execution of the expression evaluation.
 param debugContext
Optional debug information
 param options
Elementwise options. See map().
 return
A tensor where each element is the result of
a ^ b
, wherea
andb
are the corresponding elements ofA
andB
tensors respectively.
bitwiseXorInPlace
Update the tensor
A
with the result of bitwiseXor().See bitwiseXnor() for parameter descriptions.
bitwiseXorWithOutput
Write the result of bitwiseXor() to the given output tensor,
out
.See bitwiseXor() for the remaining parameter descriptions.
 param out
The tensor to write the results to.
bitwiseXnor
Compute the bitwise XNOR of each element in
A
with the corresponding element inB
. param graph
The graph to update.
 param A
A tensor of elements.
 param B
A tensor of elements.
 param prog
The sequence to extend with the execution of the expression evaluation.
 param debugContext
Optional debug information
 param options
Elementwise options. See map().
 return
A tensor where each element is the result of
!(a ^ b)
, wherea
andb
are the corresponding elements ofA
andB
tensors respectively.
bitwiseXnorInPlace
Update the tensor
A
with the result of bitwiseXnor().See bitwiseXnor() for parameter descriptions.
bitwiseXnorWithOutput
Write the result of bitwiseXnor() to the given output tensor,
out
.See bitwiseXnor() for the remaining parameter descriptions.
 param out
The tensor to write the results to.
shiftLeft
Shift the elements of
A
left by the corresponding elements ofB
. param graph
The graph to update.
 param A
The tensor of elements which to leftshift.
 param B
The tensor of elements that describe the amount to leftshift
A
by. param prog
The sequence to extend with the execution of the expression evaluation.
 param debugContext
Optional debug information.
 param options
Elementwise options. See map().
 return
A tensor where each element is equal to a << b, where
a
andb
are the corresponding elements ofA
andB
tensors respectively.
shiftLeftInPlace
Update the tensor
A
with the result of shiftLeft().See shiftLeft() for parameter descriptions.
shiftLeftWithOutput
Write the result of shiftLeft() to the given output tensor,
out
.See shiftLeft() for the remaining parameter descriptions.
 param out
The tensor to write the results to.
shiftRight
Shift the elements of
A
right by the corresponding elements ofB
. param graph
The graph to update.
 param A
The tensor of elements which to rightshift.
 param B
The tensor of elements that describe the amount to rightshift by.
A
. param prog
The sequence to extend with the execution of the expression evaluation.
 param debugContext
Optional debug information.
 param options
Elementwise options. See map().
 return
A tensor where each element is equal to a >> b (without sign extension), where
a
andb
are the corresponding elements ofA
andB
tensors respectively.
shiftRightInPlace
Update the tensor
A
with the result of shiftRight().See shiftRight() for parameter descriptions.
shiftRightWithOutput
Write the result of shiftRight() to the given output tensor,
out
.See shiftRight() for the remaining parameter descriptions.
 param out
The tensor to write the results to.
shiftRightSignExtend
Shift the elements of
A
right with sign extension by the corresponding elements ofB
. param graph
The graph to update.
 param A
The tensor of elements which to rightshift.
 param B
The tensor of elements that describe the amount to rightshift
A
by. param prog
The sequence to extend with the execution of the expression evaluation.
 param debugContext
Optional debug information.
 param options
Elementwise options. See map().
 return
A tensor where each element is equal to
a >> b
with sign extension, wherea
andb
are the corresponding elements ofA
andB
tensors respectively.
shiftRightSignExtendInPlace
Update the tensor
A
with the result of shiftRightSignExtend().See shiftRightSignExtend() for parameter descriptions.
shiftRightSignExtendWithOutput
Write the result of shiftRightSignExtend() to the given output tensor,
out
.See shiftRightSignExtend() for the remaining parameter descriptions.
 param out
The tensor to write the results to.
logicalAnd
Compute the logical AND (
&&
) of each element inA
with the corresponding element inB
. param graph
The graph to update.
 param A
A tensor of elements.
 param B
A tensor of elements.
 param prog
The sequence to extend with the execution of the expression evaluation.
 param debugContext
Optional debug information.
 param options
Elementwise options. See map().
 return
A tensor where each element is the result of
a && b
, wherea
andb
are the corresponding elements ofA
andB
tensors respectively.
logicalAndInPlace
Update the tensor
A
with the result of logicalAnd().See logicalAnd() for parameter descriptions.
logicalAndWithOutput
Write the result of logicalAnd() to the given output tensor,
out
.See logicalAnd() for the remaining parameter descriptions.
 param out
The tensor to write the booleans to.
logicalOr
Compute the logical OR (

) of each element inA
with the corresponding element inB
. param graph
The graph to update.
 param A
A tensor of elements.
 param B
A tensor of elements.
 param prog
The sequence to extend with the execution of the expression evaluation.
 param debugContext
Optional debug information.
 param options
Elementwise options. See map().
 return
A tensor where each element is the result of
a  b
, wherea
andb
are the corresponding elements ofA
andB
tensors respectively.
logicalOrInPlace
Update the tensor
A
with the result of logicalOr().See logicalOr() for parameter descriptions.
logicalOrWithOutput
Write the result of logicalOr() to the given output tensor,
out
.See logicalOr() for the remaining parameter descriptions.
 param out
The tensor to write the booleans to.
eq
Check if each element in
A
is equal to the corresponding element inB
. param graph
The graph to update.
 param A
A tensor of elements.
 param B
A tensor of elements.
 param prog
The sequence to extend with the execution of the expression evaluation.
 param debugContext
Optional debug information
 param options
Elementwise options. See map().
 return
A tensor where each element is the result of
a == b
, wherea
andb
are the corresponding elements ofA
andB
tensors respectively.
eqInPlace
Update the tensor
A
with the result of eq().See eq() for parameter descriptions.
eqWithOutput
Write the result of eq() to the given output tensor,
out
.See eq() for the remaining parameter descriptions.
 param out
The tensor to write the booleans to.
neq
Check if each element in
A
is not equal to the corresponding element inB
. param graph
The graph to update.
 param A
A tensor of elements.
 param B
A tensor of elements.
 param prog
The sequence to extend with the execution of the expression evaluation.
 param debugContext
Optional debug information.
 param options
Elementwise options. See map().
 return
A tensor where each element is the result of
a != b
, wherea
andb
are the corresponding elements ofA
andB
tensors respectively.
neqInPlace
Update the tensor
A
with the result of neq().See neq() for parameter descriptions.
neqWithOutput
Write the result of neq() to the given output tensor,
out
.See neq() for the remaining parameter descriptions.
 param out
The tensor to write the booleans to.
gteq
Check if each element in
A
is greater than or equal to the corresponding element inB
. param graph
The graph to update.
 param A
A tensor of elements.
 param B
A tensor of elements.
 param prog
The sequence to extend with the execution of the expression evaluation.
 param debugContext
Optional debug information
 param options
Elementwise options. See map().
 return
A tensor where each element is the result of
a >= b
, wherea
andb
are the corresponding elements ofA
andB
tensors respectively.
gteqInPlace
Update the tensor
A
with the result of gteq().See gteq() for parameter descriptions.
gteqWithOutput
Write the result of gteq() to the given output tensor,
out
.See gteq() for the remaining parameter descriptions.
 param out
The tensor to write the booleans to.
gt
Check if each element in
A
is greater than the corresponding element inB
. param graph
The graph to update.
 param A
A tensor of elements.
 param B
A tensor of elements.
 param prog
The sequence to extend with the execution of the expression evaluation.
 param debugContext
Optional debug information
 param options
Elementwise options. See map().
 return
A tensor where each element is the result of
a > b
, wherea
andb
are the corresponding elements ofA
andB
tensors respectively.
gtInPlace
Update the tensor
A
with the result of gt().See gt() for parameter descriptions.
gtWithOutput
Write the result of gt() to the given output tensor,
out
.See gt() for the remaining parameter descriptions.
 param out
The tensor to write the booleans to.
lteq
Check if each element in
A
is less than or equal to the corresponding element inB
. param graph
The graph to update.
 param A
A tensor of elements.
 param B
A tensor of elements.
 param prog
The sequence to extend with the execution of the expression evaluation.
 param debugContext
Optional debug information.
 param options
Elementwise options. See map().
 return
A tensor where each element is the result of
a <= b
, wherea
andb
are the corresponding elements ofA
andB
tensors respectively.
lteqInPlace
Update the
A
with the result of lteq().See lteq() for parameter descriptions.
lteqWithOutput
Write the result of lteq() to the given output tensor,
out
.See lteq() for the remaining parameter descriptions.
 param out
The tensor to write the booleans to.
lt
Check if each element in
A
is less than the corresponding element inB
. param graph
The graph to update.
 param A
A tensor of elements.
 param B
A tensor of elements.
 param prog
The sequence to extend with the execution of the expression evaluation.
 param debugContext
Optional debug information.
 param options
Elementwise options. See map().
 return
A tensor where each element is the result of
a < b
, wherea
andb
are the corresponding elements ofA
andB
tensors respectively.
ltInPlace
Update the
A
with the result of lt().See lt() for parameter descriptions.
ltWithOutput
Write the result of lt() to the given output tensor,
out
.See lt() for the remaining parameter descriptions.
 param out
The tensor to write the boolean results to.
max
Compute the maximum of each element in
A
with the corresponding element inB
. param graph
The graph to update.
 param A
A tensor of elements.
 param B
A tensor of elements.
 param prog
The sequence to extend with the execution of the expression evaluation.
 param debugContext
Optional debug information.
 param options
Elementwise options. See map().
 return
A tensor where each element is the result of
max(a, b)
, wherea
andb
are the corresponding elements ofA
andB
tensors respectively.
maxWithOutput
Write the result of max() to the given output tensor,
out
.See max() for the remaining parameter descriptions.
 param out
The tensor to write the maximums to.
min
Compute the minimum of each element in
A
with the corresponding element inB
. param graph
The graph to update.
 param A
A tensor of elements.
 param B
A tensor of elements.
 param prog
The sequence to extend with the execution of the expression evaluation.
 param debugContext
Optional debug information.
 param options
Elementwise options. See map().
 return
A tensor where each element is the result of
min(a, b)
, wherea
andb
are the corresponding elements ofA
andB
tensors respectively.
minWithOutput
Write the result of min() to the given output tensor,
out
.See min() for the remaining parameter descriptions.
 param out
The tensor to write the minimums to.
atan2
Compute the elementwise arctangent of
A / B
. param graph
The graph to update.
 param A
A tensor of elements.
 param B
A tensor of elements.
 param prog
The sequence to extend with the execution of the expression evaluation.
 param debugContext
Optional debug information
 param options
Elementwise options. See map().
 return
A tensor where each element is the result of
arctan(a / b)
;a
andb
are the corresponding elements ofA
andB
tensors respectively.
atan2InPlace
Update the tensor
A
with the result of atan2().See atan2() for parameter descriptions.
atan2WithOutput
Write the result of atan2() to the given output tensor,
out
.See atan2() for the remaining parameter descriptions.
 param out
The tensor to write the result to.
invStdDevToVarianceInPlace
Update the
invStdDev
tensor with the result of invStdDevToVariance().See invStdDevToVariance() for parameter descriptions.
invStdDevToVarianceWithOutput
Write the result of invStdDevToVariance() to the given output tensor,
out
.See invStdDevToVariance() for the remaining parameter descriptions.
 param out
The tensor to write the variance to.
varianceToInvStdDevInPlace
Update the
variance
tensor with the result of varianceToInvStdDev().See varianceToInvStdDev() for parameter descriptions.
varianceToInvStdDevWithOutput
Write the result of varianceToInvStdDev() to the given output tensor,
out
.See varianceToInvStdDev() for the remaining parameter descriptions.
 param out
The tensor to write inverse standard deviation to.
Functions

poplar::Tensor createSliceableTensor(poplar::Graph &graph, const poplar::Type &type, const std::vector<size_t> &shape, const std::vector<size_t> &dims, const std::vector<size_t> &sizes, std::size_t minGrainSize = 0, const poplar::DebugContext &debugContext = {})
Create and map a tensor to be sliced/updated efficiently.
The returned tensor will be spread over as many tiles as possible while respecting the minimum number of elements per tile (
minGrainSize
) and still being in a form that can be sliced/updated efficiently. Parameters
graph – The Poplar graph.
type – The type of the elements.
shape – The shape of the tensor to be slice/updated.
dims – The dimensions of the tensor that will be slice/updated.
sizes – The size of the slice in each of the dimensions.
minGrainSize – The minimum elements per slice mapped to each tile
debugContext – Optional debug information.
 Returns
A tensor shape
shape
that is suitably mapped

poplar::Tensor createSliceableTensor(poplar::Graph &graph, const poplar::Type &type, const std::vector<size_t> &shape, const std::vector<size_t> &dims, const std::vector<size_t> &sizes, const SlicePlan &plan, const poplar::OptionFlags &options, const poplar::DebugContext &debugContext = {})
Create and map a tensor to be sliced/updated efficiently.
The returned tensor will be laid out according to the plan.
 Parameters
graph – The Poplar graph.
type – The type of the elements.
shape – The shape of the tensor to be slice/updated.
dims – The dimensions of the tensor that will be slice/updated.
sizes – The size of the slice in each of the dimensions.
plan – Plan describing how the slicing/updating operation will be implemented.
options – Flags controlling how the operation will be implemented.
debugContext – Optional debug information.
 Returns
A tensor shape
shape
that is suitably mapped.

poplar::Tensor createGroupedSliceableTensor(poplar::Graph &graph, const poplar::Type &type, const std::size_t groupSize, const std::vector<std::size_t> &shape, const std::vector<std::size_t> &dims, const std::vector<std::size_t> &sizes, const SlicePlan &plan, const poplar::OptionFlags &options, const poplar::DebugContext &debugContext = {})
Create and map a tensor with a group dimension to be sliced/updated efficiently.
Groups allow multiple independent slice/update operations of the same size to be done in parallel.
The
shape
,dims
, andsizes
parameters are defined as increateSliceableTensor()
and are used for every group.A valid plan must be provided. The returned tensor will be laid out according to the plan.
 Parameters
graph – The Poplar graph.
type – The type of the elements.
groupSize – The group size.
shape – The shape of the tensor to be slice/updated.
dims – The dimensions of the tensor that will be slice/updated.
sizes – The size of the slice in each of the dimensions.
plan – Plan describing how the slicing/updating operation will be implemented.
options – Flags controlling how the operation will be implemented.
debugContext – Optional debug information.
 Returns
A tensor suitably mapped with
groupSize
as the first dimension and the remaining dimensions as inshape
.

poplar::Tensor createSliceTensor(poplar::Graph &graph, const poplar::Tensor &t, const std::vector<size_t> &dims, const std::vector<size_t> &sizes, std::size_t numIndices, const poplar::DebugContext &debugContext = {})
Create and map a tensor that is used as a result of slicing, or as an input to an update.
Introspection on the tensor
t
is used to lay out the created tensor such that it can be used to efficiently updatet
. Parameters
graph – The Poplar graph.
t – The tensor to be updated.
dims – The dimensions of the tensor that will be sliced/updated.
sizes – The number of elements of each dimension in
dims
that will be sliced/updated.numIndices – The number of slices this tensor should contain.
plan – Plan describing how the slicing/updating operation will be implemented.
options – Flags controlling how the operation will be implemented.
debugContext – Optional debug information.
 Returns
A tensor with shape [numIndices, shape…] mapped appropriately to be sliced into/updated from.

poplar::Tensor createSliceTensor(poplar::Graph &graph, const poplar::Type &type, const std::vector<std::size_t> &shape, const std::vector<std::size_t> &dims, const std::vector<std::size_t> &sizes, std::size_t numIndices, const SlicePlan &plan, const poplar::OptionFlags &options, const poplar::DebugContext &debugContext = {})
Create and map a tensor that is used as a result of slicing, or as an input to an update.
The returned tensor is laid out according to the plan for the slice/update operation.
 Parameters
graph – The Poplar graph.
type – The type of the elements.
shape – The shape of the tensor to be slice/updated.
dims – The dimensions of the tensor that will be sliced/updated.
sizes – The number of elements of each dimension in
dims
that will be sliced/updated.numIndices – The number of slices this tensor should contain.
plan – Plan describing how the slicing/updating operation will be implemented.
options – Flags controlling how the operation will be implemented.
debugContext – Optional debug information.
 Returns
A tensor with shape [numIndices, shape…]

poplar::Tensor createGroupedSliceTensor(poplar::Graph &graph, const poplar::Type &type, const std::size_t groupSize, const std::vector<std::size_t> &shape, const std::vector<std::size_t> &dims, const std::vector<std::size_t> &sizes, const std::size_t numIndices, const SlicePlan &plan, const poplar::OptionFlags &options, const poplar::DebugContext &debugContext = {})
Create and map a tensor with a group dimension that is used as a result of slicing, or as an input to an update.
The group dimension is the first dimension of the output tensor.
Groups allow multiple independent slice/update operations of the same size to be done in parallel.
The
shape
,dims
, andsizes
parameters are defined as increateGroupedSliceTensor
and are used for every group.The returned tensor is laid out according to the plan for the slice/update operation. It is an error to call this function without a valid plan.
 Parameters
graph – The Poplar graph.
type – The type of the elements.
groupSize – The group size
shape – The shape of the tensor to be slice/updated.
dims – The dimensions of the tensor that will be sliced/updated.
sizes – The number of elements of each dimension in
dims
that will be sliced/updated.numIndices – The number of slices this tensor should contain.
plan – Plan describing how the slicing/updating operation will be implemented.
options – Flags controlling how the operation will be implemented.
debugContext – Optional debug information.
 Returns
A tensor with shape [groupSize, numIndices, shape…]

poplar::Tensor createIndicesTensor(poplar::Graph &graph, const std::vector<std::size_t> &dims, std::size_t numIndices, const SlicePlan &plan, const poplar::OptionFlags &options, const poplar::DebugContext &debugContext = {})
Create and map a tensor to contain indices for slicing or updating a tensor efficiently.
 Parameters
graph – The Poplar graph.
dims – The dimensions of a tensor to be sliced/updated that will be sliced/updated using these indices.
numIndices – The number of indices this tensor should contain
plan – Plan describing how the slicing/updating operation will be implemented.
options – Flags controlling how the operation will be implemented.
debugContext – Optional debug information.
 Returns
A tensor of shape [numIndices, dims.size()] mapped appropriately to be used as the indices for a slice/update operation. Element type is always UNSIGNED_INT.

poplar::Tensor createGroupedIndicesTensor(poplar::Graph &graph, const std::size_t groupSize, const std::vector<std::size_t> &dims, const std::size_t numIndices, const SlicePlan &plan, const poplar::OptionFlags&, const poplar::DebugContext &debugContext = {})
Create and map a tensor with a group dimension to contain indices for slicing or updating a tensor efficiently.
Groups allow multiple independent slice/update operations of the same size to be done in parallel. Indices have the same bounds for every element in the group.
dims
is defined as in the nongrouped version. They do not include a group dimension.It is an error to call this function without a valid plan and
groupSize
must match the group size with which the operation is planned inplan
. Parameters
graph – The Poplar graph.
groupSize – The size of the group
dims – The dimensions of a tensor to be sliced/updated that will be sliced/updated using these indices.
numIndices – The number of indices this tensor should contain
plan – Plan describing how the slicing/updating operation will be implemented.
options – Flags controlling how the operation will be implemented.
debugContext – Optional debug information.
 Returns
A tensor of shape [groupSize, numIndices, dims.size()] mapped appropriately to be used as the indices for a slice/update operation. Element type is always UNSIGNED_INT.

poplar::Tensor createSliceableTensorFromSlice(poplar::Graph &graph, const poplar::Tensor &s, const std::vector<std::size_t> &dims, const std::vector<std::size_t> &numSlices, const poplar::DebugContext &debugContext = {})
Create and map a tensor to be sliced/updated.
The tensor is mapped in a way that can be efficiently sliced and updated to/from the given slice tensor. It will be distributed across as many tiles as the given slice and with the same contiguous regions on each tile. The tensor’s shape and mapping are derived from the reference slice tensor.
 Parameters
graph – The Poplar graph.
s – The reference slice.
dims – The dimensions of the returned tensor that will be sliced.
numSlices – The number of independent slices in each sliced dimension.
debugContext – Optional debug information.
 Returns
A tensor to be sliced/updated.

poplar::Tensor dynamicSlice(poplar::Graph &graph, const poplar::Tensor &t, const poplar::Tensor &offset, const std::vector<std::size_t> &dims, const std::vector<std::size_t> &sizes, poplar::program::Sequence &prog, const poplar::DebugContext &debugContext = {}, const poplar::OptionFlags &options = {})
Slice a tensor based on offsets specified by a tensor.
dims
gives the dimensions to slice,sizes
defines the size of the slice in those dimensions andoffset
gives the base offsets on each execution.offset
[0],dims
andsizes
must have the same size.offset
may have a second dimension with an element per tile, which can eliminate exchange.** Dynamic slice options **
remapOutOfBoundIndices
(true, false) [=false] Out of bounds indices are mapped to index 0.paddingIndexUsed
(true, false) [=false] Padding index equal to the size of the slice dimension of tensort
may be used in the indices. The actual padding values returned for a padding index are zeros.
Note
If the elements of a single slice cannot be well spread across available tiles (for example, because it has fewer elements than the available number of tiles) then the operation as a whole will also have poor tile balance and consequently poor tile memory balance.
 Parameters
graph – The Poplar graph.
t – The source tensor.
offset – A tensor of offsets at which the output is extracted.
dims – The dimensions of
t
to slice.sizes – The size of the slice in each of the dimensions in
dims
.prog – The program to be extended
options – Option flags
debugContext – Optional debug information.
 Returns
The specified subtensor

poplar::Tensor createDynamicSliceOutput(poplar::Graph &graph, const poplar::Tensor &t, const poplar::Tensor &offset, const std::vector<std::size_t> &dims, const std::vector<std::size_t> &sizes, const poplar::DebugContext &debugContext = {}, const poplar::OptionFlags &options = {})
Create a tensor for use with
dynamicSliceWithOutput
.Parameters, allowed options and the return values are the same as
dynamicSlice
except there is noprog
parameter. Parameters
graph – The Poplar graph.
t – The source tensor.
offset – A tensor of offsets at which the output is extracted.
dims – The dimensions of
t
to slice.sizes – The size of the slice in each of the dimensions in
dims
.options – Option flags
debugContext – Optional debug information.
 Returns
The specified subtensor

void dynamicSliceWithOutput(poplar::Graph &graph, const poplar::Tensor &output, const poplar::Tensor &t, const poplar::Tensor &offset, const std::vector<std::size_t> &dims, const std::vector<std::size_t> &sizes, poplar::program::Sequence &prog, const poplar::DebugContext &debugContext = {}, const poplar::OptionFlags &options = {})
Slice a tensor based on offsets specified by a tensor.
dims
gives the dimensions to slice,sizes
defines the size of the slice in those dimensions andoffset
gives the base offsets on each execution.offset
[0],dims
andsizes
must have the same size.offset
may have a second dimension with an element per tile, which can eliminate exchange.see
dynamicSlice
for information onoptions
 Parameters
graph – The Poplar graph.
output – The output tensor, This should ideally be created with
createSliceTensor
to maximise efficiency,t – The source tensor.
offset – A tensor of offsets at which the output is extracted.
dims – The dimensions of
t
to slice.sizes – The size of the slice in each of the dimensions in
dims
.prog – The program to be extended
debugContext – Optional debug information.
debugContext – Optional debug information
options – Option to control remapping of indices and padding

poplar::Graph::TileToTensorMapping getSliceMapping(poplar::Graph &graph, const poplar::Tensor &t, const std::vector<std::size_t> &dims, const std::vector<std::size_t> &sizes)
Get the tile mapping for a slice of a tensor.
dims
gives the dimensions to slice,sizes
defines the size of the slice in those dimensions. Parameters
graph – The Poplar graph.
t – The source tensor.
dims – The dimensions of
t
to slice.sizes – The size of the slice in each of the dimensions in
dims
.

void dynamicUpdate(poplar::Graph &graph, const poplar::Tensor &t, const poplar::Tensor &s, const poplar::Tensor &offset, const std::vector<std::size_t> &dims, const std::vector<std::size_t> &sizes, poplar::program::Sequence &prog, const poplar::DebugContext &debugContext = {}, const poplar::OptionFlags &options = {})
Update a subtensor at offsets read from a tensor.
dims
gives the dimensions that are partially updated, bysizes
elements, at offsetsoffset
. Unspecified dimensions are copied in full with zero offset.offset
[0],dims
andsizes
must have the same size.offset
may have a second dimension with an element per tile, which can eliminate exchange.see
dynamicSlice
for information onoptions
 Parameters
graph – The Poplar graph.
t – The tensor to update.
s – The updates.
offset – The offset within
t
to be updated.dims – The dimensions to be dynamically updated.
sizes – The size of the update in each of the dimensions in
dims
.prog – The program to be extended.
debugContext – Optional debug information.
options – Option flags

poplar::Tensor multiSlice(poplar::Graph &graph, const poplar::Tensor &t, const poplar::Tensor &offsets, const std::vector<std::size_t> &dims, const std::vector<std::size_t> &sizes, poplar::program::Sequence &prog, const SlicePlan &plan, const poplar::OptionFlags &options, const poplar::DebugContext &debugContext = {})
Take multiple slices from a base tensor.
The returned tensor will have a rank one greater than
t
. Its outer dimension will beoffsets.dim(0)
. Note thatdims
refers to the dimensions oft
.t
can be created usingcreateSliceableTensor()
to ensure efficient mapping.dims
andsizes
must be size 1.** multiSlice options **
remapOutOfBoundIndices
(true, false) [=false] Out of bounds indices are mapped to index 0.paddingIndexUsed
(true, false) [=false] Padding index equal to the size of the slice dimension of tensort
may be used in the indices. The actual padding values returned for a padding index are zeros. Padding index is excluded from index validation if it is enabled.
 Parameters
graph – The Poplar graph.
t – The tensor being sliced.
offsets – The offsets within
t
to be sliced.dims – The dimensions of
t
to be sliced.sizes – The size of the update in each of the dimensions in
dims
.prog – The program to be extended.
plan – Plan describing how the operation will be implemented.
options – Flags controlling how the operation will be implemented.
debugContext – Optional debug information.

poplar::Tensor groupedMultiSlice(poplar::Graph &graph, const poplar::Tensor &t, const poplar::Tensor &offsets, const std::vector<std::size_t> &dims, const std::vector<std::size_t> &sizes, poplar::program::Sequence &prog, const SlicePlan &plan, const poplar::OptionFlags &options, const poplar::DebugContext &debugContext = {})
Take multiple slices from a base tensor where the
base
tensor and theoffsets
tensor have the group dimension as the first dimension.The indices given in the
offsets
tensor should be in the range [0,base.dim(1)
]. Indices outside of this range are allowed but return undefined values.The returned tensor will have a rank one greater than
t
. Its outer dimension is the group size and the second dimension will beoffsets.dim(1)
.dims
indexes the dimensions of each group int
. This makes it consistent with grouped variants of functions to create tensors.t
can be created usingcreateGroupedSliceableTensor()
to ensure efficient mapping.dims
andsizes
must be size 1.see
multiSlice
for informtion onoptions
passed to this function in addition to those passed forplan
. Parameters
graph – The Poplar graph.
t – The tensor being sliced.
offsets – The offsets within
t
to be sliced.dims – The dimensions of
t
for each group.sizes – The size of the update in each of the dimensions in
dims
.prog – The program to be extended.
plan – Plan describing how the operation will be implemented.
options – Flags controlling how the operation will be implemented.
debugContext – Optional debug information.

poplar::Tensor multiSlice(poplar::Graph &graph, const poplar::Tensor &t, poplar::ArrayRef<unsigned> offsets, std::size_t dim, poplar::program::Sequence &prog, const poplar::DebugContext &debugContext = {}, const poplar::OptionFlags &optionFlags = {})
Take multiple slices from a base tensor.
The returned tensor will have a rank one greater than
t
. Its outer dimension will beoffsets.size()
. Note thatdim
refers to a dimension oft
. Any entry inoffset
greater than equal to the size of dims int
returns a slice filled with zeros.see other overloads of multiSlice for information on
optionFlags
. Parameters
graph – The Poplar graph.
t – The tensor being sliced.
offsets – The offsets within
t
to be sliced.dim – The dimension of
t
to be sliced.prog – The program to be extended.
debugContext – Optional debug information.
optionFlags – Option flags.

void multiUpdate(poplar::Graph &graph, const poplar::Tensor &t, const poplar::Tensor &s, const poplar::Tensor &offsets, const std::vector<std::size_t> &dims, const std::vector<std::size_t> &sizes, poplar::program::Sequence &prog, const SlicePlan &plan, const poplar::OptionFlags &options, const poplar::DebugContext &debugContext = {})
Update multiple slices in a tensor.
See overloads of multiSlice for information on
options
.dims
andsizes
must be size 1. Parameters
graph – The Poplar graph.
t – The tensor being updated.
s – The slices to insert.
offsets – The offsets within
t
to be updated.dims – The dimensions of
t
to be updated.sizes – The size of the update in each of the dimensions in
dims
.prog – The program to be extended.
plan – Plan describing how the operation will be implemented.
options – Flags controlling how the operation will be implemented.
debugContext – Optional debug information.

void groupedMultiUpdate(poplar::Graph &graph, const poplar::Tensor &t, const poplar::Tensor &s, const poplar::Tensor &offsets, const std::vector<std::size_t> &dims, const std::vector<std::size_t> &sizes, poplar::program::Sequence &prog, const SlicePlan &plan, const poplar::OptionFlags &options, const poplar::DebugContext &debugContext = {})
Update multiple slices in a tensor with a group dimension.
The tensors
t
s
, andoffsets
have group dimension as the first dimension.dims
andsizes
must be size 1.See overloads of multiSlice for information on
options
. Parameters
graph – The Poplar graph.
t – The tensor being updated.
s – The slices to insert.
offsets – The offsets within
t
to be updated.dims – The dimensions of each group of
t
to be updated.sizes – The size of the update in each of the dimensions in
dims
.prog – The program to be extended.
plan – Plan describing how the operation will be implemented.
options – Flags controlling how the operation will be implemented.
debugContext – Optional debug information.

void multiUpdateAdd(poplar::Graph &graph, const poplar::Tensor &t, const poplar::Tensor &s, const poplar::Tensor &offsets, const poplar::Tensor &scale, const std::vector<std::size_t> &dims, const std::vector<std::size_t> &sizes, poplar::program::Sequence &prog, const SlicePlan &plan, const poplar::OptionFlags &options, const poplar::DebugContext &debugContext = {})
Accumulate multiple slices in a tensor.
dims
andsizes
must be size 1.See overloads of multiSlice for information on
options
. Parameters
graph – The Poplar graph.
t – The tensor being updated (must be rank 2).
s – The slices to accumulate.
offsets – The offsets within
t
to be accumulated.scale – The scaling to apply to the update. The type of the tensor should be the same as that of
t
ands
except for the case whent
ands
are of type HALF. In which casescale
can be of type FLOAT or HALF.dims – The dimensions of
t
to be accumulated (must be rank 1).sizes – The size of the accumulate in each of the dimensions in
dims
.prog – The program to be extended.
plan – Plan describing how the operation will be implemented.
options – Flags controlling how the operation will be implemented.
debugContext – Optional debug information.

void groupedMultiUpdateAdd(poplar::Graph &graph, const poplar::Tensor &t, const poplar::Tensor &s, const poplar::Tensor &offsets, const poplar::Tensor &scale, const std::vector<std::size_t> &dims, const std::vector<std::size_t> &sizes, poplar::program::Sequence &prog, const SlicePlan &plan, const poplar::OptionFlags &options, const poplar::DebugContext &debugContext = {})
Accumulate multiple slices in a tensor where the first dimension of the tensors
t
,s
andoffsets
is the group dimension.t
,s
must be of the same typedims
andsizes
must be size 1.See overloads of multiSlice for information on
options
. Parameters
graph – The Poplar graph.
t – The tensor being updated (must be rank 3).
s – The slices to accumulate.
offsets – The offsets within
t
to be accumulated.scale – The scaling to apply to the update. The type of the tensor should be the same as that of
t
ands
except for the case whent
ands
are of type HALF. In which casescale
can be of type FLOAT or HALF.dims – The dimensions of of each group to be accumulated.
sizes – The size of the accumulate in each of the dimensions in
dims
.prog – The program to be extended.
plan – Plan describing how the operation will be implemented.
options – Flags controlling how the operation will be implemented.
debugContext – Optional debug information.

void multiUpdateAdd(poplar::Graph &graph, const poplar::Tensor &t, const poplar::Tensor &s, poplar::ArrayRef<unsigned> offsets, const poplar::Tensor &scale, std::size_t dim, poplar::program::Sequence &prog, const poplar::DebugContext &debugContext = {}, const poplar::OptionFlags &optionFlags = {})
Accumulate multiple slices in a tensor.
t
,s
must be of the same typeSee overloads of multiSlice for information on
optionsFlags
. Parameters
graph – The Poplar graph.
t – The tensor being updated.
s – The slices to accumulate.
offsets – The offsets within
t
to be accumulated.scale – The scaling to apply to the update. The type of the tensor should be the same as that of
t
ands
except for the case whent
ands
are of type HALF. In which casescale
can be of type FLOAT or HALF.dim – The dimension of
t
to be accumulated.prog – The program to be extended.
debugContext – Optional debug information.
optionFlags – Option flags.

void multiUpdateMax(poplar::Graph &graph, const poplar::Tensor &t, const poplar::Tensor &s, const poplar::Tensor &offsets, const std::vector<std::size_t> &dims, const std::vector<std::size_t> &sizes, poplar::program::Sequence &prog, const SlicePlan &plan, const poplar::OptionFlags &options, const poplar::DebugContext &debugContext = {})
Find maximum over multiple slices in a tensor.
t
,s
must have the same element type offsets[i] >= t.dim(0) are ignored.dims
andsizes
must be size 1.See overloads of multiSlice for information on
options
. Parameters
graph – The Poplar graph.
t – The tensor being updated (must be rank 2).
s – The slices to find maximum over.
offsets – The offsets within
t
to find maximum over.dims – The dimensions of
t
to find maximum over (must be rank 1).sizes – The size of the update in each of the dimensions in
dims
.prog – The program to be extended.
plan – Plan describing how the operation will be implemented.
options – Flags controlling how the operation will be implemented.
debugContext – Optional debug information.

void groupedMultiUpdateMax(poplar::Graph &graph, const poplar::Tensor &t, const poplar::Tensor &s, const poplar::Tensor &offsets, const std::vector<std::size_t> &dims, const std::vector<std::size_t> &sizes, poplar::program::Sequence &prog, const SlicePlan &plan, const poplar::OptionFlags &options, const poplar::DebugContext &debugContext = {})
Find maximum over multiple slices in a tensor with a group dimension.
The tensors
t
,s
,offsets
have groups as their first dimension. Theoffsets
tensor contains indices per group which update elements of the corresponding group.dims
andsizes
must be size 1.See overloads of multiSlice for information on
options
. Parameters
graph – The Poplar graph.
t – The tensor being updated (must be rank 2).
s – The slices to find maximum over.
offsets – The offsets within
t
to find maximum over.dims – The dimensions of each group of
t
to find maximum over (must be rank 1).sizes – The size of the update in each of the dimensions in
dims
.prog – The program to be extended.
plan – Plan describing how the operation will be implemented.
options – Flags controlling how the operation will be implemented.
debugContext – Optional debug information.

class SlicePlan
 #include <DynamicSlice.hpp>
An object representing a plan that describes how to implement a slice or update.
This can be used as a parameter to a function that will slice or update a tensor.

namespace embedding
Functions

SlicePlan plan(const poplar::Graph &graph, const poplar::Type &dataType, const std::size_t numEntries, const std::size_t outputSize, const std::vector<std::size_t> &numLookups, const poplar::OptionFlags &options)
Create a plan for implementing a set of operations on an embedding matrix.
** Embedding plan options **
usedForSlice
(true, false) [=true]If true, you intend to use this embedding plan for both a multiSlice operation. An error is thrown if set to false and
usedForUpdate
is set to false.usedForUpdate
(true, false) [=true]If true, you intend to use this embedding plan for both a multiUpdate operation. An error is thrown if set to false and
usedForSlice
is set to false.operationForUpdate
(“none”, “add”, “max”) [=”add”]Only applicable when
usedForUpdate
= true. Is the type of operation used in multiupdate. Set to “none” for multiUpdate “add” for multiUpdateAdd “max” for multiUpdateMaxavailableMemoryProportion
Decimal between 0 and 1 (inclusive) [=0.6]If set, gives the proportion of tile memory made available for temporary variables (variables that become live and die during the operation) for this operation. If not set, the operation has the freedom to use unlimited temporary memory.
See also
See also
Optimising Temporary Memory Usage for Convolutions and Matmuls on the IPU technical note for some practical examples of using
availableMemoryProportion
indicesDistribution
(uniform, onePoint) [=uniform]A description of the statistical distribution of the indices that will be sliced/updated over the input size (
numEntries
) of the operation. This is used to when estimating the runtime of the multiSlice and multiUpdate operation.uniform
Indices are assumed to be uniformly distributed over the input size of the embedding.onePoint
Indices are assumed to all be equal.
indicesAreSorted
(true, false) [=false]Plan assuming indices used in MultiUpdate/MultiUpdateOp are sorted in increasing order. The same option must then be used along with the plan when calling MultiUpdate with and without an operation.
validateIndices
(true, false) [=false]Check that all indices are valid at runtime. If any is invalid execution is aborted. Ignored if
remapOutOfBoundIndices
= true.partialType
(half, float) If not provided, defaults to using the same as the data type. Partials type should always be the same or higher precision than the data type of the embedding matrix. Is applicable only for the case whereoperationForUpdate
isadd
. It is ignored for all other operations and multislice.
 Parameters
graph – The graph the operation will be added to.
dataType – The data type of the entries in the embedding matrix and the resulting lookups from the matrix.
numEntries – Input size of embedding matrix.
outputSize – Output size of embedding matrix lookup.
numLookups – Vector of numbers of indices which will be looked up in the embedding matrix.
options – Set of option flags controlling how the operation will be implemented.
 Returns
A plan which describes how the embedding matrix lookup/update operations should be implemented.

SlicePlan plan(const poplar::Graph &graph, const poplar::Type &dataType, const std::size_t groupSize, const std::size_t numEntries, const std::size_t outputSize, const std::vector<std::size_t> &numLookups, const poplar::OptionFlags &options)
Overload of
plan
with group size.

std::vector<SlicePlan> planMultiple(const std::vector<SlicePlanningParameters> &spds)
Create multiple plans for implementing a set of operations on an embedding matrix.
The plans are computed in parallel.
 Parameters
spds – A vector of parameters to compute slice plans for. For more information about the individual parameters see
SlicePlanningParameters
andplan
. Returns
A vector of slice plans for each
SlicePlanningParameters
object passed as input.

struct SlicePlanningParameters
 #include <DynamicSlice.hpp>
A complete collection of the parameters required to compute a plan for slice/update operations.
Public Functions
Public Members

const poplar::Type dataType
The data type of the entries in the embedding matrix and the resulting lookups from the matrix.

const std::vector<std::size_t> numLookups
Vector of numbers of indices which will be looked up in the embedding matrix.

const poplar::OptionFlags optionFlags
Set of option flags controlling how the operation will be implemented.

const poplar::Type dataType

SlicePlan plan(const poplar::Graph &graph, const poplar::Type &dataType, const std::size_t numEntries, const std::size_t outputSize, const std::vector<std::size_t> &numLookups, const poplar::OptionFlags &options)