Rnn

#include <popnn/Rnn.hpp>

Functions for recurrent neural networks (RNN).

namespace popnn

Functions used in neural networks.

namespace rnn

Functions for Recurrent Neural Networks (RNN)

Typedefs

using LoopBodyType = std::function<poplar::program::Sequence(poplar::Graph &graph, const TimeStepState &time, const RnnBatchwiseFlags&, std::vector<poplar::Tensor>&, const RnnSlice &slice, std::vector<poplar::Tensor>&, poplar::program::Sequence*, const poplar::DebugNameAndId&)>

Create loop body function for the given shard.

Param graph

The graph object.

Param shardIdx

The tensor that specifies the starting sequence index for the current shard.

Param seqIdx

The tensor that iterates over the range of input sequences that are mapped on the current shard, beginning from 0.

Param batchwiseFlags

Flags that indicate batches for which the current step is within the batchwise step limit.

Param state

State tensors.

Param slice

The input/output tensors for a specific shard.

Param created

The output tensors which are created by this function.

Param prog

The program initialization sequence.

Param dnai

Debug name and Id.

Return

Loop body function for the given shard.

using GatherBodyType = std::function<poplar::program::Sequence(poplar::Graph &graph, const RnnSlice &slice, unsigned stepsPerGather, poplar::program::Sequence*, const poplar::DebugNameAndId&)>

Create gather body function for the given shard.

Param graph

The graph object

Param slice

The input/output tensors for a specific shard

Param stepsPerGather

The time step interval between calls to this function.

Param prog

The program initialization sequence

Param dnai

Debug name and Id

Return

Gather body function for the given shard.

Functions

poplar::Tensor createInitialState(poplar::Graph &graph, const RnnParams &params, bool isOutput, unsigned multiple, unsigned numShards, const poplar::DebugContext &debugContext = {})

Create state tensor to be used in all recurrences of the RNN.

The tensor shape is [multiple, batchSize, size] where size is determined by whether the state tensor is an input or output tensor, depending on isOutput. If the RNN is sharded, a tensor of this shape is created for each shard.

Parameters

  • graph – The graph object.

  • params – The RNN parameters.

  • isOutput – Flag that indicates that the state tensor will be an output tensor. If false, indicates that this is an input tensor.

  • multiple – The number of state variables that are concatenated into a single state tensor.

  • numShards – The number of shards to be used.

  • debugContext – Debug information.

Returns

State tensor of shape [multiple, batchSize, size].

poplar::Tensor createRecurrentTensor(poplar::Graph &graph, const RnnParams &params, unsigned size, unsigned numShards, const poplar::DebugContext &debugContext = {})

Create recurrent tensor of shape [timeSteps, batchSize, size] suitable for slicing and/or sharding of the outermost dimension.

Parameters

  • graph – The graph object.

  • params – The RNN parameters.

  • size – The innermost dimension of the tensor.

  • numShards – The number of shards to be used.

  • debugContext – Debug information.

Returns

Recurrent tensor of shape [timeSteps, batchSize, size].

poplar::Tensor createInputTensor(poplar::Graph &graph, const RnnParams &params, unsigned numShards, const poplar::DebugContext &debugContext = {})

Create input tensor of shape [timeSteps, batchSize, inputSize] suitable for slicing and/or sharding of the outermost dimension.

Parameters

  • graph – The graph object.

  • params – The RNN parameters.

  • numShards – The number of shards to be used.

  • debugContext – Debug information.

Returns

Input tensor of shape [timeSteps, batchSize, inputSize].

poplar::Tensor createOutputTensor(poplar::Graph &graph, const RnnParams &params, unsigned numShards, const poplar::DebugContext &debugContext = {})

Create a standard output tensor of shape [timeSteps, batchSize, outputSize] suitable for slicing and/or sharding of the outermost dimension.

Parameters

  • graph – The graph object.

  • params – The RNN parameters.

  • numShards – The number of shards to be used.

  • debugContext – Debug information.

Returns

Output tensor of shape [timeSteps, batchSize, outputSize].

poplar::Tensor createOutputTensor(poplar::Graph &graph, const RnnParams &params, unsigned multiple, unsigned numShards, const poplar::DebugContext &debugContext = {})

Create a single output tensor with multiple (standard) output tensors concatenated along the outermost (timeSteps) dimension.

The concatenated tensor is of shape [multiple * timeSteps, batchSize, outputSize] which is suitable for slicing and/or sharding along the outermost dimension.

Parameters

  • graph – The graph object.

  • params – The RNN parameters.

  • multiple – The number of standard output tensors to be concatenated.

  • numShards – The number of shards to be used.

  • debugContext – Debug information.

Returns

Output tensor of shape [multiple * timeSteps, batchSize, outputSize].

poplar::Tensor shiftRnnTensor(poplar::Graph &graph, const RnnParams &params, const poplar::Tensor &tBase, const poplar::Tensor &tSingle, poplar::program::Sequence &prog, unsigned numShards, const poplar::DebugContext &debugContext = {})

Create a single-step shifted RNN tensor from an input tensor.

For an input tensor tBase, n-th iteration of the RNN tensor points to the n-1-th iteration of tBase. For the 0-th iteration of the RNN tensor, a copy is made from the provided tensor tSingle tensor.

Parameters

  • graph – The graph object.

  • params – The RNN parameters.

  • tBase – The tensor to shift.

  • tSingle – The tensor to be copied in the 0-th iteration.

  • prog – The program to add a tensor copy.

  • numShards – The number of shards to be used.

  • debugContext – Debug information.

Returns

RNN tensor which is a single-step shifted version of tBase.

std::vector<poplar::Tensor> Rnn(poplar::Graph &graph, const RnnParams &params, bool reverse, const std::vector<poplar::Tensor> &initState, const StateSequence &stateSequence, const std::vector<poplar::Tensor> &inputs, const poplar::Tensor *interimIn, poplar::Tensor *interimOut, const std::vector<poplar::Tensor> &outputs, const std::vector<poplar::Tensor> &created, poplar::program::Sequence &prog, const LoopBodyType &loopFn, unsigned numShards, poplar::OptionFlags &options, const poplar::DebugContext &debugContext = {})

Run custom Recurrent Neural Net cell implementation recurrently.

RNN options

  • codeReuse (true, false) [=false]

    If true, the custom RNN implementation defined by the loopFn parameter will be reused by every shard. If false, the RNN code is duplicated for every shard.

Parameters

  • graph – The graph to which the RNN cell belongs.

  • params – The RNN parameters.

  • reverse – Process tensors in reverse order, so beginning with the last element.

  • initState – The state tensors that specify the initial states.

  • stateSequence – Optionally, specify that the recurrent updates of a state tensor need to be stored in a user-defined output tensor.

  • inputs – The input tensors for each recurrence.

  • *interimIn – Pointer to the intermediate inputs to cell computation.

  • *interimOut – Pointer to the intermediate outputs from cell computation.

  • output – The output tensor for each recurrence. Each tensor must be defined prior to calling Rnn().

  • created – The output tensor that is allocated by the custom implementation defined in loopFn.

  • prog – Program sequence.

  • loopFn – The loop body function for RNN cell computation which is invoked for every shard.

  • numShards – The number of shards to be used.

  • options – The RNN implementation options. See createInput().

  • debugContext – Optional debug information.

std::vector<poplar::Tensor> Rnn(poplar::Graph &graph, const RnnParams &params, const std::vector<poplar::Tensor> &initState, const StateSequence &stateSequence, const std::vector<poplar::Tensor> &inputs, const poplar::Tensor &interimIn, const unsigned numTemps, poplar::program::Sequence &prog, const LoopBodyType &loopFn, const std::vector<poplar::Tensor> &gatherInputs, const GatherBodyType &gatherFn, unsigned numShards, unsigned stepsPerGather, poplar::OptionFlags &options, const poplar::DebugContext &debugContext = {})

Run custom Recurrent Neural Net cell callback at every time step in decrementing order.

At each time step, create a temporary variable and pass it to the gatherFn callback function which gets called at a time step interval determined by the stepsPerGather parameter.

RNN options

  • codeReuse (true, false) [=false]

    If true, the custom RNN implementation defined by the loopFn parameter will be reused by every shard. If false, the RNN code is duplicated for every shard.

Parameters

  • graph – The graph to which the RNN cell belongs.

  • params – The RNN parameters.

  • initState – The state tensors that specify the initial states.

  • stateSequence – Optionally, specify that the recurrent updates of a state tensor need to be stored in a user-defined output tensor.

  • inputs – The input tensors to the loop body function loopFn

  • interimIn – The intermediate inputs to cell computation.

  • numTemps – The number of temporary variables of shape [batchSize, size] per time step which are to be passed to the gatherFn callback function.

  • prog – Program sequence.

  • loopFn – Function for RNN cell computation which is invoked for every time step.

  • gatherInputs – The input tensors to the gather body function gatherFn.

  • gatherFn – The gather body function which processes the temporary buffer generated by the loop body function loopFn with the time step interval between calls determined by the stepsPerGather parameter.

  • numShards – The number of shards to be used.

  • stepsPerGather – The time step interval used by the gatherFn callback.

  • options – The RNN implementation options. See createInput().

  • debugContext – Optional debug information.

struct RnnBatchwiseFlags

Public Functions

inline bool valid() const

Public Members

poplar::Tensor mask
poplar::Tensor inverse
struct RnnParams
#include <Rnn.hpp>

Structure of Recurrent Neural Network (RNN) parameters which allows for any customized implementation of the cellular part of the RNN.

Public Functions

RnnParams(poplar::Type dataType, std::size_t batchSize, std::size_t timeSteps, std::vector<std::size_t> layerSizes)
RnnParams(poplar::Type dataType, std::size_t batchSize, std::size_t maxTimeSteps, const poplar::Tensor &varTimeSteps, std::vector<std::size_t> layerSizes)
std::size_t getMaxShards(const poplar::Graph &graph) const
std::size_t getInputBytesPerTile(const poplar::Graph &graph) const
std::size_t getOutputBytesPerTile(const poplar::Graph &graph) const
bool variableTimeSteps() const
bool batchVariableTimeSteps() const

Public Members

poplar::Type dataType

The datatype used for the RNN.

std::size_t batchSize

The batch size.

std::size_t maxTimeSteps

The maximum number of RNN time steps.

std::size_t timeSteps

Deprecated:

Use RnnParams.maxTimeSteps instead.

poplar::Tensor varTimeSteps

The run-time number of RNN time steps of dimension [batchSize] If this tensor is default constructed, the number of time steps for the sequence corresponding to each batch will be set according to the maxTimeSteps member.

std::vector<std::size_t> layerSizes

For each RNN layer, the layer size parameter needs to be specified for the input and the output.

This is done using a 2-element vector where the first element is the input size and the second element is the output size of the RNN layer.

struct RnnSlice
#include <Rnn.hpp>

Tensors required for processing a single time step.

Param inputs

The input tensor sequences.

Param interimIn

The intermediate input sequence.

Param interimOut

The intermediate output sequence.

Param outputs

The output tensor sequences.

Public Members

std::vector<poplar::Tensor> inputs
poplar::Tensor interimIn
poplar::Tensor interimOut
std::vector<poplar::Tensor> outputs
struct StateSequence
#include <Rnn.hpp>

Structure that associates a particular state tensor with a user-defined output tensor.

When passed to the Rnn() function, the state tensor for each recurrence is stored in the tensor output.

Param output

The output tensor which stores the state.

Param stateIndex

The index which identifies the state tensor which will be stored in the output tensor.

Public Members

poplar::Tensor output
std::size_t stateIndex
struct TimeStepState

Public Members

poplar::Tensor begin
poplar::Tensor counter
poplar::Tensor variableSeqFlag