Loss

#include <popnn/Loss.hpp>

Loss and gradient calculations.

namespace popnn

Functions used in neural networks.

Enums

enum LossType

Values:

enumerator SUM_SQUARED_LOSS

enumerator CROSS_ENTROPY_LOSS

Functions

poplar::program::Program calcLoss(poplar::Graph &graph, const poplar::Tensor &modelOutputs, const poplar::Tensor &expected, const poplar::Tensor &loss, const poplar::Tensor &deltas, const poplar::Tensor &deltasScale, const poplar::Tensor &modelOutputScaling, LossType lossType, const poplar::DebugContext &debugContext = {})

Calculate loss and gradient for a set of activations and expected labels.

Parameters

• graph – Graph to add operations and tensors to.

• modelOutputs – 2D tensor of model outputs per-batch to calculate loss for.

• expected – One-hot encoded tensor (Labels per-batch) with the same number of rows as modelOutputs. Elements of the expected labels may be masked by using MASKED_LABEL_CODE. Such labels will not contribute to loss calculation.

• loss – 1D Tensor to store the loss per-batch. Has the same number of rows as modelOutputs.

• deltas – 2D Tensor to store deltas for each activation from the expected per-batch. Has the same dimensions as modelOutputs.

• deltasScale – Optional Tensor to scale output deltas with when the lossType is CROSS_ENTROPY_LOSS. Scaling will be deltasScale / modelOutputScaling. If no tensor is specified a default will be created initialised with 1.0.

• modelOutputScaling – Optional Tensor indicating the scaling of the modelOutputs when lossType is CROSS_ENTROPY_LOSS, normally from a softMax layer when the nonLinearity used is SOFTMAX_SCALED. If no tensor is specified a default will be created initialised with 1.0.

• lossType – Method for calculating loss measurement.

• debugContext – Optional debug information.

poplar::program::Program calcLoss(poplar::Graph &graph, const poplar::Tensor &modelOutputs, const poplar::Tensor &expected, const poplar::Tensor &loss, const poplar::Tensor &deltas, LossType lossType, const poplar::DebugContext &debugContext = {})

poplar::program::Program calcLoss(poplar::Graph &graph, const poplar::Tensor &modelOutputs, const poplar::Tensor &expected, const poplar::Tensor &loss, const poplar::Tensor &deltas, const poplar::Tensor &deltasScale, const poplar::Tensor &modelOutputScaling, const poplar::Tensor &numCorrect, LossType lossType, const poplar::DebugContext &debugContext = {})

Calculate loss, gradient, and number of correct classifications per-batch for a set of activations and expected labels.

Elements of the expected labels may be masked by using MASKED_LABEL_CODE. Such labels will not contribute to the accuracy and loss calculation.

See also

calcLoss, and calcAccuracy which this function is simply a combination of.

poplar::program::Program calcLoss(poplar::Graph &graph, const poplar::Tensor &modelOutputs, const poplar::Tensor &expected, const poplar::Tensor &loss, const poplar::Tensor &deltas, const poplar::Tensor &numCorrect, LossType lossType, const poplar::DebugContext &debugContext = {})

poplar::program::Program calcAccuracy(poplar::Graph &graph, const poplar::Tensor &modelOutputs, const poplar::Tensor &expected, const poplar::Tensor &numCorrect, const poplar::DebugContext &debugContext = {})

Calculate the number of correct classifications for a set of activations and expected labels.

Parameters

• graph – Graph to add operations and tensors to.

• modelOutputs – 2D tensor of model outputs per-batch to calculate loss for.

• expected – Labels per-batch. Elements of the expected labels may be masked by using MASKED_LABEL_CODE. Such labels will not contribute to the accuracy calculation.

• numCorrect – Tensor to store the number of correct classifications. Must be scalar, or single-element Tensor.

• activationType – Device type used for activations.

• expectedType – Device type used for expected labels.

• debugContext – Optional debug information.

poplar::Tensor argMax(poplar::Graph &graph, const poplar::Tensor &input, poplar::program::Sequence &prog, const poplar::DebugContext &debugContext = {})

Compute argmax for each of the outer dimensions of input tensor.

If input is a tensor of dim [y][x] then argmax is computed over x elements for each of the y outer dimension elements

Parameters

• graph – Graph to add operations and tensors to.

• input – 2D tensor of inputs

• prog – Program to which the graph for this operation is added

• debugContext – Optional debug information.

std::pair<poplar::Tensor, poplar::Tensor> maxAndArgMax(poplar::Graph &graph, const poplar::Tensor &input, poplar::program::Sequence &prog, const poplar::DebugContext &debugContext = {})

Compute max and argmax for each of the outer dimensions of input tensor.

If input is a tensor of dim [y][x] then max and argmax is computed over x elements for each of the y outer dimension elements

Parameters

• graph – Graph to add operations and tensors to.

• input – 2D tensor of inputs

• prog – Program to which the graph for this operation is added

• debugContext – Optional debug information.

poplar::Tensor argMin(poplar::Graph &graph, const poplar::Tensor &input, poplar::program::Sequence &prog, const poplar::DebugContext &debugContext = {})

Compute argmin for each of the outer dimensions of input tensor.

If input is a tensor of dim [y][x] then argmin is computed over x elements for each of the y outer dimension elements

Parameters

• graph – Graph to add operations and tensors to.

• input – 2D tensor of inputs

• prog – Program to which the graph for this operation is added

• debugContext – Optional debug information.

std::pair<poplar::Tensor, poplar::Tensor> minAndArgMin(poplar::Graph &graph, const poplar::Tensor &input, poplar::program::Sequence &prog, const poplar::DebugContext &debugContext = {})

Compute min and argmin for each of the outer dimensions of input tensor.

If input is a tensor of dim [y][x] then argmin is computed over x elements for each of the y outer dimension elements

Parameters

• graph – Graph to add operations and tensors to.

• input – 2D tensor of inputs

• prog – Program to which the graph for this operation is added

• debugContext – Optional debug information.

poplar::Tensor topK(poplar::Graph &graph, const poplar::Tensor &input, poplar::Tensor &indices, unsigned K, bool sort, poplar::program::Sequence &prog, const poplar::DebugContext &debugContext = {})

Find the top K elements of |input|.

Takes a 2D tensor in the form of [batch][values] and will return a tensor in the shape of [batch][K] where K is the max values of each batch of values.

Parameters

• graph – Graph to add operations and tensors to.

• input – 2D tensor of inputs

• indices – The tensor to store the indices in.

• K – The number of values to return.

• sort – If true values will be sorted in descending order.

• prog – Program to which the graph for this operation is added

• debugContext – Optional debug information.