TensorCollectives

#include <popops/TensorCollectives.hpp>

Support for collectives.

Defines

POPLIBS_DEPRECATED(x)
namespace popops

Common functions, such as elementwise and reductions.

Functions

Chunks reduceScatter(poplar::Graph &graph, const poplar::Tensor &toReduce, popops::CollectiveOperator op, poplar::program::Sequence &prog, const poplar::DebugContext &debugContext = {}, const poplar::OptionFlags &options = {})

Given a tensor of rank 2 reduce across the outermost dimension using the specified reduction operator.

This function assumes index i in the outermost dimension is mapped to IPU i. The result is distributed over IPUs such that each IPU has a slice of the final result.

Deprecated:

Use gcl::reduceScatterWithinReplica() instead

Parameters

  • graph – The graph.

  • toReduce – The tensor to reduce. Each partial should be mapped identically to the others across the IPUs within the rank.

  • op – The reduction operator (for example, popops::CollectiveOperator::ADD).

  • prog – The program sequence to add operations to.

  • debugContext – Optional debug information.

  • options – Collective options (not currently used).

Returns

A vector of chunks, where chunk i resides on IPU i. The chunks may have different numbers of elements (for example, when the number of IPUs does not exactly divide the number of elements).

poplar::Tensor allGather(poplar::Graph &graph, const Chunks &toGather, poplar::program::Sequence &prog, const poplar::DebugContext &debugContext = {}, const poplar::OptionFlags &options = {})

Broadcast data distributed over all IPUs.

This function assumes chunk i is mapped to IPU i.

Deprecated:

Use gcl::allGatherWithinReplica() instead

Parameters

  • graph – The graph.

  • toGather – The chunks to gather.

  • prog – The program sequence to add operations to.

  • debugContext – Optional debug information.

  • options – Collective options. See reduceScatter().

Returns

A 2D tensor that contains a copy of the data for each IPU. Index i in the outermost dimension of the result is mapped to IPU i.

poplar::Tensor allReduce(poplar::Graph &graph, const poplar::Tensor &toReduce, popops::CollectiveOperator op, poplar::program::Sequence &prog, const poplar::DebugContext &debugContext = {}, const poplar::OptionFlags &options = {})

Perform an all-reduce operation on the specified tensor.

This operation reduces across the outermost dimension of the input and produces a tensor with the same shape where the innermost dimension is the result of the reduction and the outermost dimension is a number of copies of the result. This function assumes index i in the outermost dimension of the input is mapped to IPU i. Index i in the outermost dimension of the result is mapped to IPU i.

Deprecated:

Use gcl::allReduceWithinReplica() instead

Parameters

  • graph – The graph.

  • toReduce – The tensor to reduce. Each partial should be mapped identically to the others across the ipus within the rank.

  • op – The reduction operator (for example, popops::CollectiveOperator::ADD).

  • prog – The program sequence to add operations to.

  • debugContext – Optional debug information.

  • options – Collective options. See reduceScatter().

Returns

A tensor with the same shape as toReduce, where the innermost dimension is the result of the reduction and the outermost dimension has a number of copies of the result.