6. PopDist Python API reference

popdist.checkNumIpusPerReplica(arg0: int) → bool

Check if the IPUs per replica in the context matches the expected

number of IPUs. False is returned if the environment variable is set and does not match the given value. On the other hand, if the environment variable is not set, a check is performed to see if the number of IPUs per replica corresponds to the expected number of IPUs.

popdist.execute_on_instances(instances, func, *args, **kwargs)

Executes func only on instances specified in the instances argument. In addition to that, this function makes sure any synchronization or collective calls only happen across those instances.

Args: instances (Set[int]): Participating instances. func (Callable): Function to be executed. args (Any): Args passed to func. kwargs (Any): Kwargs passed to func.

Returns

The result returned by func.

Parameters

• instances (Set[int]) –

• func (Callable) –

• args (Any) –

• kwargs (Any) –

Return type

Any

popdist.getDeviceId(ipusPerReplica: int = 0) → int

Gets the device id of device suitable for PopDist.

popdist.getInstanceIndex() → int

Gets the index of the current instance.

Can only be used with a uniform number of replicas per instance. Returns instance index in [0, getNumInstances()).

popdist.getLocalInstanceIndex() → int

Gets the relative index of an instance within a host.

Returns 0 if no environment variable is set.

popdist.getNumInstances() → int

Gets the total number of instances.

Can only be used with a uniform number of replicas per instance.

popdist.getNumIpusPerReplica() → int

Get the number of IPUs per replica.

A function that will try to infer the number of IPUs per replica from the environment variables. Will default to 1 if the environment variable cannot be determined.

popdist.getNumLocalReplicas() → int

Gets the number of local replicas.

Infers the number of local replicas automatically from the environment variables. Returns 1 if the environment variable is not set.

popdist.getNumTotalReplicas() → int

Get the total number of replicas.

Infer the total number of replicas from environment variables. Will default to 1 if the environment variable cannot be determined.

popdist.getReplicaIndexOffset() → int

Gets the replica index offset.

The replica index corresponds to the offset of the first replica in an instance. Returns 0 if no environment variable is set.

popdist.init()

Initializes the default backend used for performing collective operations.

Return type

None

popdist.isBackendInitialized() → bool

Checks whether the backend has been initialized.

popdist.isPopdistEnvSet()

Check if the PopDist environment is set.

Returns

True if set.

Return type

bool

popdist.isUniformReplicasPerInstance() → bool

Checks if the number of replicas per instance is uniform.

Automatically inferred from the environment variables passed to PopDist. Returns True if the number of replicas per instance is the same for all instances or if no environment is set.

popdist.registerCommunicator(participants: Set[int], communicator_id: str = '') → None

Registers a communicator that will be used for

collectives/synchronization using a subset of instances.

popdist.synchronize(communicator_id: str = '', participants: Set[int] = set()) → None

Synchronizes code execution over selected instances.

6.1. PopART

popdist.popart.configureSessionOptions(opts)

Configure PopART session options to work with the PopDist context.

Parameters

opts (popart.SessionOptions) –

Return type

None

popdist.popart.getDevice(ipusPerReplica=0, connectionType=popart.DeviceConnectionType.Always)

Get a PopART device that works with the PopDist context.

Parameters

• connectionType (popart.DeviceConnectionType) – The popart.DeviceConnectionType to pass to the popart.DeviceManager when acquiring the device. Defaults to popart.DeviceConnectionType.Always. See also popart.DeviceConnectionType.

• ipusPerReplica (int) –

Returns

A device.

Return type

popart.DeviceInfo

6.2. PopTorch

class popdist.poptorch.Options(*args, **kwargs)

An extension to PopTorch’s Options class so that it is easier to pass application-specific options to PopDist.

Parameters

• args (Any) –

• kwargs (Any) –

Return type

Any

6.3. TensorFlow 1 and 2

popdist.tensorflow.set_ipu_config(config, ipus_per_replica=None, configure_device=True)

Set the PopDist configuration options for TensorFlow.

Parameters

• config – An IPUConfig instance created with tensorflow.python.ipu.config.IPUConfig() to update.

• ipus_per_replica (int) – The number of IPUs per replica (deprecated), this value will be extrapolated from PopRun parameter “–ipus-per-replica”.

• configure_device (bool) – Whether to update config to select the IPU device for PopDist execution.

Returns

The passed config.