18.9. Session

class popxl.Session(ir, device_desc='cpu')

Parameters

ir (Ir) –
device_desc (Literal['ipu_hw', 'ipu_model', 'cpu']) –

Return type

None

__enter__()

Enter the context of this Session.

If not already attached to a device, this will attach to an available device and perform a weights_from_host.

See Section 11, Session for a more comprehensive guide to sessions and the context manager.

Return type: Session

__exit__(exc_type, exc_value, exc_tb)

Exit the context of this Session.

If you were not attached when entering this context, this will perform a weights_to_host then detach.

See Section 11, Session for a more comprehensive guide to sessions and the context manager.

__init__(ir, device_desc='cpu')

Construct a session object.

A runtime session that can execute a PopXL Ir.

Warning

The session object takes ownership of the provided Ir and it cannot be modified afterwards.

Initialise a new session.

Parameters

ir (Ir) – The IR to use for this session.
device_desc (Literal["ipu_hw", "ipu_model", "cpu"], optional) –
This can be either a DeviceInfo object created via the popart.DeviceManager or the type of IPU device to use, chosen from the following:
- ”ipu_hw”: Real IPU hardware. Uses DeviceConnectionType == OnDemand and DeviceSelectionCriterion == Random.
- ”ipu_model”: IPU model.
- ”cpu”: CPU model. Does not support replication. Defaults to “cpu”.

Raises

RuntimeError – If the desired device could not be acquired.

Return type

None

create_host_outputs()

Return a mapping from popxl.DeviceToHostStream to an empty np.ndarray.

Later, this can be passed to session.run_with_outputs, which will fill each array with the values streamed back from device.

There is an entry in the mapping for every stream in ir.get_all_d2h_streams(), for the IR that this Session was constructed for.

For stream s, the shape of the np.ndarray it maps to is:

(d, r) + s.shape

where:

d = ir.num_host_transfers

r = ir.instance_replication_factor

and all dimensions not >1 in (d, r) will be removed.

Examples:

If:

ir.num_host_transfers = 4
ir.instance_replication_factor = 16
s.shape = (2, 4)

Then the shape will be (4, 16, 2, 4)

If:

ir.num_host_transfers = 1
ir.instance_replication_factor = 16
s.shape = (2, 4)

Then the shape will be (16, 2, 4)

If:

ir.num_host_transfers = 4
ir.instance_replication_factor = 1
s.shape = (2, 4)

Then the shape will be (4, 2, 4)

If:

ir.num_host_transfers = 1
ir.instance_replication_factor = 1
s.shape = (2, 4)

Then the shape will be (2, 4)

Note

Batch serialisation is not supported, so there is no dimension for this.

Return type: Mapping[DeviceToHostStream, ndarray]

expected_inputs()

Return the list of expected inputs for this session.

Data will need to be provided for each of these when doing: run().

Returns

A list of all the host to device streams: required by this session.

Return type

List[HostToDeviceStream]

get_tensor_data(tensor)

Get the data stored in the tensor on the device including IPU and remote memory.

This will sync all the host buffers with the corresponding tensors on the device. Note this is a memory view of the data, so will not allocate extra memory for the data, but it is your responsibility to ensure the data in the tensor is live at the point of retrieval.

Parameters

tensor (Union[Variable, Constant]) – The tensor to get the data for. Must be Constant or Variable type.

Returns

The data for the tensor in question, with type and: shape the same as the device tensor.

Return type

np.ndarray

get_tensors_data(tensors)

Call get_tensor_data for multiple tensors.

This will only sync the host and device buffers once.

Parameters

tensors (Iterable[Union[Variable, Constant]]) – An iterable of the tensors to provide data for.

Raises

TypeError – If any tensor is not of type Constant, Variable.

Returns

A dictionary of: tensors and the corresponding data arrays returned.

Return type

Dict[Union[Constant, Variable], np.ndarray]

property ir: popxl.ir.Ir

Return the associated IR for this session.

Read only.

property is_attached: bool: Return if the session is attached to a device.

run(inputs=None, downcast_inputs=True)

Run run_with_outputs() but create the expected outputs and return them.

Parameters

inputs (h2dStreamBufferMaps, optional) – The inputs to the model. Defaults to None.
downcast_inputs (bool) – If True 64-bit float/ints inputs will be downcast to 32-bit variants. Defaults to True.

Returns

The map of outputs from the model.

Return type

d2hStreamBufferMaps

Raises

ValueError – If not attached to device before calling this function.

run_with_outputs(inputs=None, outputs=None, downcast_inputs=True)

Run this session with the provided inputs and outputs.

Inputs will be used as inputs to the model, and outputs will be written to by the session.

Parameters

inputs (h2dStreamBufferMaps, optional) – The inputs to the model. Defaults to None.
outputs (d2hStreamBufferMaps, optional) – The output buffers, these will be written to and modified. Defaults to None.
downcast_inputs (bool) – If True 64-bit float/ints inputs will be downcast to 32-bit variants. Defaults to True.

Raises

ValueError – If not attached to device before calling this function.

Return type

None

weights_from_host()

Copy the weights to device from the host.

Raises: ValueError – If not attached to device before calling this function.
Return type: None

weights_to_host()

Copy the weights to host from the device.

Raises: ValueError – If not attached to device before calling this function.
Return type: None

write_variable_data(tensor, data)

Write the variable tensor data from the provided host array.

This is only valid for Variable type tensors. The tensor and data must have matching shapes and dtypes.

If attached to a device, the Variable will be updated on host, then a weights_from_host will occur to update the weights on device.

If not attached to device, the Variable will be updated on host only. The next call to weights_from_host will send this updated value to device.

Parameters

tensor (Variable) – The popxl tensor to update.
data (np.ndarray) – The array to update the tensor data to.

Raises

TypeError – If the tensor is not a Variable.
TypeError – If the data types do not match.
ValueError – If the shapes do not match.

Return type

None

write_variables_data(tensors)

Call write_variable_data for multiple tensors in one go.

Like write_variable_data, the weights_from_host will only occur if already attached to device when calling this function.

The host to device transfer is delayed until the end so that it only happens once.

Raises

TypeError – If any input tensor is not of type Variable.
TypeError – If the input array data type does not match that of the associated tensor.
ValueError – If the input array shape does not match that of the associated tensor.
NotImplementedError – If the retrieval mode of the variable is “all_replicas. This is currently not supported.

Parameters

Dict[ (tensors) – A dictionary of tensors and the corresponding array to call ‘write_variable_data` with.
tensors (Dict[Variable, ndarray]) –

Return type

None