5.17. Model Runtime

3.4.0

Add IPU_VISIBLE_DEVICES environment variable which can be used to restrict an application to a specific IPU device or set of devices. See the section on controlling which devices to use in the Model Runtime: User Guide for more details. There is an improvement in performance when the CPU, memory and IPU device are on the same NUMA node.
Add monitoring statistics (such as mean and P99) which can be used in monitoring tools.

None

General improvements to Model Runtime: User Guide, including a new chapter on how errors are handled.

None

None

Support for dynamic batching in ModelRunner. Adds the ability to support dynamic batching with any batch size input data.
Support for conditional model execution in compiled graphs that contain more than one model. See Conditional execution in the Model Runtime User Guide.

Some errors could cause Model Runtime to hang instead of raising an error. This has been fixed.

None

None

None