5.3. Keras

3.1.0

New features

  • Improved support for pipelining of nested Keras Models.

  • Individual layers within a nested Keras model can now be assigned to different pipeline stages.

  • This allows nested Keras models to be split across multiple pipeline stages (and hence IPUs). Previously the entire nested model had to be in the same pipeline stage, which meant that pipeline stages could be unbalanced. It also meant that the size of the nested model was limited to one that would fit on a single IPU.

  • Refer to the following documentation for details:

  • Added automatic loss scaling (ALS) support for training using Keras optimizers.

  • ALS improves stability when training large models in mixed precision. It means that engineers do not need to spend significant time and resources to find valid loss scaling factors (note that a static value for loss scaling may not remain optimal throughout training). Refer to the automatic loss scaling documentation for details.

  • Added option to allow a user to specify the reduction method for metrics when training using replication.

  • The previous behaviour was to return the metrics from the last replica, which was unintuitive. The user now has a choice of different reduction methods. Refer to the API reference for ReplicatedMetricReductionMethod for details.

Bug Fixes

None

Other improvements

None

Known issues

None

Compatibility changes

Compatibility changes are listed in the user guide Targeting the IPU from TensorFlow 2:

3.0.0

New features

  • Fix support for keras.Input when calling build on a Model subclass.

  • Extend API for exporting models for tf.serving with the possibility to pass an optional post-processing function to be executed on the CPU as part of the exported model graph.

    This change introduces the possibility of passing optional pre-processing and post-processing functions to the serving API. It enables user export models for TensorFlow Serving with pre/post-processing computations executed on the server CPU.

Bug Fixes

None

Other improvements

None

Known issues

  • Using mixed_precision.Policy('mixed_float16') with pipelined Keras models results in compilation errors.

Compatibility changes

None