13. Viewing an Operations Graph

The Operations Graph displays a graphical representation of TensorFlow models, showing High Level Operations (HLO) and enabling you to:

  • drill down through the modules, expanding and collapsing the layers to get to the level you want;

  • view details of operations, edges and layers;

  • view the type and shape of the tensors between operations;

  • view graphs of pipelined models to see what is in each pipeline stage, and what passes between them;

  • colour items in the graph based on selected metrics (for example, code size or cycles used);

  • configure the layout with a number of advanced options.

The Operations graph shows how the HLOs are connected to each other in your TensorFlow model in a number of nested layers. Layers and the operations they contain are displayed as boxes, and the tensors that those operations use are shown as arrows between operations.

The report is shown in split-screen, the left-hand side showing the graph at the selected level, and the right-hand side showing information about the selected object in the graph in a series of tabs.

  • Pan around the operations graph by clicking and dragging your mouse anywhere on the graph.

  • Zoom in and out of the operations graph using the mouse scroll wheel.

13.1. Graph entities

There are several types of entity displayed on the operations graph. You can select them, and expand the layers and calls to drill down into the model. You can click and double-click entities on the graph to expand them (layers and calls) or to display other information about them (operations and edges).

You can customise the appearance of the entities in the operations graph by adjusting the advanced view options. The Options menu at the top of the report allows quick access to two of these layout options (Show Backward Pass and Show Edge Labels).

13.1.1. HLO layers

HLO layers are created when two or more operations have debug names which have the same prefix string, separated using the / character. They are displayed as boxes with solid borders and square corners:

HLO Layer

  • Click on a layer to see its details displayed on the right-hand side.

  • Double-click on a layer to expand it. All of that layer’s sub-layers are now displayed within the original layer, which is now displayed as a box around the sub-layer entities. Double-click that outer layer box to return to the original, enclosing layer.

  • Other layers within the operations graph can be displayed by selecting them from the drop-down menu at the top of the graph.

Defining HLO layers

The operations graph works best if you name your TensorFlow operations using either Keras layers or tf.variable_scope (see the TensorFlow documentation for name_scope. Currently the view assumes the top level tf.variable_scope is called all, which is a common convention in the public examples.

For best reults, all forward operations should be in the all layer, and all backward operation names should start with gradients/all.

It has been found that TensorFlow 2 GradientTape does not work well, as it does not record the scope names for the backward pass. For best results, use the TensorFlow 1 compat optimizers.

13.1.2. High Level Operations (HLOs)

HLOs are displayed as boxes with solid, black borders with rounded corners:

HLO Name

  • Click on an HLO to show its details in the tabs on the right-hand side of the report.

Operations that are disconnected from the rest of the graph are displayed as boxes with dashed, grey borders with rounded corners:

Disconnected HLO

13.1.3. HLO Calls/fusion operations

HLO Calls or fusion operations are displayed as boxes with double borders and rounded corners:

HLO Call

  • Click on an HLO Call to show its details in the tabs on the right-hand side of the report.

  • Double-click an HLO Call to expand it and see the operations within it.

  • Expanded HLO Calls show up as a list of breadcrumbs at the top of the graph, and you can click on those to retrace your steps.

13.1.4. Edge tensors

Tensors: Data that is used as the input to, or output from, operations in the graph are displayed as arrows. Their labels are either the tensor’s shape and type (for single tensor), or how many of them there are (for multiple tensors).

  • Click on a tensor to see its details in the tabs on the right-hand side.

There are several options to show or hide labels in the Advanced Options tab.

13.2. Selected entity information tabs

When you select an entity from the graph (a layer, operation, call, or edge), its name is displayed at the top of the right-hand side of the report, and some extra information associated with the entity is displayed in the tabs beneath. This includes:

  • Summary tab: Displays different information depending on which type of graph entity is selected:

    • When a layer is selected, this displays some statistics about the layer in general, such as total estimated FLOPS.

    • When an operation or call is selected, the inputs and outputs to the operation are displayed, together with their associated tensor shape, as well as statistics about code size (OnTileExecute, DoExchange and Total) and estimated FLOPS.

    • When an edge tensor is selected, the source (From) and destination (To) operations are displayed, as well as tensor shape and size.

  • Program Tree tab: Any program tree steps associated with the selected operation.

  • Details tab: Graphs of various metrics plotted against tiles, including code size, cycles and FLOPS.

  • Debug Info tab: Any debug information associated with the selected operation.

  • Advanced Options tab: See Advanced options for details of how to customise the appearance of the operations graph.

13.3. Highlighting operations by metric

The operations graph has a feature to add colour to graph entities based on various metrics. Entities that have a relatively high value for that metric are coloured with a “hot” colour (red) to highlight those operations that are costly in terms of memory, cycles, and so on, and entities with a relatively low value for the metric are coloured with a “cool” colour (blue).

  • From the Highlight drop-down menu at the top of the report screen, select a metric that you want to use to highlight certain operations.

Metrics that are available to use for highlighting include:

  • None: Switch off highlighting

  • Code size (Total)

  • Code size (OnTileExecute)

  • Code size (DoExchange)

  • Estimated Cycles (Total)

  • Measured Cycles (Total)

  • FLOPs

The colour key for metric values is displayed in the top right-hand corner of the graph, showing the highest and lowest values present in the current view.

13.4. Advanced options

The right-most tab on the right hand side of the report shows a number of display options for laying out the items in the operations graph.

  • Check and uncheck options to display or hide information on the graph.

The Options menu at the top of the report allows quick access to two of these layout options (Show Backward Pass and Show Edge Labels).