10. Viewing a Liveness Report

The Liveness Report shows which of the not-always-live variables are allocated at certain points in your program, and gives a detailed breakdown of variable memory usage by the compute set that they’re in. There are two main areas to the report:

• the top half of the report shows the graph of memory usage against compute set.

• the bottom half of the report shows details about the variables that are used within the selected compute set.

For a standard machine learning model that you’re training, for example a ResNET, you’ll generally see a curve that rises to a peak and drops again. The rising portion of this curve is the memory usage during the forward pass of the training algorithm, where many activations are created (not-always-live variables), and the peak represents the point where the maximum number of activations exist. As the curve drops again, which represents the backwards pass of the training algorithm, the activations are “released” after being used to update the weights.

When you’re inspecting a liveness graph, it’s informative to look at the peak of this curve, and select the corresponding compute set to show the how the variable usage is contributing to the greatest memory utilisation.

10.1. Navigating Liveness reports

You can move around the main report graphs using your mouse, as described in the Adjusting report size section.

10.2. The Liveness graph

• Hover your mouse pointer over the graph to see a callout containing the compute set ID, together with the amount of memory used by that compute set.

• Click on the graph to see that compute set’s stack details below the graph, which displays its name and memory size. You can select multiple program steps by holding down Shift and clicking other program steps. Each of these steps is then displayed in its own column on the Not-Always-Live Variables and Vertices tabs.

• You can choose to display memory usage statistics for the selected tiles. See the relevant Preferences section for details.

• You can plot the lifetime of not-always-live variables directly on the graph. See Not-Always-Live Variables for more details. This is an experimental feature.

If you’re comparing two reports, you can choose to display their liveness graphs together or separately. See the Merge Graphs Viewing options section.

10.2.1. Selecting a source

You can choose whether you want to select an individual tile, or, if the report was generated using Poplar SDK 1.2 or later, select an individual IPU.

To select particular tiles or IPUs:

• Click on the Select Source drop-down list at the top of the graph and select the source you want to use for the graph. Each source will have its own plot.

• All tiles: Shows liveness data for all the tiles (the default)

• Worst tiles: Shows the two tiles that have the highest memory usage during program execution, and you can select either of them to view.

• If the report was generated in Poplar 1.2 or later, you can also select one or more IPUs from this list.

There is a Poplar Engine setting which allows you to capture more than the default two worst tiles. Please refer to the a Poplar API Reference for full details of these options.

10.2.2. Filtering steps

You can concentrate on the steps you’re interested in by filtering based on the step name. Type the name into the Search box and press Enter. Any step whose name matches the search text is displayed in the execution graph, while all other steps are moved to a separate dataset in the graph, and greyed-out.

To cancel the filtering, click on the small x at the right-hand end of the Search box.

10.2.3. Viewing options

There are several options for viewing Liveness Report graphs, which you can select from the Options drop-down menu in the top right-hand corner of the report screen:

• Include Always-Live: Include a second trace in the graph that shows variables that are always present during the entire execution of the program, and always have the same memory address (for example stack). Note that you can show and hide each of the traces on the graph by clicking on their colour key, just below the x-axis.

• Include empty stacks: Include stacks that contain no variables.

• Stack IPUs: Display any selected IPUs stacked or not. See the Selecting a source section for details.

• Show Max Memory: Display a line on the graph showing the maximum memory limit for total memory, a selected tile, or a selected IPU. Note that this requires the Include Always-Live option, above, to be enabled, and Stack IPUs to be disabled. For reports that contain replicas, this figure is the max memory per tile multiplied by the number of tiles per replica.

• Merge Graphs: Merge source and target report graphs together, or keep them as separate graphs. When displayed separately, you can navigate each graph independently, and select different program steps from each report. If your two reports contain different numbers of program steps, you’ll be able to select the corresponding steps from each to view their details. When the graphs are merged together, you navigate them together “as one”. When you click on the graph, the two program steps you select (from the source and target reports) may not correspond to the same element of each program.

10.3. Liveness stack details

In the bottom half of the Liveness report screen, you’ll see a tabbed list of live variable details that show their memory usage. These details are described in Viewing enhanced debug information.

10.3.1. Viewing enhanced debug information

If you have captured enhanced debug information when compiling your program, it is visible in the Always-Live and Not-Always-Live Variables tabs. See the section on the Debug Information file for details of how to capture this information.

If a variable has a small disclosure arrow next to it, click it to show enhanced debug information. You can see a list of software layers on the left (for example, Poplar and PopLibs). Selecting a layer shows the debug information that has been added by that layer.

For all variables you have information captured from the Poplar layer. This may include:

• whether it’s a Variable, Cloned Variable or Constant,

• the shape of the variable as an array of dimensions,

• the type of variable (for example Half),

• for cloned variables, which variable it was cloned from, and the method by which it was cloned,

If the variable was created as part of a PopLibs API call, you can also see the following information:

• the PopLibs API call

• the input tensors to the API call

• the arguments to the API call

• the output tensors to the API call

Note the variable may not be an output of the PopLibs API call. It could be an internal variable created for the operation. Depending on your application, you may see further debug information for the framework and/or application.

By default, the debug information for the first PopLibs and Poplar call is shown. You can choose to show all PopLibs/Poplar calls that are made internally by clicking the gear icon in the top right-hand corner of the debug information box and selecting Show All Debug Infos. For instance, you may see the debug information for the PopLibs matmul API call. If you enable the option to Show All Debug Infos, you will see the internal implementation PopLibs calls, which include the PopLibs convolution API call. (matmul is implemented as a convolution.)

10.3.2. Always-Live Variables

This tab shows the variables that are always live. Their data must always be available and therefore they cannot share memory with any other variable.

10.3.3. Not-Always-Live Variables

This tab shows the variables that are not always live. At certain points in the program we do not need to store any data in them, and therefore other variables can be allocated at the same location. Variables are never “allocated” or “deallocated” at runtime. All variables are statically allocated at compile time and always have a fixed address.

You can select one or more of the not-always-live variables to see their lifetime displayed on the graph above. Click on the small icon at the right-hand end of their listing, and they will be added to the plot, showing when they were created, and when they were destroyed. Each variable plotted also appears as a small blue box above the graph, and you can click the small x within them to remove them from the graph. This is an experimental feature.

10.3.4. Vertices

This tab shows the vertex functions that are contained in the selected compute set.

• The origin of each vertex, in a small blue box, is displayed after the vertex name, indicating whether the vertex was written in C++ or Assembler (ASM).

• The number of that vertex type created is also shown.

10.3.5. Cycle estimates

This tab shows the cycle estimates of each tile on an IPU. This is only available when using an IPUModel.

This is an experimental feature, and can be enabled and disabled in the Preferences.

10.3.6. Show differences between selected tiles

When comparing reports the difference between values is displayed in red or green text on the table view. You can remove variables that have the same value by enabling the Show differences between selected tiles option. This is accessible by clicking on the cog icon and checking the box in the drop-down menu. This filters out all variables that have the same value, and leaves only those that are different.