Examples and Tutorials

Overview

Tutorials repository

Simple example programs and tutorials are available in the Graphcore GitHub tutorials repository. This includes:

• Tutorials to help you get started using the Poplar SDK and Graphcore tools to run code on the IPU.

• Feature examples: small code examples showing you how to use various software features when developing for IPUs.

• Simple application examples: basic applications written in different frameworks targeting the IPU.

• Kernel benchmarks: code for benchmarking the performance of some selected types of neural network layers on the IPU, using TensorFlow or our PopART framework.

The tutorials repository also contains code used in technical notes, videos and blogs.

Examples repository

Sample applications are available in the Graphcore GitHub examples repository.

TensorFlow 1

The following examples are provided for TensorFlow 1. These include full models as well as examples of how to measure performance, use multiple IPUs and implement custom ops in Poplar, among other things.

These examples are tested against TensorFlow 1. Some of them will run with TensorFlow 2 without any changes.

Tutorials

• Tutorial 1: Porting a simple example

• Tutorial 2: Loops and data pipelines

• Training a model using half- and mixed-precision floating point

• Converting a model to run on multiple IPUs with pipelining

Feature examples

These include examples of how to measure performance, use multiple IPUs and implement custom ops in Poplar.

Performance

• Inspecting tensors

  This example trains simple pipelined and non-pipelined models on the MNIST numeral data set and shows how tensors (containing activations and gradients) can be returned to the host for inspection using outfeed queues. This can be useful for debugging a model.

• I/O benchmarking

  The MNIST simple application shows how to use dataset_benchmark() to determine the maximum achievable throughput for a given dataset.

Using multiple IPUs

Simple examples demonstrating and explaining different ways of using multiple IPUs are provided.

• Pipelining and replication are used to parallelise and speed up training, whereas sharding is generally used to simply fit a model in memory. Concurrent pipelining is a variant of pipelining.

• PopDist training example

  This shows how to enable distributed training of a TensorFlow model by using the PopDist API.

Custom ops

• Custom op example

  Code that demonstrates how to define your own custom op using Poplar and PopLibs and use it in TensorFlow 1.

• Custom op example with gradient

  Code that demonstrates how to define your own custom op using Poplar and PopLibs, and use it in TensorFlow 1. Also shows how to define the gradient of your custom op so that you can use automatic differentiation and operations that depend on it, such as the minimize() method of an optimizer.

Other examples

• IPUEstimator

  Example of using the IPU implementation of the TensorFlow Estimator API.

• Configuring IPU connections

  A code example which demonstrates how to use set_ipu_connection_type() to control if and when the IPU device is acquired.

Simple application examples

• Classifying hand-written digits

  The MNIST dataset is a well-known example of a basic machine learning task. This is an example of its implementation on IPUs. This example also shows how to use dataset_benchmark() to determine the maximum achievable throughput for a given dataset.

• Classifying a wide range of images with a pretrained ResNet-18

  Residual networks are famous for their image recognition performance. This example lets you use a pretrained ResNet-18 model for image classification.

Kernel benchmarks

• Kernel benchmarking

  Code for benchmarking the performance of some selected neural network layers.

Example applications

• BERT

• Click through rate: Deep Interest Evolution Network

• Click through rate: Deep Interest Network

• CNN Inference including ResNet, MobileNet & EfficientNet

• CNN Training including ResNet, ResNeXt & EfficientNet

• Contrastive Divergence VAE using MCMC methods

• Dynamic Sparsity: Autoregressive Language Modelling

• Dynamic Sparsity: MNIST RigL

• Example reinforcement learning policy model

• Recommendation using Autoencoders

• Sales forecasting example

• YOLO v3 object detection

• UNet Industrial

  This is an image segmentation model for industrial use cases.

• Markov chain Monte Carlo methods

  These are well known techniques for solving integration and optimisation problems in large-dimensional spaces. Their applications include algorithmic trading and computational biology. This example uses the TensorFlow Probability library.

• CosmoFlow

  This is a deep learning model for calculating cosmological parameters. The model primarily consists of 3D convolutions, pooling operations, and dense layers.

• Block sparsity

  Examples of performing block-sparse computations on the IPU in TensorFlow. Includes the Poplar code for two block-sparse custom ops (matrix multiplication and softmax), which are used to construct a simple MNIST example.

TensorFlow 2

Tutorials

• TensorFlow 2 Keras: How to run on the IPU

• Using Infeed and Outfeed Queues in TensorFlow 2

• Using TensorBoard in TensorFlow 2 on the IPU

Feature examples

• CIFAR-10 with IPUEstimator

  This example shows how to train a model to sort images from the CIFAR-10 dataset using the IPU implementation of the TensorFlow Estimator API.

• IMDB Sentiment Prediction

  These examples train an IPU model with an embedding layer and an LSTM to predict the sentiment of an IMDB review.

• Inspecting tensors using custom outfeed layers and a custom optimizer

  This example trains a choice of simple fully connected models on the MNIST numeral dataset and shows how tensors (containing activations and gradients) can be returned to the host via outfeeds for inspection.

• PopDist training example

  This shows how to enable distributed training of a TensorFlow 2 model by using the PopDist API.

• Re-computation Checkpoints

  This example demonstrates using checkpointing of intermediate values to reduce live memory peaks with a simple Keras LSTM model.

Simple application examples

• Simple MNIST training example

  This example trains a simple two-layer fully connected model on the MNIST numeral dataset.

Example applications

• Adversarial Generalized Method of Moments

  This example is an implementation of Adversarial Generalized Method of Moments, an approach for solving statistical problems based on generative adversarial networks with a wide variety of applications.

• Graph Neural Network Example

  This example uses the Spektral GNN library to predict the heat capacity of various molecules in the QM9 dataset.

PyTorch

Tutorials

A set of tutorials to introduce the PyTorch framework support for the IPU.

• Tutorial 1: Basics

• Tutorial 2: Efficient data loading

• Tutorial 3: Mixed precision floating point

• Tutorial 4: Observing tensors in PyTorch

• Tutorial 5: Parallel execution using pipelining

• Fine-tuning BERT with HuggingFace and PopTorch

Feature examples

• Octave Convolutions

  A novel convolutional layer in neural networks. This example shows an implementation of how to train the model and run it for inference.

• Using a custom op in a PyTorch model

  This example describes how to use a custom op created in Poplar for the PopTorch. It does not describe the creation of a custom op with Poplar in C++, only the use of the op within a PyTorch model.

• PopDist training example

  This shows how to enable distributed training of a PyTorch model by using the PopDist API.

Simple application examples

Complete examples of models

• Classifying hand-written digits

  The MNIST dataset is a well-known example of a basic machine learning task.

Pre-trained models

• Hugging Face BERT

  A pre-trained BERT model made available by Hugging Face and which is implemented in PyTorch. This example consists of running one of the pre-trained BERT model on an IPU for an inference session.

Other examples

• PopART MNIST

  An example of how to export a PyTorch model as an ONNX file and reuse this file with Graphcore’s PopART.

Example applications

• BERT

  An implementation of BERT models in PyTorch for the IPU, leveraging the HuggingFace Transformers library.

• CNN Training & Inference including ResNet, ResNeXt & EfficientNet

PopART / ONNX

There are several examples demonstrating how to use the Poplar Advanced Runtime (PopART). These include full models as well as examples of how to use multiple IPUs, implement custom ops in Poplar and other key features provided by PopART.

Feature examples

Multi-IPU examples

• Sharding a Model over Multiple IPUs

  This demo shows how to “shard” (split) a model over multiple IPUs using PopART.

• Pipelining a Model over Multiple IPUs

  This demo shows how to use pipelining in PopART on a very simple model consisting of two dense layers.

• Utilising Streaming Memory with Phased Execution

  This example runs a network in inference mode over two IPUs by splitting it in several execution phases and keeping the weights in Streaming Memory.

Custom operators

• Custom Operators

  This directory contains two example implementations of custom operators for PopART (Cube and LeakyReLU). Both examples create an operation definition with forward and backward parts, and include a simple inference script to demonstrate using the operators.

Other examples

• Data Callbacks

  This example creates a simple computation graph and uses callbacks to feed data and retrieve the results. Time between host-device transfer and receipt of the result on the host is computed and displayed for a range of different data sizes.

• Automatic and Manual Recomputing

  This example shows how to use manual and automatic recomputation in PopART with a seven-layer DNN and generated data.

Simple application examples

• Simple MNIST Examples

  Contains two simple models, one linear and one using convolutions trained on the MNIST dataset.

Kernel benchmarks

• Kernel Synthetic Benchmarks

  Contains synthetic benchmarks for models with two types of layer (LSTM and 2D Convolution) and synthetic data in training and inference.

Example applications

• BERT <https://github.com/graphcore/examples/tree/master/nlp/bert/popart>

• Conformer for speech recognition <https://github.com/graphcore/examples/tree/master/speech/conformer/popart>

  This PopART application implements a speech recognition model using conformer blocks as described in the paper Conformer: Convolution-augmented Transformer for Speech Recognition.

• Deep Voice 3

  This PopART application implements the Deep Voice 3 model for text-to-speech synthesis as described in the paper Deep Voice 3: Scaling Text-to-Speech with Convolutional Sequence Learning.

• ResNext Inference

  ResNeXt is a simple, highly modularized network architecture for image classification. The network is constructed by repeating a building block that aggregates a set of transformations with the same topology.

• Block Sparsity

  Examples of of using PopART to perform block-sparse computations on the IPU, including the Poplar code for two block-sparse custom ops (matrix multiplication and softmax). These are used to construct a number of examples and test scripts, including a simple MNIST example.

PopVision profiling and analysis tools

Tutorials

• Tutorial 1: instrumenting applications and using the PopVision System Analyser

• Tutorial 2: accessing profiling information with libpva

• Poplar Tutorial 4: profiling output with the PopVision Graph Analyser

Poplar and PopLibs

Tutorials and examples that show how to use the Poplar and PopLibs libraries that underlie the other frameworks. Particularly useful if you need to extend the existing frameworks by writing a custom operation, for example.

Tutorials

• Tutorial 1: programs and variables

• Tutorial 2: using PopLibs

• Tutorial 3: writing vertex code

• Tutorial 4: profiling output

• Tutorial 5: matrix-vector multiplication

• Tutorial 6: matrix-vector multiplication optimisation

Feature examples

• Advanced example

  This example performs a simple computation but demonstrates some of the advanced features of Poplar (for example, using dedicated programs for the I/O, using PopLibs, saving/reusing executables, choosing the number of IPUs). It is a good follow-up to what the Poplar tutorials.

• Prefetch

  This example shows how to implement prefetching of the input stream of a Poplar program using a callback, in order to maximise I/O performance.

Simple application examples

• Simple MNIST Examples

  Contains a simple linear model trained on the MNIST dataset.

Example applications

• Monte Carlo ray tracing

  A proof of concept Monte Carlo ray tracing application.