Examples and Tutorials
Contents
Overview
Tutorials repository
Simple example programs and tutorials are available in the Graphcore GitHub tutorials repository. This includes:
Tutorials to help you get started using the Poplar SDK and Graphcore tools to run code on the IPU.
Feature examples: small code examples showing you how to use various software features when developing for IPUs.
Simple application examples: basic applications written in different frameworks targeting the IPU.
Kernel benchmarks: code for benchmarking the performance of some selected types of neural network layers on the IPU, using TensorFlow or our PopART framework.
The tutorials repository also contains code used in technical notes, videos and blogs.
Examples repository
Sample applications are available in the Graphcore GitHub examples repository. This includes:
Applications: example applications written in different frameworks targeting the IPU.
Code examples: smaller models and code examples.
TensorFlow 1
The following examples are provided for TensorFlow 1. These include full models as well as examples of how to measure performance, use multiple IPUs and implement custom ops in Poplar, among other things.
These examples are tested against TensorFlow 1. Some of them will run with TensorFlow 2 without any changes.
Tutorials
Feature examples
These include examples of how to measure performance, use multiple IPUs and implement custom ops in Poplar.
Performance
-
This example trains simple pipelined and non-pipelined models on the MNIST numeral data set and shows how tensors (containing activations and gradients) can be returned to the host for inspection using outfeed queues. This can be useful for debugging a model.
-
The MNIST simple application shows how to use
dataset_benchmark()
to determine the maximum achievable throughput for a given dataset.
Using multiple IPUs
Simple examples demonstrating and explaining different ways of using multiple IPUs are provided.
Pipelining and replication are used to parallelise and speed up training, whereas sharding is generally used to simply fit a model in memory. Concurrent pipelining is a variant of pipelining.
-
This shows how to enable distributed training of a TensorFlow model by using the PopDist API.
Custom ops
-
Code that demonstrates how to define your own custom op using Poplar and PopLibs and use it in TensorFlow 1.
Custom op example with gradient
Code that demonstrates how to define your own custom op using Poplar and PopLibs, and use it in TensorFlow 1. Also shows how to define the gradient of your custom op so that you can use automatic differentiation and operations that depend on it, such as the
minimize()
method of an optimizer.
Other examples
-
Example of using the IPU implementation of the TensorFlow Estimator API.
-
A code example which demonstrates how to use
set_ipu_connection_type()
to control if and when the IPU device is acquired.
Simple application examples
Classifying hand-written digits
The MNIST dataset is a well-known example of a basic machine learning task. This is an example of its implementation on IPUs. This example also shows how to use
dataset_benchmark()
to determine the maximum achievable throughput for a given dataset.Classifying a wide range of images with a pretrained ResNet-18
Residual networks are famous for their image recognition performance. This example lets you use a pretrained ResNet-18 model for image classification.
Kernel benchmarks
-
Code for benchmarking the performance of some selected neural network layers.
Example applications
Code examples
-
This is an image segmentation model for industrial use cases.
Markov chain Monte Carlo methods
These are well known techniques for solving integration and optimisation problems in large-dimensional spaces. Their applications include algorithmic trading and computational biology. This example uses the TensorFlow Probability library.
-
This is a deep learning model for calculating cosmological parameters. The model primarily consists of 3D convolutions, pooling operations, and dense layers.
-
Examples of performing block-sparse computations on the IPU in TensorFlow. Includes the Poplar code for two block-sparse custom ops (matrix multiplication and softmax), which are used to construct a simple MNIST example.
TensorFlow 2
Tutorials
Feature examples
-
This example shows how to train a model to sort images from the CIFAR-10 dataset using the IPU implementation of the TensorFlow Estimator API.
-
These examples train an IPU model with an embedding layer and an LSTM to predict the sentiment of an IMDB review.
Inspecting tensors using custom outfeed layers and a custom optimizer
This example trains a choice of simple fully connected models on the MNIST numeral dataset and shows how tensors (containing activations and gradients) can be returned to the host via outfeeds for inspection.
-
This shows how to enable distributed training of a TensorFlow 2 model by using the PopDist API.
Simple application examples
-
This example trains a simple two-layer fully connected model on the MNIST numeral dataset.
Example applications
Adversarial Generalized Method of Moments
This example is an implementation of Adversarial Generalized Method of Moments, an approach for solving statistical problems based on generative adversarial networks with a wide variety of applications.
-
This example uses the Spektral GNN library to predict the heat capacity of various molecules in the QM9 dataset.
PyTorch
Tutorials
A set of tutorials to introduce the PyTorch framework support for the IPU.
Feature examples
-
A novel convolutional layer in neural networks. This example shows an implementation of how to train the model and run it for inference.
Using a custom op in a PyTorch model
This example describes how to use a custom op created in Poplar for the PopTorch. It does not describe the creation of a custom op with Poplar in C++, only the use of the op within a PyTorch model.
-
This shows how to enable distributed training of a PyTorch model by using the PopDist API.
Simple application examples
Complete examples of models
Classifying hand-written digits
The MNIST dataset is a well-known example of a basic machine learning task.
Pre-trained models
-
A pre-trained BERT model made available by Hugging Face and which is implemented in PyTorch. This example consists of running one of the pre-trained BERT model on an IPU for an inference session.
Other examples
-
An example of how to export a PyTorch model as an ONNX file and reuse this file with Graphcore’s PopART.
Example applications
-
An implementation of BERT models in PyTorch for the IPU, leveraging the HuggingFace Transformers library.
CNN Training & Inference including ResNet, ResNeXt & EfficientNet
PopART / ONNX
There are several examples demonstrating how to use the Poplar Advanced Runtime (PopART). These include full models as well as examples of how to use multiple IPUs, implement custom ops in Poplar and other key features provided by PopART.
Feature examples
Multi-IPU examples
Sharding a Model over Multiple IPUs
This demo shows how to “shard” (split) a model over multiple IPUs using PopART.
Pipelining a Model over Multiple IPUs
This demo shows how to use pipelining in PopART on a very simple model consisting of two dense layers.
Utilising Streaming Memory with Phased Execution
This example runs a network in inference mode over two IPUs by splitting it in several execution phases and keeping the weights in Streaming Memory.
Custom operators
-
This directory contains two example implementations of custom operators for PopART (Cube and LeakyReLU). Both examples create an operation definition with forward and backward parts, and include a simple inference script to demonstrate using the operators.
Other examples
-
This example creates a simple computation graph and uses callbacks to feed data and retrieve the results. Time between host-device transfer and receipt of the result on the host is computed and displayed for a range of different data sizes.
Automatic and Manual Recomputing
This example shows how to use manual and automatic recomputation in PopART with a seven-layer DNN and generated data.
Simple application examples
-
Contains two simple models, one linear and one using convolutions trained on the MNIST dataset.
Kernel benchmarks
-
Contains synthetic benchmarks for models with two types of layer (LSTM and 2D Convolution) and synthetic data in training and inference.
Example applications
Conformer for speech recognition
This PopART application implements a speech recognition model using conformer blocks as described in the paper Conformer: Convolution-augmented Transformer for Speech Recognition.
-
This PopART application implements the Deep Voice 3 model for text-to-speech synthesis as described in the paper Deep Voice 3: Scaling Text-to-Speech with Convolutional Sequence Learning.
-
ResNeXt is a simple, highly modularized network architecture for image classification. The network is constructed by repeating a building block that aggregates a set of transformations with the same topology.
Code examples
-
Examples of of using PopART to perform block-sparse computations on the IPU, including the Poplar code for two block-sparse custom ops (matrix multiplication and softmax). These are used to construct a number of examples and test scripts, including a simple MNIST example.
Poplar and PopLibs
Tutorials and examples that show how to use the Poplar and PopLibs libraries that underlie the other frameworks. Particularly useful if you need to extend the existing frameworks by writing a custom operation, for example.
Tutorials
Feature examples
-
This example performs a simple computation but demonstrates some of the advanced features of Poplar (for example, using dedicated programs for the I/O, using PopLibs, saving/reusing executables, choosing the number of IPUs). It is a good follow-up to what the Poplar tutorials.
-
This example shows how to implement prefetching of the input stream of a Poplar program using a callback, in order to maximise I/O performance.
Simple application examples
-
Contains a simple linear model trained on the MNIST dataset.
Example applications
-
A proof of concept Monte Carlo ray tracing application.