3. Environment preparation
This chapter introduces the host server hardware and operating system, describes how to start an IPU environment with a Docker container and describes how to verify that the IPU is accessible and working correctly.
3.1. Host server CPU architecture
The host server that runs the Poplar graph engine and communicates with IPUs must have one of the following CPU architectures:
Intel
AMD
3.2. Host server operating system
The following operating systems are supported for the host server:
CentOS 7.6
Debian 10
Ubuntu 20.04
3.3. Docker
Recommended Docker version: 20.10.11, build dea9396
Refer to the Docker installation instructions for Ubuntu or CentOS.
3.4. Poplar SDK
Poplar SDK version 3.1 or later is needed.
3.4.1. Installing the Poplar SDK on the host server
Download the Poplar SDK tarball from the Graphcore Downloads portal. You can find the Poplar SDK software in the Poplar tab. You may have to click Load More Packages to find the tarball for your Linux platform.
You can check which OS you are running by using the command:
$ lsb_release -a
The tarball is named as follows:
poplar_sdk-[os]-[poplar_ver].tar.gz
where [os] is the host OS and [poplar_ver] is the SDK software version number.
An example of the Poplar SDK tarball for Ubuntu 18.04 and version 2.6.0 of the SDK is:
poplar_sdk-ubuntu_18_04-2.6.0-33d3efd05d.tar.gz
Install the SDK by unpacking the tarball as follows:
$ tar -xvzf poplar_sdk-[os]-[ver].tar.gz
For details of the components included in the Poplar SDK, refer to the Contents of the SDK section in the Poplar SDK Overview document.
3.5. Inspection of IPU hardware
To list all C600 IPU devices, run:
$ ls -alt /dev/ipu*
Three files are listed for each C600 IPU:
/dev/ipu{id}/dev/ipu{id}_ex/dev/ipu{id}_mem
For example, the following output lists the files corresponding to an C600 IPU device with ID 0.
$ ls -alt /dev/ipu*
crw-rw-rw- 1 root root 486, 0 Sep 26 03:48 /dev/ipu0
crw-rw-rw- 1 root root 486, 1 Sep 26 03:48 /dev/ipu0_ex
crw-rw-rw- 1 root root 486, 2 Sep 26 03:48 /dev/ipu0_mem
Note
The ls -alt /dev/ipu* command returns null when run on an IPU-M2000 or on a Bow-2000.
Note
If no device files are listed with ls -alt /dev/ipu*, contact the system administrator to check whether the IPU hardware and driver are properly installed.
3.6. Install PopRT
As described in Section 2.3.1, Model compilation overview, PopRT is used to compile the model into a PopEF file, so you need to install PopRT. There are two installation methods:
3.6.1. Installation with a Docker container
PopRT provides a container image so you can get started quickly. As mentioned in Section 3.7, Docker containers for the Poplar SDK, we recommended using gc-docker to start the container when the Poplar SDK is installed on the host server. This simplifies mapping the IPU device to the container.
The command to start the container with gc-docker is as follows. If the Poplar SDK is not installed on the host server, refer to Section 3.7.2, Run a Docker container.
$ gc-docker -- --rm \
graphcorecn/poprt-staging:latest \
--help
If the command is executed successfully, the following output will be obtained.
usage: python -m poprt.cli [-h]
[--available_memory_proportion AVAILABLE_MEMORY_PROPORTION]
...
[--skip_passes SKIP_PASSES] [-v]
poprt is a tool for quickly deploying ONNX models on IPUs.
optional arguments:
-h, --help show this help message and exit
...
-v, --version print version
3.6.2. Installation with pip
When you need to integrate PopRT with other tools, install it with pip and ensure that it runs in the environment of the corresponding Poplar SDK version.
$ pip3 install <PopRT_wheel>
For more information about PopRT installation, refer to the PopRT documentation.
3.7. Docker containers for the Poplar SDK
You can pull a container image with the Poplar SDK from Docker Hub. Graphcore provides container images for the Poplar SDK, machine learning frameworks such as TensorFlow and PyTorch, and for tools to interact with IPU devices. You can pull different images to use according to your needs or build new images if required. For more information about container images, refer to the user guide Using IPUs from Docker.
3.7.1. gc-docker
gc-docker is a command line tool in the Poplar SDK, which can provide Docker with IPU device information. For more information about gc-docker, refer to Graphcore Command Line Tools. To use this command, you need to install the Poplar SDK on the host server and enable it with the following command in Bash or Z Shell.
$ source [poplar-installation-path]/enable.sh
If the Poplar SDK is not installed on the host, refer to Section 3.7.2, Run a Docker container.
You can view the generated IPU device information using gc-docker -e. As shown below, the ipu0 device is mapped to the container with -d 0.
$ gc-docker -e -d 0
/usr/bin/docker run --device=/dev/ipu0:/dev/ipu0 --device=/dev/ipu0_ex:/dev/ipu0_ex --device=/dev/ipu0_mem:/dev/ipu0_mem --ipc=host
The following command will start a container and map the ipu0 device to the container. If you need to map other devices, change the -d 0 parameter to the hardware ID assigned by the system administrator.
$ gc-docker -d 0 -- --rm -it \
graphcorecn/poplar-staging:latest
3.7.2. Run a Docker container
We recommend that you use gc-docker to start a Docker container. If the Poplar SDK is not available on the host server, the container can also be started with docker run. In this case, you need to map hardware resources to the container with the --device option. For example:
$ docker run --rm -it \
--network=host \
--device=/dev/ipu0:/dev/ipu0 \
--device=/dev/ipu0_ex:/dev/ipu0_ex \
--device=/dev/ipu0_mem:/dev/ipu0_mem \
graphcorecn/poplar-staging:latest
Note
In the case of testing on an IPU-M2000 or Bow-2000, use --device=/dev/infiniband/, or consult your system administrator.
Note
Contact your system administrator to get the ID of the device available for you to use. Use this ID for the
-dand--deviceparameters in thegc-dockeranddocker runcommands.For IPU resource management in the K8s cluster, refer to .
3.7.3. Query the IPU status from a Docker container
To query the status of the IPU from within a Docker container use:
$ gc-monitor
Listing 3.1 shows an example of the expected output. If an error message is returned, contact the system administrator to confirm that the IPU driver is properly installed.
+---------------+---------------------------------------------------------------------------------------------+
| gc-monitor | Installed driver: 1.1.6 |
+------+--------------------+----------+-------+----------+----+----+----------------+------+------+----------+
| Slot | Serial | ICU FW | Type | Speed | Ln | ID | PCIe ID | IPU# |LnErrS|HostLnErrS|
+------+--------------------+----------+-------+----------+----+----+----------------+------+------+----------+
| 102 | 0063.0063.822391 | 2.6.4 | C600 |16.0 GT/s | 8 | 0 | 0000:35:00.0 | 0 | 0 | - |
+------+--------------------+----------+-------+----------+----+----+----------------+------+------+----------+
| 100 | 0079.0063.822391 | 2.6.4 | C600 |16.0 GT/s | 8 | 1 | 0000:37:00.0 | 0 | 0 | - |
+------+--------------------+----------+-------+----------+----+----+----------------+------+------+----------+
| 0 | 0269.0063.822391 | 2.6.4 | C600 |16.0 GT/s | 8 | 2 | 0000:39:00.0 | 0 | 0 | - |
+------+--------------------+----------+-------+----------+----+----+----------------+------+------+----------+
| 0 | 0189.0063.822391 | 2.6.4 | C600 |16.0 GT/s | 8 | 3 | 0000:3b:00.0 | 0 | 0 | - |
+------+--------------------+----------+-------+----------+----+----+----------------+------+------+----------+
| 106 | 0280.0063.822391 | 2.6.4 | C600 |16.0 GT/s | 8 | 4 | 0000:9c:00.0 | 0 | 0 | - |
+------+--------------------+----------+-------+----------+----+----+----------------+------+------+----------+
| 104 | 0019.0063.822391 | 2.6.4 | C600 |16.0 GT/s | 8 | 5 | 0000:9e:00.0 | 0 | 0 | - |
+------+--------------------+----------+-------+----------+----+----+----------------+------+------+----------+
| 0 | 0224.0063.822391 | 2.6.4 | C600 |16.0 GT/s | 8 | 6 | 0000:a0:00.0 | 0 | 0 | - |
+------+--------------------+----------+-------+----------+----+----+----------------+------+------+----------+
| 0 | 0050.0063.822391 | 2.6.4 | C600 |16.0 GT/s | 8 | 7 | 0000:a2:00.0 | 0 | 0 | - |
+------+--------------------+----------+-------+----------+----+----+----------------+------+------+----------+
+--------------------------------------------------------------------------------------------------+
| No attached processes |
+--------------------------------------------------------------------------------------------------+
Note
Before running any application examples, you can use gc-monitor to check whether there are idle devices. This step is omitted in subsequent examples.