Note: Searching from the top-level index page will search all documents. Searching from a specific document will search only that document.
Find an exact phrase: Wrap your search phrase in "" (double quotes) to only get results where the phrase is exactly matched. For example "PyTorch for the IPU" or "replicated tensor sharding"
Prefix query: Add an * (asterisk) at the end of any word to indicate a prefix query. This will return results containing all words with the specific prefix. For example tensor*
Fuzzy search: Use ~N (tilde followed by a number) at the end of any word for a fuzzy search. This will return results that are similar to the search word. N specifies the “edit distance” (fuzziness) of the match. For example Polibs~1
Words close to each other:~N (tilde followed by a number) after a phrase (in quotes) returns results where the words are close to each other. N is the maximum number of positions allowed between matching words. For example "ipu version"~2
Logical operators. You can use the following logical operators in a search:
+ signifies AND operation
| signifies OR operation
- negates a single word or phrase (returns results without that word or phrase)
The airflow for the IPU-M2000s is from the edge containing the network ports to the edge containing the fans. Therefore, the network ports face the cold aisle.
The airflow for the R6525 server is from the edge containing bezel and display to the edge containing the network ports. Therefore, the network ports face the hot aisle.
The IPU-M2000 mounting system requires a rail-to-rail distance of 720mm. Details of this adjustment will depend on the choice of rack.
The vertical accessory channels should be positioned at the very front and very rear of the rack. If necessary, move these from their shipping positions.
The rear vertical rack rails should be positioned such that there is 20mm of distance between the rear face of the vertical rack rail and the racks rear frame. This should result in a square symbol being visible through the alignment window at the top and bottom of the rail. Details of this adjustment will depend on the choice of rack.
The front vertical rack rails should be positioned tight against the front vertical cable organisers such that only a single diamond symbol is visible through the alignment window at the top and bottom of the rail.
The IPU-M2000 rail kit comprises two mated inner and outer rack rails and an accessory bag containing screws. The inner rail fixes to the body of the IPU-M2000 and the outer rail fixes to the vertical rack rails in the server cabinet.
Separate the mated inner and outer rails:
Fully extend the rails by pulling on the end which has the captive thumb screw attached:
Whilst pulling on the thumb screw end of the rails, push the white plastic release tab towards the thumb screw end:
The inner and outer rails will now separate:
Repeat these steps for the number of IPU-M2000s to be installed.
Fix the inner rail to the body of the IPU-M2000:
Offer up the inner rail to the side of the IPU-M2000 and ensure that all fixing pins are sitting within the enlarged opening of the retention channel. The inner rails are the thinner of the two separated rails and have a captive thumb screw at one end. The inner rail should be oriented such that the captive thumb screw end is at the end of the IPU-M2000 containing the network ports. The inner rails are mirrored and are not handed so the procedure for inner rail fixing is the same for both the left- and right-hand inner rails.
Push the inner rail towards the front of the IPU-M2000 (containing the network ports): you should hear a click as the latching mechanism locks behind the head of a fixing pin.
Ensure all fixing pins are correctly engaged with their respective retention channel.
Locate the four flat head fixing screws from the rack rail accessory bag:
Using the above screws, fix the inner rail to the body of the IPU-M2000:
The inner rails are now securely affixed to the IPU-M2000 body.
Repeat these steps for the number of IPU-M2000s to be installed.
The following sections describe the installation of the IPU-M2000s and server into the rack.
In the case of an IPU‑POD4 DA system, one IPU-M2000 is installed. If subsequently you plan to expand to an IPU‑POD16 DA system, you may wish to leave enough free rack space (3 rows) between the IPU-M2000 and the server.
In the case of an IPU‑POD16 DA system, four IPU-M2000s are installed. All IPU-M2000s and the server should be adjacent to each other in the rack.
The installation of PDUs is not covered in this guide since that will depend upon the PDUs selected.
Earlier in the guide we fixed the inner rack rails to the IPU-M2000 body. We now need to install the outer rack rails into the rack and install the IPU-M2000s.
Install the outer rack rails into the rack
It is possible to identify the front and rear of the outer rail by finding the large metal latching mechanism – this is to be located at the rear of the rack.
The outer rail is also embossed with the text “FRONT” at the front end of the rail.
For each rack position where there is to be an IPU-M2000 installed, perform the steps below with both the left-hand and right-hand outer rack rails:
Pull on each end of the outer rail to adjust the rail length to suite your rack
Locate the front end of the outer rail and hold it behind the square holes in the vertical rack rail for your installation. Pull the outer rail towards the vertical rack rail and the latching mechanism will click and hold the outer rail in place:
Locate the rear end of the outer rail and slightly open the large metal latch, then press the upper and lower locating pins into the square holes. Release the large metal latch and the outer rail will now be secured to the vertical rack rail:
Included in the rack rail accessory bag are two screws and two washers. One screw with one washer should be screwed through the vertical rack rail and into the outer rack rail threaded hole. The washer should be used in such a way that the washer sits flush with the head of the screw – like a cup.
This should be repeated for both outer rack rails.
To install an IPU-M2000 into the rails
Pull the sliding rail located within the outer rack rail completely forward such that it locks into the fully extended position:
Place the IPU-M2000 onto an appropriately suited server lift and adjust the height such that it is suitable for the sliders. If a lift is not available, then this is a two-person operation.
Slide the protruding inner rails into the receiving channel of the extended outer rails:
Whilst the server lift is supporting the full weight of the IPU-M2000, slide the IPU-M2000 into the extended outer rails until you feel both sides engage a stopping mechanism.
Simultaneously pull on the blue tabs for the release mechanism at each side of the IPU-M2000 and then push the IPU-M2000 fully into the rack:
Screw the captive thumb screw into the inner rack rail:
Repeat these steps for the number of IPU-M2000s to be installed.
This section describes the installation of a Dell RG525 server. Other approved and qualified servers are available. To find out more speak to Graphcore sales or your Graphcore channel partner for options.
The server should be installed above the IPU-M2000(s).
Note
The R6525 is installed such that the rear of the server (containing the management and data ports) is the opposite side of the rack to the ports on the front of the IPU-M2000(s). This is due to airflow direction.
Remove and discard the cable management arm brackets from the rear of each tool-less sliding rail.
Install the tool-less sliding rail kit(s) in the required location.
Pull out the rail and fit the server to the rail ensuring the T pins on the side of the server locate in the slots on the rail. Ensure that the power supplies on the server face the rear of the rack.
Note
Use an appropriate server lift or have two people installing the servers to ensure correct fitting
Push the server gently from the front to lock it into the slides then press the tab on the side of the slides and push the server fully home in the rack. Repeat the above process for each server if installing multiple servers.
Remove the Velcro tape from the light pipes on the rear of the servers.
Remove the small plastic tab on the left front side of the server bezel and clip the bezel in place on the front of the server ensuring the connection pins on the right-hand side of the bezel line up with the connector on the server, as shown below:
The method for installing the PDUs will depend on the choice of PDU type and location. If the PDUs are to be installed horizontally within the rack, the recommendation is that these are positioned beneath the lowest IPU-M2000. Allowing some space between the lowest IPU-M2000 and the PDUs may make power cabling easier.
The following sections detail the wiring of the IPU-POD DA system within the rack. The cabling is very straightforward, as indicated by diagrams of the completed systems in Fig. 4.2 and Fig. 4.3. The following sections take you through wiring up each group of cables in turn.
Using 0.3M OSFP cables, link the top row of four IPU-link ports (5-8) to the bottom row of four IPU-link ports (1-4) in the IPU-M2000 that is installed directly above (see Table 4.1 and Fig. 4.5). The top row (5-8) of the top-most IPU-M2000 (#4) and the bottom row (1-4) of the bottom-most IPU-M2000 (#1), are left unconnected.
Before attempting to install the OSFP cables, it is beneficial to manipulate the cable to form a tight loop such that the white side of the connector tabs face away from each other. During manufacture and shipping, the cables can form quite a stiff shape, so manipulating the cables before installing them reduces stresses on the socket during install.
After installing a cable, pull gently on the black cable to ensure the plugs are firm in the sockets on the IPU-M2000.
Note
The white tab is on the top of the cable when inserted into the IPU-M2000.
Using 0.15m red Ethernet Cat5e cables, link the top two central Sync-Link ports (6-7) of each IPU-M2000 to the bottom two central Sync-Link ports (2-3) of the IPU-M2000 that is installed directly above (see Table 4.2 and Fig. 4.7). The top row (5-8) of the top-most IPU-M2000 (#4) and the bottom row (1-4) of the bottom-most IPU-M2000 (#1), are left unconnected.
Using 0.3m blue Ethernet Cat5e cables, link the top management port (2) of each IPU-M2000 to the bottom management port (1) of the IPU-M2000 that is installed directly above (see Table 4.3 and Fig. 4.9). The top management port (2) of the top-most IPU-M2000 (#4) and the bottom management port (1) of the bottom-most IPU-M2000 (#1), are left unconnected.
Using a 1.5m blue Ethernet Cat5e cable, link the top management port (port #2) of the top IPU-M2000 to management port #1 of the R6525 server, as given in Table 4.4.
Only one of these (port #2) should be connected from each IPU-M2000 to the server with 1.5m QSFP cables, as shown in Table 4.5. In an IPU‑POD4 DA system, only IPU-M2000 #1 is present.
The method for cabling the PDUs will depend on the choice of PDU type and location. Each IPU-M2000 has two power inputs and these should be supplied from separate power distribution units for redundancy.
Re-install the front and rear doors. Ensure the earth cables are reconnected to the cable on the rack. Re-install the top and bottom side panels on each side of the rack.
