4. Physical mounting

4.1. Preparing the rack

4.1.1. Unit orientation

The airflow for the IPU-M2000s is from the edge containing the network ports to the edge containing the fans. Therefore, the network ports face the cold aisle.

The airflow for the R6525 server is from the edge containing bezel and display to the edge containing the network ports. Therefore, the network ports face the hot aisle.

4.1.2. Rack preparation

The IPU-M2000 mounting system requires a rail-to-rail distance of 720mm. Details of this adjustment will depend on the choice of rack.

The vertical accessory channels should be positioned at the very front and very rear of the rack. If necessary, move these from their shipping positions.

_images/m2000_da_rack_prep.png

Fig. 4.1 Rack preparation

4.1.3. Adjusting front and rear vertical rails

The rear vertical rack rails should be positioned such that there is 20mm of distance between the rear face of the vertical rack rail and the racks rear frame. This should result in a square symbol being visible through the alignment window at the top and bottom of the rail. Details of this adjustment will depend on the choice of rack.

The front vertical rack rails should be positioned tight against the front vertical cable organisers such that only a single diamond symbol is visible through the alignment window at the top and bottom of the rail.

4.1.4. Installing the IPU-M2000 rack rails

The IPU-M2000 rail kit comprises two mated inner and outer rack rails and an accessory bag containing screws. The inner rail fixes to the body of the IPU-M2000 and the outer rail fixes to the vertical rack rails in the server cabinet.

  1. Separate the mated inner and outer rails:

    1. Fully extend the rails by pulling on the end which has the captive thumb screw attached:

      _images/m2000_rails_1.png
    2. Whilst pulling on the thumb screw end of the rails, push the white plastic release tab towards the thumb screw end:

      _images/m2000_rails_2.png
    3. The inner and outer rails will now separate:

      _images/m2000_rails_3.png
    4. Repeat these steps for the number of IPU-M2000s to be installed.

  2. Fix the inner rail to the body of the IPU-M2000:

  1. Offer up the inner rail to the side of the IPU-M2000 and ensure that all fixing pins are sitting within the enlarged opening of the retention channel. The inner rails are the thinner of the two separated rails and have a captive thumb screw at one end. The inner rail should be oriented such that the captive thumb screw end is at the end of the IPU-M2000 containing the network ports. The inner rails are mirrored and are not handed so the procedure for inner rail fixing is the same for both the left- and right-hand inner rails.

_images/m2000_rails_4.png
  1. Push the inner rail towards the front of the IPU-M2000 (containing the network ports): you should hear a click as the latching mechanism locks behind the head of a fixing pin.

_images/m2000_rails_5.png
  1. Ensure all fixing pins are correctly engaged with their respective retention channel.

  2. Locate the four flat head fixing screws from the rack rail accessory bag:

_images/m2000_rails_6.png
  1. Using the above screws, fix the inner rail to the body of the IPU-M2000:

_images/m2000_rails_7.png

The inner rails are now securely affixed to the IPU-M2000 body.

  1. Repeat these steps for the number of IPU-M2000s to be installed.

  1. Place the outer rails to one side for later use:

_images/m2000_rails_8.png

4.2. Installing the equipment

The following sections describe the installation of the IPU-M2000s and server into the rack.

In the case of an IPU‑POD4 DA system, one IPU-M2000 is installed. If subsequently you plan to expand to an IPU‑POD16 DA system, you may wish to leave enough free rack space (3 rows) between the IPU-M2000 and the server.

In the case of an IPU‑POD16 DA system, four IPU-M2000s are installed. All IPU-M2000s and the server should be adjacent to each other in the rack.

The installation of PDUs is not covered in this guide since that will depend upon the PDUs selected.

4.2.1. Installing the IPU-M2000s

Earlier in the guide we fixed the inner rack rails to the IPU-M2000 body. We now need to install the outer rack rails into the rack and install the IPU-M2000s.

  1. Install the outer rack rails into the rack

It is possible to identify the front and rear of the outer rail by finding the large metal latching mechanism – this is to be located at the rear of the rack.

_images/m2000_install_1.png

The outer rail is also embossed with the text “FRONT” at the front end of the rail.

For each rack position where there is to be an IPU-M2000 installed, perform the steps below with both the left-hand and right-hand outer rack rails:

  1. Pull on each end of the outer rail to adjust the rail length to suite your rack

  2. Locate the front end of the outer rail and hold it behind the square holes in the vertical rack rail for your installation. Pull the outer rail towards the vertical rack rail and the latching mechanism will click and hold the outer rail in place:

    _images/m2000_install_2.png
  3. Locate the rear end of the outer rail and slightly open the large metal latch, then press the upper and lower locating pins into the square holes. Release the large metal latch and the outer rail will now be secured to the vertical rack rail:

    _images/m2000_install_3.png
  4. Included in the rack rail accessory bag are two screws and two washers. One screw with one washer should be screwed through the vertical rack rail and into the outer rack rail threaded hole. The washer should be used in such a way that the washer sits flush with the head of the screw – like a cup.

    This should be repeated for both outer rack rails.

    _images/m2000_install_4.png
    _images/m2000_install_5.png
  1. To install an IPU-M2000 into the rails

    1. Pull the sliding rail located within the outer rack rail completely forward such that it locks into the fully extended position:

      _images/m2000_install_6.png
    2. Place the IPU-M2000 onto an appropriately suited server lift and adjust the height such that it is suitable for the sliders. If a lift is not available, then this is a two-person operation.

      _images/m2000_install_7.png
    3. Slide the protruding inner rails into the receiving channel of the extended outer rails:

      _images/m2000_install_8.png
    4. Whilst the server lift is supporting the full weight of the IPU-M2000, slide the IPU-M2000 into the extended outer rails until you feel both sides engage a stopping mechanism.

    5. Simultaneously pull on the blue tabs for the release mechanism at each side of the IPU-M2000 and then push the IPU-M2000 fully into the rack:

      _images/m2000_install_9.png
      _images/m2000_install_10.png
    6. Screw the captive thumb screw into the inner rack rail:

      _images/m2000_install_11.png
    7. Repeat these steps for the number of IPU-M2000s to be installed.

4.2.2. Installing the host server

This section describes the installation of a Dell RG525 server. Other approved and qualified servers are available. To find out more speak to Graphcore sales or your Graphcore channel partner for options.

The server should be installed above the IPU-M2000(s).

Note

The R6525 is installed such that the rear of the server (containing the management and data ports) is the opposite side of the rack to the ports on the front of the IPU-M2000(s). This is due to airflow direction.

  1. Remove and discard the cable management arm brackets from the rear of each tool-less sliding rail.

    _images/server_install_1.png
  2. Install the tool-less sliding rail kit(s) in the required location.

    _images/server_install_2.png
  3. Pull out the rail and fit the server to the rail ensuring the T pins on the side of the server locate in the slots on the rail. Ensure that the power supplies on the server face the rear of the rack.

    Note

    Use an appropriate server lift or have two people installing the servers to ensure correct fitting

  4. Push the server gently from the front to lock it into the slides then press the tab on the side of the slides and push the server fully home in the rack. Repeat the above process for each server if installing multiple servers.

  5. Remove the Velcro tape from the light pipes on the rear of the servers.

  6. Remove the small plastic tab on the left front side of the server bezel and clip the bezel in place on the front of the server ensuring the connection pins on the right-hand side of the bezel line up with the connector on the server, as shown below:

    _images/server_install_3.png

4.2.3. Installing the PDUs

The method for installing the PDUs will depend on the choice of PDU type and location. If the PDUs are to be installed horizontally within the rack, the recommendation is that these are positioned beneath the lowest IPU-M2000. Allowing some space between the lowest IPU-M2000 and the PDUs may make power cabling easier.

4.3. Cabling the system

The following sections detail the wiring of the IPU-POD DA system within the rack. The cabling is very straightforward, as indicated by diagrams of the completed systems in Fig. 4.2 and Fig. 4.3. The following sections take you through wiring up each group of cables in turn.

_images/m2000_da_x1_cabling.png

Fig. 4.2 IPU‑POD4 DA system

_images/m2000_da_x4_cabling.png

Fig. 4.3 IPU‑POD16 DA system

4.3.3. IPU-M2000 to IPU-M2000 management cabling (Cat5e)

Note

This step is only required for IPU‑POD16 DA systems.

There are 2 Cat5e management ports in the middle of each IPU-M2000.

_images/m2000_mng_ports.png

Fig. 4.8 IPU-M2000 Cat5e management ports

Using 0.3m blue Ethernet Cat5e cables, link the top management port (2) of each IPU-M2000 to the bottom management port (1) of the IPU-M2000 that is installed directly above (see Table 4.3 and Fig. 4.9). The top management port (2) of the top-most IPU-M2000 (#4) and the bottom management port (1) of the bottom-most IPU-M2000 (#1), are left unconnected.

Table 4.3 IPU-M2000 management port mapping

From IPU-M2000 management connections

To IPU-M2000 management connections

Cables

IPU-M2000 # 3 management ports 2

IPU-M2000 # 4 management ports 1

Cat5e 0.3m blue

IPU-M2000 # 2 management ports 2

IPU-M2000 # 3 management ports 1

Cat5e 0.3m blue

IPU-M2000 # 1 management ports 2

IPU-M2000 # 2 management ports 1

Cat5e 0.3m blue

The figure below shows the IPU-M2000 management cabling for an IPU‑POD16 DA system.

_images/m2000_mng_cabling.png

Fig. 4.9 IPU‑POD16 DA system (IPU-M2000 management cabling)

4.3.4. IPU-M2000 to server management cabling (Cat5e)

Note

This step is required for both IPU‑POD4 and IPU‑POD16 DA systems.

There are 2 Cat5e management ports in the middle of each IPU-M2000.

_images/m2000_mng_ports.png

Fig. 4.10 IPU-M2000 Cat5e management ports

Using a 1.5m blue Ethernet Cat5e cable, link the top management port (port #2) of the top IPU-M2000 to management port #1 of the R6525 server, as given in Table 4.4.

Table 4.4 IPU-M2000 management port mapping

From IPU-M2000 management connection

To server management connection

Cables

Top IPU-M2000 management port 2

R6525 server management port 1

Cat5e 1.5m blue

Fig. 4.11 and Fig. 4.12 show the IPU-M2000 to server management cabling for an IPU‑POD4 DA system and an IPU‑POD16 DA system respectively.

_images/m2000_da_x1_server.png

Fig. 4.11 IPU‑POD4 DA system (IPU-M2000 to server management cabling)

_images/m2000_da_x4_server.png

Fig. 4.12 IPU‑POD16 DA system (IPU-M2000 to server management cabling)

4.3.5. IPU-M2000 to server data cabling (QSFP)

Note

This step is required for both IPU‑POD4 and IPU‑POD16 DA systems.

There are two QSFP RNIC data ports on the left of each IPU-M2000.

_images/m2000_RNIC_ports.png

Fig. 4.13 IPU-M2000 RNIC ports

Only one of these (port #2) should be connected from each IPU-M2000 to the server with 1.5m QSFP cables, as shown in Table 4.5. In an IPU‑POD4 DA system, only IPU-M2000 #1 is present.

Table 4.5 IPU-M2000 RNIC port mapping

IPU-M2000 RNIC port

Server port

Cables

IPU-M2000 # 4 port 2

Server NIC slot 2, port 2

QSFP 1.5m

IPU-M2000 # 3 port 2

Server NIC slot 2, port 1

QSFP 1.5m

IPU-M2000 # 2 port 2

Server NIC slot 3, port 2

QSFP 1.5m

IPU-M2000 # 1 port 2

Server NIC slot 3, port 1

QSFP 1.5m

Fig. 4.14 and Fig. 4.15 show the IPU-M2000 data cabling for both IPU‑POD4 DA and IPU‑POD16 DA systems.

_images/m2000_da_x1_server_data.png

Fig. 4.14 IPU‑POD4 DA system (IPU-M2000 data cabling)

_images/m2000_da_x4_server_data.png

Fig. 4.15 IPU‑POD16 DA system (IPU-M2000 data cabling)

Note

IPU-M2000 #1 (the lowest in the stack) is always connected to Port 1 in server slot 3.

4.4. Power cabling

The method for cabling the PDUs will depend on the choice of PDU type and location. Each IPU-M2000 has two power inputs and these should be supplied from separate power distribution units for redundancy.

4.5. Completing the rack

The following steps describe completing the rack: fitting blanking panels and re-installing the doors and side panels, if required.

4.5.1. Blanking panels

For proper airflow it is recommended to install 1U blanking panels in every unoccupied rack slot at the front of the rack.

_images/blanking_panels.png

Fig. 4.16 1U blanking panels

4.5.2. Rack completion

Re-install the front and rear doors. Ensure the earth cables are reconnected to the cable on the rack. Re-install the top and bottom side panels on each side of the rack.