6. Replacing an IPU-Machine in a Pod

These instructions describe how to replace an IPU-Machine in an otherwise functional Pod.

6.1. Diagnosing IPU-Machine problems

On the bottom right of the front of the IPU-Machine are a set of three LED indicators (Fig. 6.1, Fig. 6.2). When the warning LED in the middle is amber, this indicates that a service action is required.

_images/ipu_machine_warning.png

Fig. 6.1 Front LED indicators on the IPU-Machine.

_images/ipu_machine_warning_led_inset.png

Fig. 6.2 When the warning LED (middle) is amber, this indicates that a service action is required.

There are many causes for the amber LED to light up. The details of the cause and what actions need to be taken are given in the system event log. You can access the system event log with the IPMI interface commands.

Note

Not all causes of the amber LED lighting up require the IPU-Machine to be replaced. If the IPU-Machine needs to be replaced, then follow the procedure described from Section 6.2, Ordering a replacement IPU-Machine to Section 6.8, Confirming replacement.

If you need additional help with troubleshooting problems with an IPU-Machine, contact Graphcore Support.

6.2. Ordering a replacement IPU-Machine

See Section 2, Ordering spare or replacement parts for how to order a replacement IPU-Machine.

6.3. Tools needed for the replacement

You need the following to replace an IPU-Machine in a Pod:

  1. A server lift or another person if you do not have a server lift.

6.4. Unplugging cables

  1. From the rear of the IPU-Machine, disconnect both power cables.

  2. From the front of the IPU-Machine, disconnect all cables connecting the faulty IPU-Machine to neighbouring IPU-Machines and to the host server.

6.5. Removing an IPU-Machine from a Pod

To remove an IPU-Machine from the rack:

  1. Prepare an appropriate server lift and adjust the height such that it is suitable for the IPU-Machine sliders. If a lift is not available, then this is a two-person operation.

  2. Unscrew the captive thumb screws at the front of both the inner rack rails (Fig. 6.3).

    _images/m2000_rail_11.png

    Fig. 6.3 Remove the thumb screws at the front

  3. Completely slide out the IPU-Machine on the rails.

  4. Pull on the white tabs (Fig. 6.4) on both sides of the IPU-Machine to release it. Pull the IPU-Machine forwards until it starts sliding out of the outer rails.

    _images/m2000_removal_white_release_tab.jpg

    Fig. 6.4 Location of white release tab

  5. Slide the IPU-Machine onto the server lift, if available, or two people should carry the IPU-Machine.

6.6. Installing IPU-Machine

  1. Pull the sliding rail located within the outer rack rail completely forward such that it locks into the fully extended position (Fig. 6.5).

    _images/m2000_rail_6.png

    Fig. 6.5 IPU-Machine rack rail kit: sliders fully extended

  2. Place the IPU-Machine onto an appropriate server lift and adjust the height such that it is suitable for the sliders (Fig. 6.6). If a lift is not available, this is a two person operation.

    _images/m2000_rail_7.png

    Fig. 6.6 Server lift for IPU-Machine

  3. Slide the protruding inner rails (on the IPU-Machine) into the receiving channel of the extended outer rails (Fig. 6.7).

    _images/m2000_rail_8.png

    Fig. 6.7 Slide IPU-Machine inner rails into outer rails

  4. Whilst the server lift is supporting the full weight of the IPU-Machine (or with two people carrying the IPU-Machine if not using a server lift), slide the IPU-Machine into the extended outer rails until you feel both sides engage a stopping mechanism (Fig. 6.9).

  5. Then, simultaneously pull on the blue tabs for the release mechanism at each side of the IPU-Machine and then push the IPU-Machine unit fully into the rack (Fig. 6.8 and circled in Fig. 6.9).

    _images/m2000_rail_10.png

    Fig. 6.8 Blue tab release mechanism

    _images/m2000_rail_9.png

    Fig. 6.9 Location of blue tab release mechanism

  6. Finally, screw the captive thumb screw into the inner rack rail. (Fig. 6.10).

    _images/m2000_rail_11.png

    Fig. 6.10 Re-attach the IPU-Machine to the inner rack rail by tightening the captive thumb screws”

  7. The IPU-Machine is now installed.

6.7. Connecting cables

  1. On the front of the IPU-Machine, plug in all necessary cables to connect the replacement IPU-Machine to neighbouring IPU-Machines and to the host server.

  2. Connect both the power cables to the rear of the IPU-Machine.

6.8. Confirming replacement

Confirm that the amber warning light on the front of the IPU-Machine is not lit.