5. Users and allocations

Before users can access the IPU hardware, you need to create a user account and allocate it some IPUs to use.

5.1. Users

A user account has access to the user-level APIs provided by the V-IPU controller. When you initialise the V-IPU controller in secure mode (see Section 4, Securing the installation), this creates a default admin user and assigns an access key.

You can add new users using the create user command:

$ vipu-admin create user alice
create user (alice): success. Access key is '685X_K4uCzN'.
vipu configuration file for the user 'alice' written: alice.vipu-cli.hcl
Use 'vipu --config alice.vipu-cli.hcl' to switch to the user context.

You can restrict users to particular allocations using the --allocations option in the create user command (see Section 5.2, Allocations for more details).

Users that are assigned to the same allocation have access to all partitions in this allocation. Therefore it is important that the users of the same allocation have a trust relationship. A typical allocation could be a team within an organization and the users are team members.

After you have created users in the V-IPU controller, they can access the V-IPU controller using the vipu command:

$ vipu list partitions --api-user-id alice --api-access-key '685X_K4uCzN'

In order to avoid using keys on the command-line you can also create a configuration file, which can be used with the --config option:

$ vipu --config alice.vipu-cli.hcl list partitions

Warning

For backward compatibility with older V-IPU installations, V-IPU controller doesn’t authenticate users unless vipu-server is started with the --secure mode flag. This means that all users have access to all allocations. With V-IPU 1.18, this behaviour will change and users will have to be created explicitly.

For more information on user-level operations such as creating partitions and running applications, see the Virtual-IPU User Guide.

5.2. Allocations

An allocation entity makes IPU-Machines available to create partitions. Therefore, a cluster must have at least one allocation in order to create partitions in it. The IPU-Machines in an allocation must come from a single cluster. Each IPU-Machine can only be assigned to one allocation.

The partitions on different allocations are fully isolated. The physical links between IPUs in partitions in different allocations are untrained. A user can only access information about allocations and partitions in their assigned allocations.

The size of an allocation is a multiple of IPU-Machines, that is, a multiple of four IPUs. When the allocation is contained within a single logical rack, such as a Bow Pod64, the valid sizes are 4, 8, 16, 32 and 64 IPUs. When the allocation spans multiple logical racks, the valid sizes are multiples of 64. For example, in a Bow Pod256 which has four Bow Pod64 logical racks, the valid sizes would be 64, 128, 192 and 256.

Allocations are uniquely identified by their name. You can create an allocation called “project1” in a cluster by specifying the number of IPUs, as follows:

$ vipu-admin create allocation project1 --cluster=cl0 --size=16

The size can’t be greater than the size of the specified cluster.

Allocations are persistent. You can list or remove them but you can’t change the IPUs assigned to an allocation.

You can assign users to one or more allocations using the --allocations option when creating a user:

$ vipu-admin create user alice --allocations project1,project2

By default, the Virtual-IPU controller will attempt to assign IPU-Machines to the requested allocation based on the requirements given. You can also explicitly choose IPU-Machines to add to the allocation.

You can choose the IPU-Machines to add to the allocation by specifying the index that identifies a group of IPU-Machines. To fetch and display the available choices for a particular allocation size, use the create allocation --options command. Note that the list of the available allocation placements for the given size is cached at the client side so the list must first be fetched before creating an allocation using the index. The example below shows the available allocation placements of 8 IPUs within a 16 IPU cluster:

$ vipu create allocation project1 --options --size 4
 Index       | Cluster | Size | Agents
--------------------------------------
 0           | cl00    | 4    | ipum-0
             |         |      | ipum-1
 1           | cl00    | 4    | ipum-2
             |         |      | ipum-3
--------------------------------------

Only valid allocation placement options are shown. The options for an allocation size are calculated using an allocation mapping algorithm on the V-IPU controller. The algorithm ensures that the selected IPUs satisfy the topology and routing requirements of the allocation size. In addition, restrictions are in place to make sure that IPU fragmentation is avoided as much as possible while still enabling manual IPU selection.

You can create an allocation with a specific selection of IPU-Machines as follows:

$ vipu create allocation project1 --size 4 --index 1

This selects the two IPU-Machines ipum-2 and ipum-3 with the index 1, from the table above.

When creating a partition, you can indicate which allocation the IPUs are taken from:

$ vipu-admin create partitions app1 --allocation project1 --size 16

If you don’t specify an allocation, the partition will be created in one of the allocations that can fit the desired partition size.

As administrator, you can use the list partitions command to list all the partitions that users have created in the system.

$ vipu-admin list partitions

 Cluster     | Allocation | Partition | ILDs | GW Routing | ILD Routing | Size | GCDs | State
-----------------------------------------------------------------------------------------------
 cl0         | project1   | part0     | 1    | N/A        | DNC         | 8    | 1    | ACTIVE
 cl0         | project2   | part1     | 1    | N/A        | DNC         | 8    | 1    | ACTIVE
-----------------------------------------------------------------------------------------------

Note that when end-users use this command, they are only able to list the partitions within the allocations that they have been assigned:

$ vipu list partitions --api-user-id alice --api-access-key '685X_K4uCzN'

 Cluster     | Allocation | Partition | ILDs | GW Routing | ILD Routing | Size | GCDs | State
-----------------------------------------------------------------------------------------------
 cl0         | project1   | part0     | 1    | N/A        | DNC         | 8    | 1    | ACTIVE
-----------------------------------------------------------------------------------------------