4. Supported types

This section provides more information about the types supported on the IPU. See also, Section 5, Vertex vector types.

4.1. Scalar types

The scalar types supported by Poplar are shown in Table 4.1. In addition:

  • By default the char type is signed.

  • long is the same as int.

  • long double is the same as double.

  • The underlying type of an enumerated type is int.

  • Function pointers are the same as data pointers.

  • Only a subset of types are supported as fields of classes derived from the Vertex or MultiVertex classes. See Section 3.2.2, Allowed field types as vertex state for more information.

Table 4.1 Scalar data types

Type

Size (bits)

Align (bits)

Meaning

char

8

8

Character type

short

16

16

Short integer

int

32

32

Integer

long

32

32

Integer

long long

64

64

Integer

quarter

8

8

8-bit float

half

16

16

16-bit IEEE float

float

32

32

32-bit IEEE float

double

64

64

64-bit IEEE float

long double

64

64

64-bit IEEE float

void *

32

32

Data pointer

4.2. Floating point types

The IPU has hardware support for float (32 bit) and half (16 bit) operations. There is also limited support for operations on quarter (8-bit floats).

You can indicate the type of a floating-point literal with the appropriate suffix. This extends the standard C++ for floating point literals.

  • half literals are indicated with _h; for example, 3.14_h

  • float literals are indicated with f or F

  • long double literals are indicated with l or L

  • If a suffix is not specified, the value has the type double

Warning

Although the compiler supports double and long double, there is no hardware support for 64-bit floating-point operations and so any calculations will be implemented in software. Therefore, you should be careful to avoid default promotions to double.

4.2.1. Half on the IPU

The IPU instruction set does not support operations on scalar half values, only vectors.

If operations on half values are not vectorised, either explicitly by the user (see Section 4.3, Vector types) or the compiler, then they may be promoted to float. The compiler will use vector operations, wherever possible for half types. If not, it will either promote the values to float, or broadcast a single half value to a half2 so it can use vector operations, then discard one half of the vector.

4.2.2. Quarter on the IPU

Poplar has support for 8-bit floating point data. These formats are mainly used to reduce storage requirements. Most arithmetic, apart from some convolutions, is done by casting the values to and from half.

There are two 8-bit float data formats, F143 and F152. These names refer to the number of bits used to represent the sign, exponent and mantissa. As with other floating point representations, the exponent is subject to a bias. This bias is different for each of the two formats as shown in Table 4.2.

The format for the values in the tensor are defined by poplar::QuarterMetadata. A single QuarterMetadata can be used for all the values in one or more tensors.

Table 4.2 Float8 formats in Poplar

QuarterMetadata Format

Sign bits

Exponent bits

Mantissa bits

Exponent bias

Smallest positive value

Largest positive value

F143

1

4

3

-8

2-10

240.0

F152

1

5

2

-16

2-17

57,344.0

More details about the numerical properties of these two 8-bit floating point data types can be found in arXiv paper 8-Bit Numerical Formats for Deep Neural Networks.

Because of the limited numerical range of 8-bit floating point numbers, the metadata also defines a scaling parameter, fp8Scale. The scale parameter must be in the range -31 to 31. Internally, the 8-bit floating point data is multiplied by pow2(fp8Scale). You can use a positive fp8Scale to accommodate large numerical ranges or you can use negative values for smaller numerical ranges.

4.2.3. Half and quarter on the CPU

For CPU targets, half is, by default, an alias for float and sizeof(half) will be 4.

The parameter accurateHalf can be set to true when creating a CPU or IPU Model target, in which case half will be correctly implemented as 16-bit IEEE floating point. This will be slower, but will produce the same results as the IPU.

Codelets should be written to be generic to the size of half so that changing this setting requires no code changes.

For CPU targets, quarter is stored in one byte and sizeof(quarter) will be 1.

4.3. Vector types

Poplar provides a number of vector types for representing short vectors of most scalar types. The supported vector types are shown in Table 4.3. Only the floating point vectors have direct support in the IPU instruction set.

Table 4.3 Vector data types

Type

Size (bits)

Align (bits)

Meaning

iset

char2

16

16

Vector of 2 char values

×

uchar2

16

16

Vector of 2 unsigned char values

×

char4

32

32

Vector of 4 char values

×

uchar4

32

32

Vector of 4 unsigned char values

×

int2

64

64

Vector of 2 int values

×

uint2

64

64

Vector of 2 unsigned int values

×

int4

128

128

Vector of 4 int values

×

uint4

128

128

Vector of 4 unsigned int values

×

long2

64

64

Vector of 2 long values

×

long4

128

128

Vector of 4 long values

×

longlong2

128

128

Vector of 2 long long values

×

longlong4

256

256

Vector of 4 long long values

×

short2

32

32

Vector of 2 short values

×

ushort2

32

32

Vector of 2 unsigned short values

×

short4

64

64

Vector of 4 short values

×

ushort4

64

64

Vector of 4 unsigned short values

×

float2

64

64

Vector of 2 float values

float4

128

128

Vector of 4 float values

half2

32

32

Vector of 2 half values

half4

64

64

Vector of 4 half values

The vector types are defined with the vector extensions defined by GCC and Clang, using the vector_size variable attribute.

4.4. Structure types

Structure types pack according to the standard rules:

  • Field offsets are aligned according to the field’s type.

  • A structure is aligned according to the maximum alignment of its members.

  • Tail padding is added to make the structure’s size a multiple of its alignment.

4.5. Bit fields

The following types may be specified in a bit-field’s declaration: char, short, int, long , long long and enum.

If an enum type has negative values, enum bit-fields are signed. Otherwise, if a signed integer type of the specified width is not able to represent all enum values then enum bit-fields are unsigned. Otherwise, enum bit-fields are signed. All other bit-field types are signed unless explicitly unsigned.

Bit-fields pack from the least significant end of the allocation unit. Each non-zero bit field is allocated at the first available bit offset that allows the bit field to be placed in a properly aligned container of the declared type. Non bit-field members are allocated at the first available offset satisfying their declared type’s size and alignment constraints.

A zero-width bit-field forces padding until the next bit-offset aligned with the bit field’s declared type.

Unnamed bit-fields are allocated space in the same manner as named bit-fields.

A structure is aligned according to each of the bit field’s declared types in addition to the types of any other members. Both zero-width and unnamed bit fields are taken into account when calculating a structure’s alignment.