PyTorch for the IPU: User Guide

• 1. Introduction
  • 1.1. Data batching
  • 1.2. Distributed execution
  • 1.3. Constraints
  • 1.4. Other resources
• 2. Installation
  • 2.1. Using a Python virtual environment
  • 2.2. Setting the environment variables
  • 2.3. Validating the setup
• 3. From PyTorch to PopTorch
  • 3.1. Preparing your data
  • 3.2. Creating your model
    • 3.2.1. Training
    • 3.2.2. Inference
  • 3.3. The training loop
  • 3.4. Multiple/custom losses
  • 3.5. Optimizers
  • 3.6. Going further
• 4. Features
  • 4.1. Options
  • 4.2. Model wrapping functions
    • 4.2.1. poptorch.trainingModel
    • 4.2.2. poptorch.inferenceModel
    • 4.2.3. poptorch.PoplarExecutor
    • 4.2.4. poptorch.isRunningOnIpu
  • 4.3. Parallel execution
    • 4.3.1. Annotation tools
      • poptorch.Block and poptorch.BeginBlock
      • poptorch.Stage and poptorch.AutoStage
      • poptorch.Phase
      • Advanced annotation with strings
    • 4.3.2. Parallel execution strategies
      • poptorch.ShardedExecution
      • poptorch.PipelinedExecution
      • Phased execution
  • 4.4. Optimizers
    • 4.4.1. Loss scaling
    • 4.4.2. Velocity scaling (SGD only)
    • 4.4.3. Accumulation types
    • 4.4.4. Constant attributes
  • 4.5. Custom ops
    • 4.5.1. poptorch.ipu_print_tensor
    • 4.5.2. poptorch.identity_loss
    • 4.5.3. poptorch.MultiConv
    • 4.5.4. poptorch.custom_op
    • 4.5.5. poptorch.nop
    • 4.5.6. poptorch.serializedMatMul
    • 4.5.7. poptorch.set_available_memory
  • 4.6. Miscellaneous functions
  • 4.7. Half / float 16 support
  • 4.8. Profiling
  • 4.9. Precompilation and caching
    • 4.9.1. Caching
    • 4.9.2. Precompilation
  • 4.10. Environment variables
    • 4.10.1. Logging level
    • 4.10.2. Profiling
    • 4.10.3. IPU Model
    • 4.10.4. Wait for an IPU to become available
    • 4.10.5. Enable executable caching
• 5. Efficient data batching
  • 5.1. poptorch.DataLoader
  • 5.2. poptorch.AsynchronousDataAccessor
  • 5.3. poptorch.Options.deviceIterations
  • 5.4. poptorch.Options.replicationFactor
  • 5.5. poptorch.Options.Training.gradientAccumulation
• 6. IPU supported operations
  • 6.1. Torch operations
    • 6.1.1. Tensor operations
      • Creation ops
      • Indexing, slicing, joining and mutating ops
      • Random samplers
    • 6.1.2. Math operations
      • Pointwise ops
      • Reduction ops
      • Comparison ops
      • Other ops
      • BLAS and LAPACK Operations
  • 6.2. Torch.nn operations
    • 6.2.1. Containers
    • 6.2.2. Convolution layers
    • 6.2.3. Pooling layers
    • 6.2.4. Padding layers
    • 6.2.5. Activations
    • 6.2.6. Normalization layers
    • 6.2.7. Recurrent layers
    • 6.2.8. Linear layers
    • 6.2.9. Dropout
    • 6.2.10. Sparse layers
    • 6.2.11. Loss functions
    • 6.2.12. Vision Layers
  • 6.3. Float 16 operations
    • 6.3.1. Casting
    • 6.3.2. Creation functions
    • 6.3.3. Normalization
• 7. Examples
  • 7.1. MNIST example
• 8. Experimental features
  • 8.1. Distributed execution
  • 8.2. torch.nn.CTCLoss
• 9. Reference
  • 9.1. Options
  • 9.2. Helpers
  • 9.3. PopTorch Ops
  • 9.4. Model wrapping functions
  • 9.5. Parallel execution
  • 9.6. Optimizers
  • 9.7. Data batching
• 10. Index
• 11. Trademarks & copyright
• 12. Changelog
  • 12.1. v2.0 (Poplar SDK 2.0)
    • 12.1.1. New features
  • 12.2. v1.0 (Poplar SDK 1.4)
    • 12.2.1. New features
    • 12.2.2. Known issues
  • 12.3. v0.1 (Poplar SDK 1.3)
    • 12.3.1. New features