Logo
PopDist and PopRun: User Guide
Version: latest
  • 1. Introduction
    • 1.1. Installation
    • 1.2. Validating the installation
    • 1.3. Replicas and instances
  • 2. Launching applications with PopRun
    • 2.1. Launch modes
    • 2.2. Multi-host setup
      • 2.2.1. SSH setup
        • Install and configure OpenSSH on all hosts
        • Create SSH key pair
        • Authorise key pair on all hosts
        • Verify SSH setup
      • 2.2.2. PopRun SSH distribution tool
      • 2.2.3. Network interfaces
      • 2.2.4. File system setup
    • 2.3. Caching compiled executables
    • 2.4. Application launches
      • 2.4.1. Single instance
      • 2.4.2. Multi instance / single host
      • 2.4.3. Multi instance / multi host
    • 2.5. Process placement and non-uniform memory access (NUMA)
    • 2.6. Offline compilation
    • 2.7. Using with PopVision Graph Analyser
      • 2.7.1. POPLAR_ENGINE_OPTIONS and PopVision Graph Analyser
      • 2.7.2. Launching without POPLAR_ENGINE_OPTIONS
      • 2.7.3. Launching with POPLAR_ENGINE_OPTIONS
    • 2.8. V-IPU settings
      • 2.8.1. V-IPU server address
      • 2.8.2. PopRun and IPU over fabric settings
      • 2.8.3. V-IPU partition, cluster, and allocation
    • 2.9. Storing and loading command line arguments
    • 2.10. Troubleshooting and solutions to common problems
      • 2.10.1. PopRun cannot find or create partition
      • 2.10.2. Program cannot acquire devices
  • 3. PopRun features
    • 3.1. Usage
    • 3.2. Tips and tricks
      • 3.2.1. Passing environment variables
      • 3.2.2. Character escaping
      • 3.2.3. File output forwarding
      • 3.2.4. Topology table
      • 3.2.5. Generating auto-completion file
  • 4. Poplar distributed configuration library (PopDist)
    • 4.1. Horovod installation
      • 4.1.1. PyTorch
      • 4.1.2. TensorFlow 2
      • 4.1.3. TensorFlow 1
      • 4.1.4. PopART
    • 4.2. PopDist collectives
      • 4.2.1. Running code on a subset of instances
    • 4.3. PopDist API
    • 4.4. PopDist examples
      • 4.4.1. PyTorch
      • 4.4.2. TensorFlow 1
      • 4.4.3. TensorFlow 2
    • 4.5. Conclusion
  • 5. PopDist C++ API reference
  • 6. PopDist Python API reference
    • 6.1. PopART
    • 6.2. PopTorch
    • 6.3. TensorFlow 1 and 2
  • 7. PopRun changelog
    • 7.1. v2.3 (Poplar SDK 2.3)
      • 7.1.1. New features
    • 7.2. v2.2 (Poplar SDK 2.2)
      • 7.2.1. New features
    • 7.3. v2.1 (Poplar SDK 2.1)
      • 7.3.1. New features
    • 7.4. v2.0 (Poplar SDK 2.0)
      • 7.4.1. New features
    • 7.5. v1.0 (Poplar SDK 1.4)
      • 7.5.1. New features
  • 8. PopDist changelog
    • 8.1. v2.6 (Poplar SDK 2.6)
    • 8.2. v2.3 (Poplar SDK 2.3)
    • 8.3. v2.2 (Poplar SDK 2.2)
      • 8.3.1. New features
    • 8.4. v2.1 (Poplar SDK 2.1)
      • 8.4.1. New features
    • 8.5. v2.0 (Poplar SDK 2.0)
      • 8.5.1. New features
    • 8.6. v1.0 (Poplar SDK 1.4)
      • 8.6.1. New features
  • 9. Known issues
    • 9.1. Race condition with multiple users of the same partition
  • 10. Index
  • 11. Trademarks & copyright
PopDist and PopRun: User Guide

Search help

Note: Searching from the top-level index page will search all documents. Searching from a specific document will search only that document.

  • Find an exact phrase: Wrap your search phrase in "" (double quotes) to only get results where the phrase is exactly matched. For example "PyTorch for the IPU" or "replicated tensor sharding"
  • Prefix query: Add an * (asterisk) at the end of any word to indicate a prefix query. This will return results containing all words with the specific prefix. For example tensor*
  • Fuzzy search: Use ~N (tilde followed by a number) at the end of any word for a fuzzy search. This will return results that are similar to the search word. N specifies the “edit distance” (fuzziness) of the match. For example Polibs~1
  • Words close to each other: ~N (tilde followed by a number) after a phrase (in quotes) returns results where the words are close to each other. N is the maximum number of positions allowed between matching words. For example "ipu version"~2
  • Logical operators. You can use the following logical operators in a search:
    • + signifies AND operation
    • | signifies OR operation
    • - negates a single word or phrase (returns results without that word or phrase)
    • () controls operator precedence


Revision 70615515.