5.10. Custom patterns

The concept of a “pattern” comes from the PopART framework. The role of a pattern is to match ops in the PopART IR and modify or replace the matched ops, as a means of graph optimisation.

A typical application scenario for the custom PopART pattern is when a user implements a high-performance op and replaces some performance bottleneck ops with this high-performance op through the custom PopART pattern, thereby improving the performance of the entire model.

This chapter helps users understand how to implement custom PopART patterns and use them in PopRT.

Before reading this chapter, it is necessary to be familiar with the PopART framework.

5.10.1. Implementing custom PopART patterns

To implement a custom PopART pattern in PopRT, at least one C++ file needs to be written.

This example shows how the Relu Op in the diagram is replaced with the Negative Op, which can be compiled into a standalone dynamic library and linked when using PopRT:

Listing 5.15 replace_relu_with_neg_pattern.cpp

// Copyright (c) 2022 Graphcore Ltd. All rights reserved.
#include <string>

#include <popart/graph.hpp>
#include <popart/op/negate.hpp>
#include <popart/op/relu.hpp>
#include <popart/op/sqrt.hpp>
#include <popart/operators.hpp>
#include <popart/patterns/pattern.hpp>
#include <popart/patterns/patterns.hpp>

namespace popart {
class IArray;
class Tensor;
} // namespace popart

using namespace popart;

class ReplaceReluWithNeg : public PreAliasPattern {
public:
  bool matches(Op *op) const override { return op->isConvertibleTo<ReluOp>(); }

  std::vector<const Tensor *> touches(Op *) const override { return {}; }

  bool apply(Op *op) const override {
    std::cout << "Custom pattern ReplaceReluWithNeg applied in "
              << op->debugName() << std::endl;

    auto negOp = makeReplacementOpInIr(Onnx::Operators::Neg_6, op);

    auto inputId  = op->inId(ReluOp::getInIndex());
    auto outputId = op->outId(ReluOp::getOutIndex());
    op->disconnectAllInputs();
    op->disconnectAllOutputs();
    op->getGraph().eraseOp(op->id);

    negOp->connectInTensor(NegateOp::getInIndex(), inputId);
    negOp->connectOutTensor(NegateOp::getOutIndex(), outputId);
    negOp->setup();

    return true;
  }
};

namespace {
static PatternCreator<ReplaceReluWithNeg>
    ReplaceReluWithNegPatternCreator("ReplaceReluWithNeg",
                                     /* default enabled = */ false,
                                     /* default mandatory = */ false);
} // namespace

To implement custom PopART patterns, two parts are typically required as follows:

Inherit PreAliasPattern and implement its apply method.
Register the pattern with PatternCreator.

5.10.2. Using custom PopART patterns in PopRT

First, the source code of the pattern needs to be compiled into a dynamic link library. The following is the command for compiling the example:

g++ \
    -std=c++14 \
    -fPIC \
    -O3 \
    -DONNX_NAMESPACE=onnx \
    replace_relu_with_neg_pattern.cpp \
    -shared \
    -lpopart \
    -o custom_pattern.so

Then, the dynamic library containing custom patterns can be linked with the --custom_library_so_paths parameter of the PopRT CLI:

poprt  --custom_library_so_paths path/to/shared/library

There are several ways to use custom PopART patterns in PopRT. When setting a pattern, you can use :value after the pattern name to specify whether to disable or enable the pattern.

If :value is not used after the pattern name, the pattern is enabled.
If :value is specified after the pattern name, and value is one of True, true or 1, the pattern will be enabled.
Otherwise the pattern is disabled.

Method 1: Use `PatternCreator` to enable the pattern by default

static PatternCreator<ReplaceReluWithNeg>
    ReplaceReluWithNegPatternCreator("ReplaceReluWithNeg",
                                    /* default enabled = */ true);

With this method, the use of pattern is determined during the compilation period. After linking with the dynamic library of this pattern, it will be executed by default and cannot be dynamically controlled and during run time.

Method 2: Configure a pattern using the Python API

You can use the CompilerOptions().custom_patterns API to configure patterns in Python. In the following case, the ReplaceReluWithNeg pattern is enabled, and the InPlace pattern is disabled.

from poprt.compiler import Compiler, CompilerOptions

opts = CompilerOptions()
opts.custom_patterns = ["ReplaceReluWithNeg", "InPlace:0"]
Compiler.compile_and_export(model, outputs, popef_path, opts)

Method 3: Config the specified pattern using CLI

You can also enable custom patterns using the PopRT CLI. Multiple patterns can be separated by commas. In the following case, the ReplaceReluWithNeg pattern is enabled and the InPlace pattern is disabled.

poprt \
    --input_model model.onnx \
    --output_model model_export.onnx \
    --export_popef \
    --output_dir model \
    --custom_library_so_paths build/custom_patterns.so \
    --compiler_options custom_patterns=ReplaceReluWithNeg,InPlace:0

Search help