5.10. Developing custom patterns

The concept of a “pattern” comes from the PopART framework. The role of a pattern is to match ops in the PopART IR and modify or replace the matched ops, as a means of graph optimisation.

A typical application scenario for the custom PopART pattern is when a user implements a high-performance op and replaces some performance bottleneck ops with this high-performance op through the custom PopART pattern, thereby improving the performance of the entire model.

This chapter helps users understand how to implement custom PopART patterns and use them in PopRT.

Before reading this chapter, it is necessary to be familiar with the PopART framework.

5.10.1. Implementing custom PopART patterns

To implement a custom PopART pattern in PopRT, at least one C++ file needs to be written.

This example shows how the Relu Op in the diagram is replaced with the Negative Op, which can be compiled into a standalone dynamic library and linked when using PopRT:

Listing 5.15 replace_relu_with_neg_pattern.cpp

// Copyright (c) 2022 Graphcore Ltd. All rights reserved.
#include <string>

#include <popart/graph.hpp>
#include <popart/op/negate.hpp>
#include <popart/op/relu.hpp>
#include <popart/op/sqrt.hpp>
#include <popart/operators.hpp>
#include <popart/patterns/pattern.hpp>
#include <popart/patterns/patterns.hpp>

namespace popart {
class IArray;
class Tensor;
} // namespace popart

using namespace popart;

class ReplaceReluWithNeg : public PreAliasPattern {
public:
  bool matches(Op *op) const override { return op->isConvertibleTo<ReluOp>(); }

  std::vector<const Tensor *> touches(Op *) const override { return {}; }

  bool apply(Op *op) const override {
    std::cout << "Custom pattern ReplaceReluWithNeg applied in "
              << op->debugName() << std::endl;

    auto negOp = makeReplacementOpInIr(Onnx::Operators::Neg_6, op);

    auto inputId  = op->inId(ReluOp::getInIndex());
    auto outputId = op->outId(ReluOp::getOutIndex());
    op->disconnectAllInputs();
    op->disconnectAllOutputs();
    op->getGraph().eraseOp(op->id);

    negOp->connectInTensor(NegateOp::getInIndex(), inputId);
    negOp->connectOutTensor(NegateOp::getOutIndex(), outputId);
    negOp->setup();

    return true;
  }
};

namespace {
static PatternCreator<ReplaceReluWithNeg>
    ReplaceReluWithNegPatternCreator("ReplaceReluWithNeg",
                                     /* default enabled = */ false,
                                     /* default mandatory = */ false);
} // namespace

To implement custom PopART patterns, two parts are typically required as follows:

• Inherit PreAliasPattern and implement its apply method.

• Register the pattern with PatternCreator.

5.10.2. Using custom PopART patterns in PopRT

First, the source code of the pattern needs to be compiled into a dynamic link library. The following is the command for compiling the example:

g++ \
    -std=c++14 \
    -fPIC \
    -O3 \
    -DONNX_NAMESPACE=onnx \
    replace_relu_with_neg_pattern.cpp \
    -shared \
    -lpopart \
    -o custom_pattern.so

Then, the dynamic library containing custom patterns can be linked with the --custom_library_so_paths parameter of the PopRT CLI:

poprt  --custom_library_so_paths path/to/shared/library

There are currently two methods for using custom PopART patterns in PopRT.

Method 1: Use PatternCreator to enable the pattern by default

static PatternCreator<ReplaceReluWithNeg>
    ReplaceReluWithNegPatternCreator("ReplaceReluWithNeg",
                                    /* default enabled = */ true);

With this method, the use of pattern is determined during the compilation period. After linking with the dynamic library of this pattern, it will be executed by default and cannot be dynamically controlled and during run time.

Method 2: Enable the specified pattern using CLI parameters

You can also enable custom patterns using the PopRT CLI. Multiple patterns can be separated by commas.

For example:

poprt \
    --input_model model.onnx \
    --output_model model_export.onnx \
    --export_popef \
    --output_dir model \
    --custom_library_so_paths build/custom_patterns.so \
    --compiler_options custom_patterns=ReplaceReluWithNeg