Recipes
=======
Recipes are relatively generic, built-in functionnalities for manipulating a compute graph in Aidge. Some are built with Aidge's graph matching engine, do not hesitate to have a look at their source code to understand how they work and build similar functionnalities!
.. contents::
:depth: 2
:local:
🚧 The list of recipes is still growing!
Adapt to backend
----------------
Adapt a graph to the available kernels of a backend. The following transformations
can be performed at the inputs and/or the outputs of operators:
- ``Cast``: change of data type;
- ``Transpose``: change of data format.
.. tab-set::
.. tab-item:: Python
.. autofunction:: aidge_core.adapt_to_backend
.. tab-item:: C++
.. doxygenfunction:: Aidge::adaptToBackend
Constant folding
----------------
Fold constant operators (like ONNX Simplifier).
.. tab-set::
.. tab-item:: C++
.. doxygenfunction:: Aidge::constantFolding
Convert Conv to MatMul
----------------------
Convert ``Conv`` operators to ``Unfold`` (im2col operation) + ``MatMul`` + ``Reshape``.
.. tab-set::
.. tab-item:: C++
.. doxygenfunction:: Aidge::constantFolding
Input graph:
```mermaid
%%{init: {'flowchart': { 'curve': 'monotoneY'}, 'fontFamily': 'Verdana' } }%%
flowchart TB
Producer_3("conv2_w\n(Producer#3)"):::producerCls
Conv_1("conv2\n(Conv#1)")
Conv_0("conv1\n(Conv#0)")
Producer_2("conv1_b\n(Producer#2)"):::producerCls
Producer_1("conv1_w\n(Producer#1)"):::producerCls
Producer_4("conv3_w\n(Producer#4)"):::producerCls
Conv_2("conv3\n(Conv#2)")
Producer_5("conv3_b\n(Producer#5)"):::producerCls
Producer_0("dataProvider\n(Producer#0)"):::producerCls_rootCls
Producer_3-->|"0 [7, 4, 3, 3]→1"|Conv_1
Conv_1-->|"0 [2, 7, 9, 20]→0"|Conv_2
Conv_0-->|"0 [2, 4, 11, 22]→0"|Conv_1
Producer_2-->|"0 [4]→2"|Conv_0
Producer_1-->|"0 [4, 3, 3, 3]→1"|Conv_0
Producer_4-->|"0 [10, 7, 1, 1]→1"|Conv_2
Producer_5-->|"0 [10]→2"|Conv_2
Producer_0-->|"0 [2, 3, 13, 24]→0"|Conv_0
input0((in#0)):::inputCls--->|"→2"|Conv_1
Conv_2--->|"0 [2, 10, 5, 10]→"|output0((out#0)):::outputCls
classDef inputCls fill:#afa
classDef outputCls fill:#ffa
classDef externalCls fill:#ccc
classDef producerCls fill:#ccf
classDef genericCls fill:#f9f9ff,stroke-width:1px,stroke-dasharray: 5 5
classDef metaCls stroke-width:5px
classDef rootCls stroke:#f00
classDef producerCls_rootCls stroke:#f00,fill:#ccf
classDef genericCls_rootCls stroke:#f00,fill:#f9f9ff,stroke-width:1px,stroke-dasharray: 5 5
classDef metaCls_rootCls stroke:#f00,stroke-width:5px
```
Output graph:
```mermaid
%%{init: {'flowchart': { 'curve': 'monotoneY'}, 'fontFamily': 'Verdana' } }%%
flowchart TB
Producer_0("dataProvider\n(Producer#0)"):::producerCls_rootCls
MatMul_2("conv3_matmul\n(MatMul#2)")
Producer_7("conv3_reshape_shape_prod\n(Producer#7)"):::producerCls
Reshape_2("conv3_reshape\n(Reshape#2)")
Add_1("conv3_add\n(Add#1)")
Producer_8("conv3_b_reshape_0\n(Producer#8)"):::producerCls
Producer_1("conv1_w_reshape_0\n(Producer#1)"):::producerCls
Unfold_2("conv3_unfold\n(Unfold#2)")
Producer_3("conv1_b_reshape_0\n(Producer#3)"):::producerCls
Unfold_0("conv1_unfold\n(Unfold#0)")
MatMul_0("conv1_matmul\n(MatMul#0)")
Producer_2("conv1_reshape_shape_prod\n(Producer#2)"):::producerCls
Reshape_0("conv1_reshape\n(Reshape#0)")
Add_0("conv1_add\n(Add#0)")
Unfold_1("conv2_unfold\n(Unfold#1)")
MatMul_1("conv2_matmul\n(MatMul#1)")
Producer_5("conv2_reshape_shape_prod\n(Producer#5)"):::producerCls
Reshape_1("conv2_reshape\n(Reshape#1)")
Producer_4("conv2_w_reshape_0\n(Producer#4)"):::producerCls
Producer_6("conv3_w_reshape_0\n(Producer#6)"):::producerCls
Producer_0-->|"0 [2, 3, 13, 24]→0"|Unfold_0
MatMul_2-->|"0 [2, 10, 50]→0"|Reshape_2
Producer_7-->|"0 [4]→1"|Reshape_2
Reshape_2-->|"0 [2, 10, 5, 10]→0"|Add_1
Producer_8-->|"0 [1, 10, 1, 1]→1"|Add_1
Producer_1-->|"0 [4, 27]→0"|MatMul_0
Unfold_2-->|"0 [2, 7, 50]→1"|MatMul_2
Producer_3-->|"0 [1, 4, 1, 1]→1"|Add_0
Unfold_0-->|"0 [2, 27, 242]→1"|MatMul_0
MatMul_0-->|"0 [2, 4, 242]→0"|Reshape_0
Producer_2-->|"0 [4]→1"|Reshape_0
Reshape_0-->|"0 [2, 4, 11, 22]→0"|Add_0
Add_0-->|"0 [2, 4, 11, 22]→0"|Unfold_1
Unfold_1-->|"0 [2, 36, 180]→1"|MatMul_1
MatMul_1-->|"0 [2, 7, 180]→0"|Reshape_1
Producer_5-->|"0 [4]→1"|Reshape_1
Reshape_1-->|"0 [2, 7, 9, 20]→0"|Unfold_2
Producer_4-->|"0 [7, 36]→0"|MatMul_1
Producer_6-->|"0 [10, 7]→0"|MatMul_2
Add_1--->|"0 [2, 10, 5, 10]→"|output0((out#0)):::outputCls
classDef inputCls fill:#afa
classDef outputCls fill:#ffa
classDef externalCls fill:#ccc
classDef producerCls fill:#ccf
classDef genericCls fill:#f9f9ff,stroke-width:1px,stroke-dasharray: 5 5
classDef metaCls stroke-width:5px
classDef rootCls stroke:#f00
classDef producerCls_rootCls stroke:#f00,fill:#ccf
classDef genericCls_rootCls stroke:#f00,fill:#f9f9ff,stroke-width:1px,stroke-dasharray: 5 5
classDef metaCls_rootCls stroke:#f00,stroke-width:5px
```
Expand meta operators
---------------------
Expand meta operators, replacing them with their inner graph (flatten the graph).
.. tab-set::
.. tab-item:: Python
.. autofunction:: aidge_core.expand_metaops
.. tab-item:: C++
.. doxygenfunction:: Aidge::expandMetaOps
Explicit Cast Move
------------------
Insert Cast and Move operators where needed (thus removing all implicit data type conversion and backend change data movement).
.. tab-set::
.. tab-item:: C++
.. doxygenfunction:: Aidge::explicitCastMove
Explicit Transpose
------------------
Insert Transpose operators where needed to ensure no transposition needs to be done at the Operator level (thus removing all implicit data format conversion).
.. tab-set::
.. tab-item:: C++
.. doxygenfunction:: Aidge::explicitTranspose
Fuse BatchNorm
--------------
Fuse batch normalization with the preceding Conv or FC operator, if possible.
.. tab-set::
.. tab-item:: Python
.. autofunction:: aidge_core.fuse_batchnorm
.. tab-item:: C++
.. doxygenfunction:: Aidge::fuseBatchNorm(std::shared_ptr graphView)
Fuse MatMul and Add to FC
-------------------------
Fuse MatMul optionnally followed by Add operator into a FC operator.
.. tab-set::
.. tab-item:: Python
.. autofunction:: aidge_core.matmul_to_fc
.. tab-item:: C++
.. doxygenfunction:: Aidge::fuseMulAdd(std::shared_ptr graphView)
Fuse to meta operator
---------------------
Fuse each sub-graph matching a query in a Meta Operator.
.. tab-set::
.. tab-item:: Python
.. autofunction:: aidge_core.fuse_to_metaops
.. tab-item:: C++
.. doxygenfunction:: Aidge::fuseToMetaOps
MatMul tiling
-------------
Tile any ``MatMul`` operator to several fixed size matrix multiplications.
For instance, for a MatMul of size 80x80 and a tiling of 16x16, this will tile
the MatMul operator to 25 (5 by 5) MatMul operators of size 16x16, with Slice
operators inserted at the inputs and Concat operators inserted at the outputs.
This is especially useful when matrix multiplication must be mapped to fixed
maximum size hardware TPU (Tensor Processing Unit) or MMA (Matrix Multiplication
Accelerator). This recipe can be combined with the ``convToMatMul`` recipe in
order to convert convolutions to matrix multiplication beforehand, and
``constantFolding`` recipe to fold sliced constant tensors.
.. tab-set::
.. tab-item:: C++
.. doxygenfunction:: Aidge::matMulTiling
Initial graph:
```mermaid
%%{init: {'flowchart': { 'curve': 'monotoneY'}, 'fontFamily': 'Verdana' } }%%
flowchart TB
MatMul_0("matmul1
(MatMul#0)"):::rootCls
Producer_1("w1
(Producer#1)"):::producerCls
Producer_0("dataProvider
(Producer#0)"):::producerCls
MatMul_0--->|"0 [2, 3, 80, 80]→"|output0((out#0)):::outputCls
Producer_1-->|"0 [2, 3, 80, 80]→1"|MatMul_0
Producer_0-->|"0 [2, 3, 80, 80]→0"|MatMul_0
classDef inputCls fill:#afa
classDef outputCls fill:#ffa
classDef externalCls fill:#ccc
classDef producerCls fill:#ccf
classDef genericCls fill:#f9f9ff,stroke-width:1px,stroke-dasharray: 5 5
classDef metaCls stroke-width:5px
classDef rootCls stroke:#f00
classDef producerCls_rootCls stroke:#f00,fill:#ccf
classDef genericCls_rootCls stroke:#f00,fill:#f9f9ff,stroke-width:1px,stroke-dasharray: 5 5
classDef metaCls_rootCls stroke:#f00,stroke-width:5px
```
Graph generated by a single step of the ``matMulTiling`` recipe (after the very first matrix multiplication split):
```mermaid
%%{init: {'flowchart': { 'curve': 'monotoneY'}, 'fontFamily': 'Verdana' } }%%
flowchart TB
Producer_7(Producer#7):::producerCls
MatMul_1(MatMul#1)
Concat_0(Concat#0)
Producer_1(Producer#1):::producerCls
Producer_2(Producer#2):::producerCls
Producer_3(Producer#3):::producerCls
Producer_4(Producer#4):::producerCls
Producer_5(Producer#5):::producerCls
Producer_6(Producer#6):::producerCls
Identity_0(Identity#0):::rootCls
Slice_0(Slice#0)
Producer_0(Producer#0):::producerCls
MatMul_0(MatMul#0)
Identity_1(Identity#1)
Slice_1(Slice#1)
Producer_7-->|"0 [2]→4"|Slice_1
MatMul_1-->|"0 [2, 3, 64, 80]→1"|Concat_0
Producer_1-->|"0 [2]→2"|Slice_0
Producer_2-->|"0 [2]→3"|Slice_0
Producer_3-->|"0 [2]→4"|Slice_0
Producer_4-->|"0 [2]→1"|Slice_1
Producer_5-->|"0 [2]→2"|Slice_1
Producer_6-->|"0 [2]→3"|Slice_1
Identity_0-->|"0 [2, 3, 80, 80]→0"|Slice_0
Identity_0-->|"0 [2, 3, 80, 80]→0"|Slice_1
Slice_0-->|"0 [2, 3, 16, 80]→0"|MatMul_0
Producer_0-->|"0 [2]→1"|Slice_0
MatMul_0-->|"0 [2, 3, 16, 80]→0"|Concat_0
Identity_1-->|"0 [2, 3, 80, 80]→1"|MatMul_1
Identity_1-->|"0 [2, 3, 80, 80]→1"|MatMul_0
Slice_1-->|"0 [2, 3, 64, 80]→0"|MatMul_1
input0((in#0)):::inputCls--->|"→0[2, 3, 80, 80]"|Identity_0
input1((in#1)):::inputCls--->|"→0[2, 3, 80, 80]"|Identity_1
Concat_0--->|"0 [2, 3, 80, 80]→"|output0((out#0)):::outputCls
classDef inputCls fill:#afa
classDef outputCls fill:#ffa
classDef externalCls fill:#ccc
classDef producerCls fill:#ccf
classDef genericCls fill:#f9f9ff,stroke-width:1px,stroke-dasharray: 5 5
classDef metaCls stroke-width:5px
classDef rootCls stroke:#f00
classDef producerCls_rootCls stroke:#f00,fill:#ccf
classDef genericCls_rootCls stroke:#f00,fill:#f9f9ff,stroke-width:1px,stroke-dasharray: 5 5
classDef metaCls_rootCls stroke:#f00,stroke-width:5px
```
Remove Dropout
--------------
Remove Dropout operators.
.. tab-set::
.. tab-item:: C++
.. doxygenfunction:: Aidge::removeDropout(std::shared_ptr graphView)
Remove Flatten
--------------
Remove Flatten operators.
.. tab-set::
.. tab-item:: Python
.. autofunction:: aidge_core.remove_flatten
.. tab-item:: C++
.. doxygenfunction:: Aidge::removeFlatten(std::shared_ptr graphView)