Understanding Aidge’s scheduling#
Aidge introduces a well-defined consumer-producer (C-P) model for operator implementations, similar to transaction-level modeling (TLM) for electronic design. The C-P model of an operator implementation specifies how much data is consumed and produced by the operator implementation at each execution step (i.e. at each forward pass). C-P model can be specified as precise amounts of data (number of elements) or arbitrary data quantity (token). The C-P model execution path is decoupled from the data execution path, thus allowing to statically schedule the graph execution without providing the actual operator’s implementation.
Aidge’s base scheduler use this C-P model to statically schedule a graph before execution. Scheduling is always static in Aidge.
[1]:
# First import some utility methods used in the tutorial:
import sys, os
sys.path.append(os.path.abspath(os.path.join('..')))
import tuto_utils
/home/ob222806/.local/lib/python3.10/site-packages/matplotlib/projections/__init__.py:63: UserWarning: Unable to import Axes3D. This may be due to multiple versions of Matplotlib being installed (e.g. as a system package and as a pip package). As a result, the 3D projection is not available.
warnings.warn("Unable to import Axes3D. This may be due to multiple versions of "
To generate the static scheduling of a graph, here for example MobileNetv2, just do:
[2]:
import aidge_core
import aidge_onnx
import aidge_backend_cpu
aidge_model = aidge_onnx.load_onnx("../Learning/mobilenetv2-7.onnx", verbose=False)
aidge_model.set_backend("cpu")
# Create the Scheduler
scheduler = aidge_core.SequentialScheduler(aidge_model)
scheduler.generate_scheduling()
# Display static scheduling
scheduler.save_static_scheduling_diagram("scheduling")
tuto_utils.visualize_mmd("scheduling.mmd")
[NOTICE] - - mobilenetv20_features_conv0_fwd (Conv)
[NOTICE] - - mobilenetv20_features_batchnorm0_fwd (BatchNormalization)
[NOTICE] - - mobilenetv20_features_relu0_fwd (Relu)
[NOTICE] - - mobilenetv20_features_linearbottleneck0_conv0_fwd (Conv)
[NOTICE] - - mobilenetv20_features_linearbottleneck0_batchnorm0_fwd
[NOTICE] (BatchNormalization)
[NOTICE] - - mobilenetv20_features_linearbottleneck0_relu0_fwd (Relu)
[NOTICE] - - mobilenetv20_features_linearbottleneck0_conv1_fwd (Conv)
[NOTICE] - - mobilenetv20_features_linearbottleneck0_batchnorm1_fwd
[NOTICE] (BatchNormalization)
[NOTICE] - - mobilenetv20_features_linearbottleneck0_relu1_fwd (Relu)
[NOTICE] - - mobilenetv20_features_linearbottleneck0_conv2_fwd (Conv)
[NOTICE] - - mobilenetv20_features_linearbottleneck0_batchnorm2_fwd
[NOTICE] (BatchNormalization)
[NOTICE] - - mobilenetv20_features_linearbottleneck1_conv0_fwd (Conv)
[NOTICE] - - mobilenetv20_features_linearbottleneck1_batchnorm0_fwd
[NOTICE] (BatchNormalization)
[NOTICE] - - mobilenetv20_features_linearbottleneck1_relu0_fwd (Relu)
[NOTICE] - - mobilenetv20_features_linearbottleneck1_conv1_fwd (Conv)
[NOTICE] - - mobilenetv20_features_linearbottleneck1_batchnorm1_fwd
[NOTICE] (BatchNormalization)
[NOTICE] - - mobilenetv20_features_linearbottleneck1_relu1_fwd (Relu)
[NOTICE] - - mobilenetv20_features_linearbottleneck1_conv2_fwd (Conv)
[NOTICE] - - mobilenetv20_features_linearbottleneck1_batchnorm2_fwd
[NOTICE] (BatchNormalization)
[NOTICE] - - mobilenetv20_features_linearbottleneck2_conv0_fwd (Conv)
[NOTICE] - - mobilenetv20_features_linearbottleneck2_batchnorm0_fwd
[NOTICE] (BatchNormalization)
[NOTICE] - - mobilenetv20_features_linearbottleneck2_relu0_fwd (Relu)
[NOTICE] - - mobilenetv20_features_linearbottleneck2_conv1_fwd (Conv)
[NOTICE] - - mobilenetv20_features_linearbottleneck2_batchnorm1_fwd
[NOTICE] (BatchNormalization)
[NOTICE] - - mobilenetv20_features_linearbottleneck2_relu1_fwd (Relu)
[NOTICE] - - mobilenetv20_features_linearbottleneck2_conv2_fwd (Conv)
[NOTICE] - - mobilenetv20_features_linearbottleneck2_batchnorm2_fwd
[NOTICE] (BatchNormalization)
[NOTICE] - - mobilenetv20_features_linearbottleneck2_elemwise_add0 (Add)
[NOTICE] - - mobilenetv20_features_linearbottleneck3_conv0_fwd (Conv)
[NOTICE] - - mobilenetv20_features_linearbottleneck3_batchnorm0_fwd
[NOTICE] (BatchNormalization)
[NOTICE] - - mobilenetv20_features_linearbottleneck3_relu0_fwd (Relu)
[NOTICE] - - mobilenetv20_features_linearbottleneck3_conv1_fwd (Conv)
[NOTICE] - - mobilenetv20_features_linearbottleneck3_batchnorm1_fwd
[NOTICE] (BatchNormalization)
[NOTICE] - - mobilenetv20_features_linearbottleneck3_relu1_fwd (Relu)
[NOTICE] - - mobilenetv20_features_linearbottleneck3_conv2_fwd (Conv)
[NOTICE] - - mobilenetv20_features_linearbottleneck3_batchnorm2_fwd
[NOTICE] (BatchNormalization)
[NOTICE] - - mobilenetv20_features_linearbottleneck4_conv0_fwd (Conv)
[NOTICE] - - mobilenetv20_features_linearbottleneck4_batchnorm0_fwd
[NOTICE] (BatchNormalization)
[NOTICE] - - mobilenetv20_features_linearbottleneck4_relu0_fwd (Relu)
[NOTICE] - - mobilenetv20_features_linearbottleneck4_conv1_fwd (Conv)
[NOTICE] - - mobilenetv20_features_linearbottleneck4_batchnorm1_fwd
[NOTICE] (BatchNormalization)
[NOTICE] - - mobilenetv20_features_linearbottleneck4_relu1_fwd (Relu)
[NOTICE] - - mobilenetv20_features_linearbottleneck4_conv2_fwd (Conv)
[NOTICE] - - mobilenetv20_features_linearbottleneck4_batchnorm2_fwd
[NOTICE] (BatchNormalization)
[NOTICE] - - mobilenetv20_features_linearbottleneck4_elemwise_add0 (Add)
[NOTICE] - - mobilenetv20_features_linearbottleneck5_conv0_fwd (Conv)
[NOTICE] - - mobilenetv20_features_linearbottleneck5_batchnorm0_fwd
[NOTICE] (BatchNormalization)
[NOTICE] - - mobilenetv20_features_linearbottleneck5_relu0_fwd (Relu)
[NOTICE] - - mobilenetv20_features_linearbottleneck5_conv1_fwd (Conv)
[NOTICE] - - mobilenetv20_features_linearbottleneck5_batchnorm1_fwd
[NOTICE] (BatchNormalization)
[NOTICE] - - mobilenetv20_features_linearbottleneck5_relu1_fwd (Relu)
[NOTICE] - - mobilenetv20_features_linearbottleneck5_conv2_fwd (Conv)
[NOTICE] - - mobilenetv20_features_linearbottleneck5_batchnorm2_fwd
[NOTICE] (BatchNormalization)
[NOTICE] - - mobilenetv20_features_linearbottleneck5_elemwise_add0 (Add)
[NOTICE] - - mobilenetv20_features_linearbottleneck6_conv0_fwd (Conv)
[NOTICE] - - mobilenetv20_features_linearbottleneck6_batchnorm0_fwd
[NOTICE] (BatchNormalization)
[NOTICE] - - mobilenetv20_features_linearbottleneck6_relu0_fwd (Relu)
[NOTICE] - - mobilenetv20_features_linearbottleneck6_conv1_fwd (Conv)
[NOTICE] - - mobilenetv20_features_linearbottleneck6_batchnorm1_fwd
[NOTICE] (BatchNormalization)
[NOTICE] - - mobilenetv20_features_linearbottleneck6_relu1_fwd (Relu)
[NOTICE] - - mobilenetv20_features_linearbottleneck6_conv2_fwd (Conv)
[NOTICE] - - mobilenetv20_features_linearbottleneck6_batchnorm2_fwd
[NOTICE] (BatchNormalization)
[NOTICE] - - mobilenetv20_features_linearbottleneck7_conv0_fwd (Conv)
[NOTICE] - - mobilenetv20_features_linearbottleneck7_batchnorm0_fwd
[NOTICE] (BatchNormalization)
[NOTICE] - - mobilenetv20_features_linearbottleneck7_relu0_fwd (Relu)
[NOTICE] - - mobilenetv20_features_linearbottleneck7_conv1_fwd (Conv)
[NOTICE] - - mobilenetv20_features_linearbottleneck7_batchnorm1_fwd
[NOTICE] (BatchNormalization)
[NOTICE] - - mobilenetv20_features_linearbottleneck7_relu1_fwd (Relu)
[NOTICE] - - mobilenetv20_features_linearbottleneck7_conv2_fwd (Conv)
[NOTICE] - - mobilenetv20_features_linearbottleneck7_batchnorm2_fwd
[NOTICE] (BatchNormalization)
[NOTICE] - - mobilenetv20_features_linearbottleneck7_elemwise_add0 (Add)
[NOTICE] - - mobilenetv20_features_linearbottleneck8_conv0_fwd (Conv)
[NOTICE] - - mobilenetv20_features_linearbottleneck8_batchnorm0_fwd
[NOTICE] (BatchNormalization)
[NOTICE] - - mobilenetv20_features_linearbottleneck8_relu0_fwd (Relu)
[NOTICE] - - mobilenetv20_features_linearbottleneck8_conv1_fwd (Conv)
[NOTICE] - - mobilenetv20_features_linearbottleneck8_batchnorm1_fwd
[NOTICE] (BatchNormalization)
[NOTICE] - - mobilenetv20_features_linearbottleneck8_relu1_fwd (Relu)
[NOTICE] - - mobilenetv20_features_linearbottleneck8_conv2_fwd (Conv)
[NOTICE] - - mobilenetv20_features_linearbottleneck8_batchnorm2_fwd
[NOTICE] (BatchNormalization)
[NOTICE] - - mobilenetv20_features_linearbottleneck8_elemwise_add0 (Add)
[NOTICE] - - mobilenetv20_features_linearbottleneck9_conv0_fwd (Conv)
[NOTICE] - - mobilenetv20_features_linearbottleneck9_batchnorm0_fwd
[NOTICE] (BatchNormalization)
[NOTICE] - - mobilenetv20_features_linearbottleneck9_relu0_fwd (Relu)
[NOTICE] - - mobilenetv20_features_linearbottleneck9_conv1_fwd (Conv)
[NOTICE] - - mobilenetv20_features_linearbottleneck9_batchnorm1_fwd
[NOTICE] (BatchNormalization)
[NOTICE] - - mobilenetv20_features_linearbottleneck9_relu1_fwd (Relu)
[NOTICE] - - mobilenetv20_features_linearbottleneck9_conv2_fwd (Conv)
[NOTICE] - - mobilenetv20_features_linearbottleneck9_batchnorm2_fwd
[NOTICE] (BatchNormalization)
[NOTICE] - - mobilenetv20_features_linearbottleneck9_elemwise_add0 (Add)
[NOTICE] - - mobilenetv20_features_linearbottleneck10_conv0_fwd (Conv)
[NOTICE] - - mobilenetv20_features_linearbottleneck10_batchnorm0_fwd
[NOTICE] (BatchNormalization)
[NOTICE] - - mobilenetv20_features_linearbottleneck10_relu0_fwd (Relu)
[NOTICE] - - mobilenetv20_features_linearbottleneck10_conv1_fwd (Conv)
[NOTICE] - - mobilenetv20_features_linearbottleneck10_batchnorm1_fwd
[NOTICE] (BatchNormalization)
[NOTICE] - - mobilenetv20_features_linearbottleneck10_relu1_fwd (Relu)
[NOTICE] - - mobilenetv20_features_linearbottleneck10_conv2_fwd (Conv)
[NOTICE] - - mobilenetv20_features_linearbottleneck10_batchnorm2_fwd
[NOTICE] (BatchNormalization)
[NOTICE] - - mobilenetv20_features_linearbottleneck11_conv0_fwd (Conv)
[NOTICE] - - mobilenetv20_features_linearbottleneck11_batchnorm0_fwd
[NOTICE] (BatchNormalization)
[NOTICE] - - mobilenetv20_features_linearbottleneck11_relu0_fwd (Relu)
[NOTICE] - - mobilenetv20_features_linearbottleneck11_conv1_fwd (Conv)
[NOTICE] - - mobilenetv20_features_linearbottleneck11_batchnorm1_fwd
[NOTICE] (BatchNormalization)
[NOTICE] - - mobilenetv20_features_linearbottleneck11_relu1_fwd (Relu)
[NOTICE] - - mobilenetv20_features_linearbottleneck11_conv2_fwd (Conv)
[NOTICE] - - mobilenetv20_features_linearbottleneck11_batchnorm2_fwd
[NOTICE] (BatchNormalization)
[NOTICE] - - mobilenetv20_features_linearbottleneck11_elemwise_add0 (Add)
[NOTICE] - - mobilenetv20_features_linearbottleneck12_conv0_fwd (Conv)
[NOTICE] - - mobilenetv20_features_linearbottleneck12_batchnorm0_fwd
[NOTICE] (BatchNormalization)
[NOTICE] - - mobilenetv20_features_linearbottleneck12_relu0_fwd (Relu)
[NOTICE] - - mobilenetv20_features_linearbottleneck12_conv1_fwd (Conv)
[NOTICE] - - mobilenetv20_features_linearbottleneck12_batchnorm1_fwd
[NOTICE] (BatchNormalization)
[NOTICE] - - mobilenetv20_features_linearbottleneck12_relu1_fwd (Relu)
[NOTICE] - - mobilenetv20_features_linearbottleneck12_conv2_fwd (Conv)
[NOTICE] - - mobilenetv20_features_linearbottleneck12_batchnorm2_fwd
[NOTICE] (BatchNormalization)
[NOTICE] - - mobilenetv20_features_linearbottleneck12_elemwise_add0 (Add)
[NOTICE] - - mobilenetv20_features_linearbottleneck13_conv0_fwd (Conv)
[NOTICE] - - mobilenetv20_features_linearbottleneck13_batchnorm0_fwd
[NOTICE] (BatchNormalization)
[NOTICE] - - mobilenetv20_features_linearbottleneck13_relu0_fwd (Relu)
[NOTICE] - - mobilenetv20_features_linearbottleneck13_conv1_fwd (Conv)
[NOTICE] - - mobilenetv20_features_linearbottleneck13_batchnorm1_fwd
[NOTICE] (BatchNormalization)
[NOTICE] - - mobilenetv20_features_linearbottleneck13_relu1_fwd (Relu)
[NOTICE] - - mobilenetv20_features_linearbottleneck13_conv2_fwd (Conv)
[NOTICE] - - mobilenetv20_features_linearbottleneck13_batchnorm2_fwd
[NOTICE] (BatchNormalization)
[NOTICE] - - mobilenetv20_features_linearbottleneck14_conv0_fwd (Conv)
[NOTICE] - - mobilenetv20_features_linearbottleneck14_batchnorm0_fwd
[NOTICE] (BatchNormalization)
[NOTICE] - - mobilenetv20_features_linearbottleneck14_relu0_fwd (Relu)
[NOTICE] - - mobilenetv20_features_linearbottleneck14_conv1_fwd (Conv)
[NOTICE] - - mobilenetv20_features_linearbottleneck14_batchnorm1_fwd
[NOTICE] (BatchNormalization)
[NOTICE] - - mobilenetv20_features_linearbottleneck14_relu1_fwd (Relu)
[NOTICE] - - mobilenetv20_features_linearbottleneck14_conv2_fwd (Conv)
[NOTICE] - - mobilenetv20_features_linearbottleneck14_batchnorm2_fwd
[NOTICE] (BatchNormalization)
[NOTICE] - - mobilenetv20_features_linearbottleneck14_elemwise_add0 (Add)
[NOTICE] - - mobilenetv20_features_linearbottleneck15_conv0_fwd (Conv)
[NOTICE] - - mobilenetv20_features_linearbottleneck15_batchnorm0_fwd
[NOTICE] (BatchNormalization)
[NOTICE] - - mobilenetv20_features_linearbottleneck15_relu0_fwd (Relu)
[NOTICE] - - mobilenetv20_features_linearbottleneck15_conv1_fwd (Conv)
[NOTICE] - - mobilenetv20_features_linearbottleneck15_batchnorm1_fwd
[NOTICE] (BatchNormalization)
[NOTICE] - - mobilenetv20_features_linearbottleneck15_relu1_fwd (Relu)
[NOTICE] - - mobilenetv20_features_linearbottleneck15_conv2_fwd (Conv)
[NOTICE] - - mobilenetv20_features_linearbottleneck15_batchnorm2_fwd
[NOTICE] (BatchNormalization)
[NOTICE] - - mobilenetv20_features_linearbottleneck15_elemwise_add0 (Add)
[NOTICE] - - mobilenetv20_features_linearbottleneck16_conv0_fwd (Conv)
[NOTICE] - - mobilenetv20_features_linearbottleneck16_batchnorm0_fwd
[NOTICE] (BatchNormalization)
[NOTICE] - - mobilenetv20_features_linearbottleneck16_relu0_fwd (Relu)
[NOTICE] - - mobilenetv20_features_linearbottleneck16_conv1_fwd (Conv)
[NOTICE] - - mobilenetv20_features_linearbottleneck16_batchnorm1_fwd
[NOTICE] (BatchNormalization)
[NOTICE] - - mobilenetv20_features_linearbottleneck16_relu1_fwd (Relu)
[NOTICE] - - mobilenetv20_features_linearbottleneck16_conv2_fwd (Conv)
[NOTICE] - - mobilenetv20_features_linearbottleneck16_batchnorm2_fwd
[NOTICE] (BatchNormalization)
[NOTICE] - - mobilenetv20_features_conv1_fwd (Conv)
[NOTICE] - - mobilenetv20_features_batchnorm1_fwd (BatchNormalization)
[NOTICE] - - mobilenetv20_features_relu1_fwd (Relu)
[NOTICE] - - mobilenetv20_features_pool0_fwd (GlobalAveragePool)
[NOTICE] - - mobilenetv20_output_pred_fwd (Conv)
[NOTICE] - - mobilenetv20_output_flatten0_reshape0 (Reshape)
Context: Consumer node (Conv2D#0) input #1
[WARNING] - No producer node attached to input#1 for node (Conv2D)
Context: Consumer node (Pad2D#0) input #0
[WARNING] - No producer node attached to input#0 for node (Pad2D)
Context: Consumer node (Pad2D#0) input #0
[WARNING] - No producer node attached to input#0 for node (Pad2D)
Context: Consumer node (ConvDepthWise2D#0) input #1
[WARNING] - No producer node attached to input#1 for node (ConvDepthWise2D)
Context: Consumer node (Pad2D#0) input #0
[WARNING] - No producer node attached to input#0 for node (Pad2D)
Context: Consumer node (Pad2D#0) input #0
[WARNING] - No producer node attached to input#0 for node (Pad2D)
Context: Consumer node (ConvDepthWise2D#0) input #1
[WARNING] - No producer node attached to input#1 for node (ConvDepthWise2D)
Context: Consumer node (Pad2D#0) input #0
[WARNING] - No producer node attached to input#0 for node (Pad2D)
Context: Consumer node (Pad2D#0) input #0
[WARNING] - No producer node attached to input#0 for node (Pad2D)
Context: Consumer node (ConvDepthWise2D#0) input #1
[WARNING] - No producer node attached to input#1 for node (ConvDepthWise2D)
Context: Consumer node (Pad2D#0) input #0
[WARNING] - No producer node attached to input#0 for node (Pad2D)
Context: Consumer node (Pad2D#0) input #0
[WARNING] - No producer node attached to input#0 for node (Pad2D)
Context: Consumer node (ConvDepthWise2D#0) input #1
[WARNING] - No producer node attached to input#1 for node (ConvDepthWise2D)
Context: Consumer node (Pad2D#0) input #0
[WARNING] - No producer node attached to input#0 for node (Pad2D)
Context: Consumer node (Pad2D#0) input #0
[WARNING] - No producer node attached to input#0 for node (Pad2D)
Context: Consumer node (ConvDepthWise2D#0) input #1
[WARNING] - No producer node attached to input#1 for node (ConvDepthWise2D)
Context: Consumer node (Pad2D#0) input #0
[WARNING] - No producer node attached to input#0 for node (Pad2D)
Context: Consumer node (Pad2D#0) input #0
[WARNING] - No producer node attached to input#0 for node (Pad2D)
Context: Consumer node (ConvDepthWise2D#0) input #1
[WARNING] - No producer node attached to input#1 for node (ConvDepthWise2D)
Context: Consumer node (Pad2D#0) input #0
[WARNING] - No producer node attached to input#0 for node (Pad2D)
Context: Consumer node (Pad2D#0) input #0
[WARNING] - No producer node attached to input#0 for node (Pad2D)
Context: Consumer node (ConvDepthWise2D#0) input #1
[WARNING] - No producer node attached to input#1 for node (ConvDepthWise2D)
Context: Consumer node (Pad2D#0) input #0
[WARNING] - No producer node attached to input#0 for node (Pad2D)
Context: Consumer node (Pad2D#0) input #0
[WARNING] - No producer node attached to input#0 for node (Pad2D)
Context: Consumer node (ConvDepthWise2D#0) input #1
[WARNING] - No producer node attached to input#1 for node (ConvDepthWise2D)
Context: Consumer node (Pad2D#0) input #0
[WARNING] - No producer node attached to input#0 for node (Pad2D)
Context: Consumer node (Pad2D#0) input #0
[WARNING] - No producer node attached to input#0 for node (Pad2D)
Context: Consumer node (ConvDepthWise2D#0) input #1
[WARNING] - No producer node attached to input#1 for node (ConvDepthWise2D)
Context: Consumer node (Pad2D#0) input #0
[WARNING] - No producer node attached to input#0 for node (Pad2D)
Context: Consumer node (Pad2D#0) input #0
[WARNING] - No producer node attached to input#0 for node (Pad2D)
Context: Consumer node (ConvDepthWise2D#0) input #1
[WARNING] - No producer node attached to input#1 for node (ConvDepthWise2D)
Context: Consumer node (Pad2D#0) input #0
[WARNING] - No producer node attached to input#0 for node (Pad2D)
Context: Consumer node (Pad2D#0) input #0
[WARNING] - No producer node attached to input#0 for node (Pad2D)
Context: Consumer node (ConvDepthWise2D#0) input #1
[WARNING] - No producer node attached to input#1 for node (ConvDepthWise2D)
Context: Consumer node (Pad2D#0) input #0
[WARNING] - No producer node attached to input#0 for node (Pad2D)
Context: Consumer node (Pad2D#0) input #0
[WARNING] - No producer node attached to input#0 for node (Pad2D)
Context: Consumer node (ConvDepthWise2D#0) input #1
[WARNING] - No producer node attached to input#1 for node (ConvDepthWise2D)
Context: Consumer node (Pad2D#0) input #0
[WARNING] - No producer node attached to input#0 for node (Pad2D)
Context: Consumer node (Pad2D#0) input #0
[WARNING] - No producer node attached to input#0 for node (Pad2D)
Context: Consumer node (ConvDepthWise2D#0) input #1
[WARNING] - No producer node attached to input#1 for node (ConvDepthWise2D)
Context: Consumer node (Pad2D#0) input #0
[WARNING] - No producer node attached to input#0 for node (Pad2D)
Context: Consumer node (Pad2D#0) input #0
[WARNING] - No producer node attached to input#0 for node (Pad2D)
Context: Consumer node (ConvDepthWise2D#0) input #1
[WARNING] - No producer node attached to input#1 for node (ConvDepthWise2D)
Context: Consumer node (Pad2D#0) input #0
[WARNING] - No producer node attached to input#0 for node (Pad2D)
Context: Consumer node (Pad2D#0) input #0
[WARNING] - No producer node attached to input#0 for node (Pad2D)
Context: Consumer node (ConvDepthWise2D#0) input #1
[WARNING] - No producer node attached to input#1 for node (ConvDepthWise2D)
Context: Consumer node (Pad2D#0) input #0
[WARNING] - No producer node attached to input#0 for node (Pad2D)
Context: Consumer node (Pad2D#0) input #0
[WARNING] - No producer node attached to input#0 for node (Pad2D)
Context: Consumer node (ConvDepthWise2D#0) input #1
[WARNING] - No producer node attached to input#1 for node (ConvDepthWise2D)
Context: Consumer node (Pad2D#0) input #0
[WARNING] - No producer node attached to input#0 for node (Pad2D)
Context: Consumer node (Pad2D#0) input #0
[WARNING] - No producer node attached to input#0 for node (Pad2D)
Context: Consumer node (ConvDepthWise2D#0) input #1
[WARNING] - No producer node attached to input#1 for node (ConvDepthWise2D)
The static scheduling is generated and displayed, without any execution of the graph. Here we see that except for Producers, the network operators execution order is strictly sequential.
To see a more interesting scheduling, one can try on a simple LSTM network. Lets first display the flatten LSTM graph we want to schedule:
[3]:
lstm = aidge_core.LSTM(in_channels=4, hidden_channels=8, seq_length=5)
# Flatten the graph:
lstm_model = aidge_core.get_connected_graph_view(lstm)
aidge_core.expand_metaops(lstm_model)
lstm_model.set_backend("cpu")
lstm_model.save("lstm_graph")
tuto_utils.visualize_mmd("lstm_graph.mmd")
Now lets generate the static scheduling for this graph:
[4]:
# Create the Scheduler
lstm_scheduler = aidge_core.SequentialScheduler(lstm_model)
lstm_scheduler.generate_scheduling()
# Display static scheduling
lstm_scheduler.save_static_scheduling_diagram("lstm_scheduling")
tuto_utils.visualize_mmd("lstm_scheduling.mmd")
In this LSTM example, the graph is cyclic and the scheduling therefore directly depends on the seq_length parameter. The generated static scheduling exhibits the early and late logical start for each operators. One can see that some operators have different early and late logical start, meaning their execution can happen anytime between these logical steps. Operators at the same logical step are garanteed to have no data dependency and may be executed in parallel. Conversely, operators with
identical early and late logical start are on the critical path in the scheduling.
There is a default C-P model associated to most operator implementations that is hybrid: when the inputs/outputs dimensions are known, it is elements-based, and when dimensions are unknown, it is token-based. There is no fundamental difference between element-based and token-based C-P model as long as operators are consuming and producing their whole input/output tensors at once at each execution step: in this case, the consumed elements will always match the produced elements anywhere in the graph, and any consumed or produced tensor can be considered as a single token. This is implicitely how the forward pass works in most DL frameworks.
When generate_scheduling() is called on a graph without known dimensions, the scheduling will be entirely token-based.
However, some operators cannot be statically scheduled with unknown dimensions! This is the case for the Pop operator: it extracts a sub-tensor along the first dimension of its input at each execution step. For exemple, with an input of shape [3, 16, 32], it will produce three [16, 32] tensors. If the input dimension is unknown, it is not possible to know how many tensors it must produce, hence how many time it must be scheduled. This is why Aidge provides the forward_dims() method:
[5]:
try:
aidge_model.forward_dims()
except Exception as error:
print(error)
[ERROR] - Missing mandatory input#0 for node [mobilenetv20_features_conv0_fwd -
[ERROR] (PaddedConv2D)]
Here, it fails with an error for the MobileNetv2 model because there is no input provided to the graph: the input dimension is unknown!
When called without argument, it is assumed that all the inputs of the graph have known dimensions (a tensor or a Producer is connected to each input). It is also possible to specify a list of expected size of the graph inputs:
[6]:
aidge_model.forward_dims(dims=[[1, 3, 16, 16]])
[NOTICE] - Reshape_Op: ignoring non-empty Shape attribute because input#1 takes
[6]:
False
[NOTICE] precedence
[WARNING] - Reshape_Op: unable to forwardDims() because output dims are data
[WARNING] dependent on input#1
[NOTICE] - Reshape_Op: ignoring non-empty Shape attribute because input#1 takes
[NOTICE] precedence
[WARNING] - Reshape_Op: unable to forwardDims() because output dims are data
[WARNING] dependent on input#1
[NOTICE] - Reshape_Op: ignoring non-empty Shape attribute because input#1 takes
[NOTICE] precedence
[WARNING] - Reshape_Op: unable to forwardDims() because output dims are data
[WARNING] dependent on input#1
[WARNING] - Unable to forward dimensions (circular dependency and/or wrong
[WARNING] dimensions and/or data dependent dimension?). Unable to compute
[WARNING] output dims for nodes ["mobilenetv20_output_flatten0_reshape0
[WARNING] (Reshape)"].
In this case, the forward_dims() method will automatically create missing input tensors of the specified size, or check that the existing inputs have the right size.
For some operators, the output dimensions cannot be deduced from its inputs dimensions alone. This is typically the case when the output dimension depends of some inputs data, rather than just dimensions, as for the Reshape operator for example. When this happens, forward_dims() will fail with a “Unable to forward dimensions” error and return False. There is a workaround however: if the required inputs data are known before model execution (for example, if the shape input of the
Reshape operator is simply a Producer), it is possible to force the evaluation of the required input data by setting the allow_data_dependency flag to True:
[7]:
aidge_model.forward_dims(dims=[[1, 3, 16, 16]], allow_data_dependency=True)
[NOTICE] - Reshape_Op: ignoring non-empty Shape attribute because input#1 takes
[NOTICE
[7]:
True
] precedence
[NOTICE] - Reshape_Op: ignoring non-empty Shape attribute because input#1 takes
[NOTICE] precedence
Beware that if some required data must be first computed, the result will be undefined, as the propagated dimensions will be invalid!
Note that when the model is executed, the operators output dimensions are automatically computed at runtime, without any need to call forward_dims() before-hand.
To summarize, three scheduling modes are possible depending on the C-P models available:
If all dimensions are known for each operator in the graph, a full data-based C-P scheduling can be performed providing that all operator C-P model supports data-based;
If only some dimensions are known, scheduling will be performed on data-based C-P model until the first node requiring tokens must be scheduled. Further nodes will then be scheduled using their token-based C-P model and any further node requiring an data-based C-P model will trigger an error;
If no dimension is known, only a full token-based scheduling can be performed, provided that all operator C-P model allows token-based.
Now, what if a graph cannot be scheduled just with the token-based C-P model (as with the Pop operator) and the data-based C-P model cannot be used because some dimensions are not known statically (as with the Reshape operator, assuming that allow_data_dependency cannot be used because some required input data must be computed first)?
In this case, the graph cannot be statically scheduled… oh, but remember: scheduling is always static in Aidge! This means that you will have to eliminate the data dependency in your graph, by either: 1) pre-compute the output dimension data dependent operator’s inputs, for example with the constant_folding() recipe, if applicable; or 2) isolate the data dependent path into a sub-graph and schedule and execute this sub-graph first, which will make the use of allow_data_dependency
possible.
Master the C-P model#
The scheduler objective is to produce data at each output node of the graph, until there is no data left to consume anymore, considering that Producers produce whole tensor data on demand.
The scheduling algorithm works the following:
Initialize the consumers list: start from the output nodes and find the required prior producers/consumers at step 2;
From the current consumers list, check if any prior consumer node is needed. A prior will generally be required for any node consuming parameters (weights and bias) that is not an input node.
If for a given node, only parent producers (at any depth) are needed to satisfy its required data, it becomes a prior.
If the prior node is a producer, it is added to the list of required producers.
If the prior node is of another type, it replaces the initial consumer in the new prior consumers list.
Prior consumers replace the initial consumers list. By construction, initial consumers will necessarily become consumers again later.
Make producers generate the required data. Producers are special nodes that generate data on demand.
Find runnable consumers. A consumer is runnable if the required data is available for all of its inputs. At this point, not all consumers are necessarily runnable because some may depend on the execution of others (when there is multiple successive priors for example).
Push runnable consumers in the list of nodes to run and update the consumer producer system. At this point, simultaneously runnable consumers have no data dependency and could be run in parallel!
Update consumers list:
If the current consumer has still data to consume (“still consumer”), it will be put back in the consumers list once the remaining consumers have been exhausted.
If the current consumer becomes a producer for other nodes, its childs become consumers.
If there is no more consumers, swap with possible “still consumer” (from step 7). This ensures that the “non-greedy” consumer behavior.
Iterate to step 2 until the consumers list is empty or there is no more runnable consumer.
Producers produce whole tensor data on demand#
Lets create a simple model:
[8]:
model = aidge_core.sequential([
aidge_core.Producer([16, 3, 512, 512], name="dataProvider"),
aidge_core.Conv2D(3, 4, [5, 5], name="conv1"),
aidge_core.ReLU(name="relu1"),
aidge_core.PaddedConv2D(4, 8, [5, 5], name="conv2", stride_dims=[1, 1], padding_dims=[2, 2, 2, 2]),
aidge_core.ReLU(name="relu2"),
aidge_core.PaddedConv2D(8, 16, [3, 3], name="conv3", stride_dims=[1, 1], padding_dims=[2, 2, 2, 2], no_bias=True),
aidge_core.ReLU(name="relu3")
])
model.set_backend("cpu")
# Create the Scheduler
scheduler = aidge_core.SequentialScheduler(model)
scheduler.generate_scheduling()
# Display static scheduling
scheduler.save_static_scheduling_diagram("scheduling")
tuto_utils.visualize_mmd("scheduling.mmd")
With the objective to generate data at the output node (relu3), the scheduling algorithm goes back to the first prior which is conv1, which will trigger the production of a whole tensor for Producer dataProvider. From that, conv1 becomes the first consumer, produces its output, then conv2 becomes a producer, etc. Once relu3 consumes its input tensor and produces an output tensor, the algorithm stops because there is no more consumer in the graph.
However, if at this point, relu3 would not have produced anything yet, that would mean, by construction, that there are still consumers somewhere in the graph. If that is the case, be aware that Producers continue to provide data on demand. Meaning if at some point, a node becomes a consumer that would only require new data at one of its Producer inputs, the required Producers would again produce a whole tensor.
Direct tensors produce whole data only once#
Direct tensors connection act like a Producer that would produce its whole tensor data only once.
Create a dataflow pipelining#
Here we create an example of C-P model that consume and produce data line by line.
[9]:
model = aidge_core.sequential([
aidge_core.Producer([1, 3, 16, 16], name="dataProvider"),
aidge_core.GenericOperator("Conv2D_DF", [aidge_core.InputCategory.Data, aidge_core.InputCategory.OptionalParam, aidge_core.InputCategory.OptionalParam], 1, name="conv1"),
aidge_core.GenericOperator("Conv2D_DF", [aidge_core.InputCategory.Data, aidge_core.InputCategory.OptionalParam, aidge_core.InputCategory.OptionalParam], 1, name="conv2"),
aidge_core.GenericOperator("Conv2D_DF", [aidge_core.InputCategory.Data, aidge_core.InputCategory.OptionalParam, aidge_core.InputCategory.OptionalParam], 1, name="conv3")
])
# Define a line-by-line dataflow C-P model for the Conv2D_DF operator.
class Conv2D_DataFlow_CP(aidge_core.ProdConso):
def __init__(self, op: aidge_core.Operator):
aidge_core.ProdConso.__init__(self, op, False)
self.state_begin = True
def get_nb_required_data(self, input_idx):
input = self.get_operator().get_input(input_idx)
if input:
if self.state_begin:
# Require 3 lines for 3x3 kernel
return aidge_core.Elts_t.data_elts(3 * input.dims()[2])
else:
return aidge_core.Elts_t.data_elts(input.dims()[2])
else:
return aidge_core.Elts_t.none_elts()
def get_required_memory(self, output_idx, inputs_size):
output = self.get_operator().get_output(output_idx)
self.state_begin = False
return aidge_core.Elts_t.data_elts(output.dims()[2])
Two methods must be overloaded to define a custom C-P model:
get_nb_required_data(): defines the amount of data required at each input to allow the operator execution. By default, it matches the data consumed at the next operator execution;get_required_memory(): defines the amount of data that will be produced at the next operator execution.
The C-P model is statefull, meaning it can hold state variables (like self.state_begin above) that change the amount of data consumed/produced at each execution step. Therefore, any operator behavior can be modeled without having to define the data path implementation as long as the C-P model does not depends on input data values.
Now, lets define an implementation for the generic operator that only holds the C-P model:
[ ]:
# Define an implementation for the Conv2D_DF operator that only contains the
# previously defined C-P model.
class GenericConv2D_DataFlow_Impl(aidge_core.OperatorImpl):
def __init__(self, op: aidge_core.Operator):
aidge_core.OperatorImpl.__init__(self, op, 'cpu')
def get_prod_conso(self):
return Conv2D_DataFlow_CP(self.get_operator())
The last step is to set the implementation of the generic operators defined in the graph to the one we just defined, and provide a function that define how to compute the operator’s output dimensions (thus enabling forward_dims() and element-based scheduling required for our element-based C-P model).
[ ]:
# Set the implementation and forward_dims for the Generic operators
for node in model.get_nodes():
if node.type() == "Conv2D_DF":
node.get_operator().set_impl(GenericConv2D_DataFlow_Impl(node.get_operator()))
node.get_operator().set_forward_dims(lambda x: [x[0]])
Now we can actually schedule the model and get a pipelined dataflow static scheduling!
The save_factorized_static_scheduling_diagram() function displays a compact form of scheduling where repetitive sequences have been factorized. The number of repetition of each sequence is specified left to the sequence (if not specified, there is no repetition).
[10]:
model.forward_dims()
# Create the Scheduler
scheduler = aidge_core.SequentialScheduler(model)
scheduler.generate_scheduling()
# Display static scheduling
scheduler.save_factorized_static_scheduling_diagram("scheduling")
tuto_utils.visualize_mmd("scheduling.mmd")
