Heterogeneous deployment on Alif Ensemble E7 (A32 + dual M55)#

The Alif Ensemble E7 combines a Cortex-A32 application processor with two Cortex-M55 cores. Deploying one application across these processing domains requires more than generating model code: the graph must be partitioned, the regions must exchange tensors, each core must receive the correct firmware image, and the Secure Enclave must provision a consistent boot configuration.

In this tutorial, you will build a small neural network, quantize it to INT8, divide it into two execution regions, generate firmware for both M55 cores, add the A32 Linux application to the system image, program the board with SETools, and inspect the communication at runtime.

The graph execution implemented here is distributed between M55-HP and M55-HE. The Cortex-A32 boots Linux and runs the application-side communication program. This separation provides a clear starting point for extending the deployment with an A32 graph region later.

1. What will run where?#

Host notebook
  |-- quantize and adapt the graph for CMSIS-NN
  |-- split the static schedule at one tensor boundary
  |-- generate HP and HE firmware + A32 rootfs overlay
  `-- SETools provisions one combined ATOC

AKE7 at boot
  |-- Cortex-A32: Linux + aidge-apss-pingpong communication application
  |-- M55-HP: graph region 0, boundary producer and coordinator
  `-- M55-HE: graph region 1 and final output producer

M55-HP -- shared SRAM payload + MHU notification --> M55-HE
M55-HP <-- shared SRAM output  + MHU completion ---- M55-HE

The M55 protocol is synchronous. HP waits for HE READY, sends PING, waits for PONG, executes region 0, copies the boundary tensor into a reserved shared-memory window, signals HE through MHU, and waits for completion. The current window is 4096 bytes.

2. Hardware and software prerequisites#

You need an Alif Ensemble E7 AppKit connected through its programming/USB interface, Docker, permission to access the serial device and the AKE7 BSP package containing apss/, multicore/, tools/, board files and validated APSS artifacts. Close every serial terminal before SETools uses the port.

Identify your probe and UART#

Connect the AppKit and run:

pyocd list

Copy the value from the Unique ID column into PROBE_UNIQUE_ID in section 7. The ID stored in the AKE7 hardware model belongs to the reference/laboratory board and normally does not match the user’s probe. When SERIAL_PORT=None, Aidge uses this unique ID to select the USB UART associated with the correct board.

You may instead set SERIAL_PORT explicitly, for example /dev/ttyACM0. In that case UART auto-detection does not require the probe ID, although keeping it configured is recommended when more than one board is connected.

J15 jumper sequence (IMPORTANT)#

  1. Power off or reset the board.

  2. Put both J15-A and J15-B in the SE position (pins 1–3 and 2–4). This routes the USB bridge to the Secure Enclave UART (SEUART).

  3. Run the deployment cell below. SETools generates the ATOC and programs APSS plus both M55 load images.

  4. Keep watching the cell output. As soon as it prints ``SETools flashing succeeded``, the serial port is released and the same call advances to UART capture. At that point, move both jumpers to U4 (following your board/lab power-handling procedure) and reset/power-cycle the board before the capture timeout expires.

  5. U4 routes application printf output to the USB bridge, so the still-running cell can read the startup and benchmark output.

SE is for Secure Enclave provisioning. U4 is for application logs. Leaving J15 on U4 during SETools prevents provisioning; leaving it on SE afterward hides the benchmark output. APSS + dual-M55 currently requires flash_method="setools".

[ ]:

from __future__ import annotations

from dataclasses import replace
from pathlib import Path
import json
import numpy as np

import aidge_backend_cpu
import aidge_core
import aidge_deploy
from aidge_core.export_utils import ExportApplication, str2aidge
from aidge_core.export_utils.config import (
    BenchmarkConfig,
    CompareConfig,
    ExportConfig,
    ValidationConfig,
)
from aidge_core.export_utils.export import export, prepare_export, quantize_graph
from aidge_core.export_utils.heterogeneous_partition import (
    find_single_transfer_split,
    partition_sequential_graph,
)
from aidge_export_arm_cortexm.benchmark_helpers.cli import ConnectionConfig

np.random.seed(7)
aidge_core.random.Generator.set_seed(7)
print("Aidge heterogeneous deployment modules imported.")

3. Build the smallest useful graph#

The current AKE7 partitioner expects one external input, one external output, at least two scheduled compute nodes, and a partition boundary crossed by exactly one tensor. We use a linear four-operator graph so that the boundary is easy to identify while both M55 cores still receive useful work.

[ ]:

model = aidge_core.sequential(
    [
        aidge_core.FC(8, 8, name="hp_fc"),
        aidge_core.ReLU(name="hp_relu"),
        aidge_core.FC(8, 4, name="he_fc"),
        aidge_core.ReLU(name="he_relu"),
    ]
)
model.set_datatype(aidge_core.dtype.float32)
model.set_backend("cpu")

# Deterministic synthetic samples keep the example self-contained and are
# used for both quantization calibration and the target benchmark.
float_arrays = [
    np.random.uniform(-1.0, 1.0, size=(1, 8)).astype(np.float32) for _ in range(8)
]
float_tensors = [aidge_core.Tensor(array.copy()) for array in float_arrays]
for tensor in float_tensors:
    tensor.to_backend("cpu")
    tensor.to_dtype(aidge_core.dtype.float32)
    tensor.set_data_format(aidge_core.dformat.nchw)

scheduler = aidge_core.SequentialScheduler(model)
scheduler.generate_scheduling()
print(
    [
        node.name()
        for node in scheduler.get_sequential_static_scheduling()
        if node.type() != "Producer"
    ]
)

4. Quantize the graph to INT8#

Quantization uses representative floating-point samples to determine the integer ranges of the graph. It converts weights and activations to the representation required by the CMSIS-NN kernels. The samples supplied to the exported model must use the same INT8 representation.

[ ]:

quantize_graph(model, float_tensors)

int8_arrays = [
    np.round(np.clip(x, -1.0, 1.0) * 127).astype(np.int8) for x in float_arrays
]
int8_tensors = [aidge_core.Tensor(array.copy()) for array in int8_arrays]
for tensor in int8_tensors:
    tensor.to_backend("cpu")
    tensor.to_dtype(str2aidge("int8"))
    tensor.set_data_format(aidge_core.dformat.nchw)

print("Graph quantized; deployment input dtype: int8")

5. Preview graph adaptation for AKE7#

ExportConfig describes the target, datatype, layout and implementation library. This diagnostic call to prepare_export() adapts a cloned graph to AKE7/CMSIS-NN and creates a static schedule so that the two-region cut can be inspected.

The returned preview_ctx does not generate a project and is never passed to deploy. The public export, including heterogeneous partitioning, BSP resolution and both M55 entrypoints, happens explicitly in section 7.

[ ]:

export_cfg = ExportConfig(
    target="ake7",
    dtype="int8",
    dformat="nchw",
    heuristic="nb_op",
    cmsis=True,
    use_docker=True,
    clone_model=True,
)
preview_ctx = prepare_export(model, export_cfg, input_tensors=[int8_tensors[0]])
print(f"Target: {preview_ctx.device.name}")
print(f"Libraries: {preview_ctx.device.lib}")
print(f"Adapted nodes: {len(preview_ctx.graph.get_nodes())}")
print(f"Planned export folder: {preview_ctx.export_folder}")

6. Preview the exact two-region cut#

The AKE7 target examines the adapted static schedule and finds the cuts crossed by exactly one tensor. It selects the valid cut nearest the schedule midpoint, which creates two contiguous regions of similar size. Producer nodes containing weights and biases stay with their consumer and are not transferred at runtime. The following cell applies the partitioning helpers directly so that the selected nodes and boundary tensor are visible before deployment.

[ ]:

split_after = find_single_transfer_split(preview_ctx.scheduler)
preview = partition_sequential_graph(
    preview_ctx.graph,
    preview_ctx.scheduler,
    ("M55_HP", "M55_HE"),
    split_after=split_after,
    region_names={"M55_HP": "hp_model", "M55_HE": "he_model"},
)

for region in preview.regions:
    print(region.processing_element, "->", [n.name() for n in region.nodes])
transfer = preview.transfers[0]
boundary = transfer.tensor
boundary_bytes = boundary.size * ((aidge_core.dtype_bit_width(boundary.dtype) + 7) // 8)
print(f"Boundary: {transfer.source_node.name()} -> {transfer.destination_node.name()}")
print(f"Transfer size: {boundary_bytes} bytes (limit: 4096 bytes)")
assert len(preview.transfers) == 1
assert boundary_bytes <= 4096

7. Export the heterogeneous application and configure deployment#

The heterogeneous and APSS choices affect generated sources, BSP contents and the board-specific main, so they belong to ExportApplication and must be known before export() runs. The exporter creates a complete HP/HE/APSS project and records these choices in the artifact manifest.

ConnectionConfig separately describes runtime access to the board. The deploy configuration selects SETools, serial capture and build options, but it does not regenerate or patch the exported project.

[ ]:

SERIAL_PORT = None  # Example: /dev/ttyACM0; None enables auto-detection.
PROBE_UNIQUE_ID = "1219980188"  # Replace with the Unique ID reported by `pyocd list`.

if SERIAL_PORT is None and not PROBE_UNIQUE_ID:
    print("WARNING: run `pyocd list` and set PROBE_UNIQUE_ID before deployment.")

conn_cfg = ConnectionConfig(
    backend="local",
    serial_port=SERIAL_PORT,
    probe_unique_id=PROBE_UNIQUE_ID,
    flash_method="setools",
    heterogeneous_deploy=True,
    apss_linux=True,
)
bench_cfg = BenchmarkConfig(
    nb_warmup=2,
    nb_iterations=5,
    profiling=False,
)
application = ExportApplication(
    kind="benchmark",
    nb_warmup=bench_cfg.nb_warmup,
    nb_iterations=bench_cfg.nb_iterations,
    profiling=bench_cfg.profiling,
    heterogeneous_deploy=conn_cfg.heterogeneous_deploy,
    apss_linux=conn_cfg.apss_linux,
)

artifact = export(
    model,
    replace(
        export_cfg,
        application=application,
        model_prequantized=True,
    ),
    input_tensors=[int8_tensors[0]],
)

deploy_cfg = aidge_deploy.DeployConfig.from_connection_config(
    conn_cfg=conn_cfg,
    target=artifact.manifest.target,
    validation=ValidationConfig(validation_on_target=False),
    benchmark=bench_cfg,
    compare=CompareConfig(),
)
deploy_cfg.use_docker = export_cfg.use_docker

print(f"Heterogeneous project: {artifact.path}")
print(f"Manifest: {artifact.manifest_path}")
deploy_cfg

8. Build, provision and run#

Before enabling this cell, verify that both J15 jumpers are in SE, the board is connected, Docker can access /dev, and no terminal owns the serial port.

The previous export cell has already partitioned the graph, generated hp_model/he_model, resolved the AKE7 BSP, created both M55 projects and prepared the APSS sources. Deployment now performs only operational stages:

  1. Build both M55 TCM load images in Docker.

  2. Build the A32 ping-pong recipe and incremental cramfs-xip rootfs.

  3. Combine APSS firmware/rootfs and both M55 images into the SETools configuration.

  4. Program the ATOC through SEUART.

  5. Capture UART output and parse the benchmark result.

The cell remains active after programming. When its console prints ``SETools flashing succeeded``, promptly move both jumpers to U4 using the safe procedure for your board and reset/power-cycle it before UART capture times out.

[ ]:

RUN_HARDWARE_DEPLOYMENT = True  # Read the J15 instructions above first.

if RUN_HARDWARE_DEPLOYMENT:
    if SERIAL_PORT is None and not PROBE_UNIQUE_ID:
        raise ValueError(
            "AKE7 UART auto-detection requires PROBE_UNIQUE_ID. "
            "Run `pyocd list` and copy the probe Unique ID, or set SERIAL_PORT explicitly."
        )
    result = aidge_deploy.deploy(artifact, deploy_cfg)
    print("Deployment output:", result.output)
    print("Peak memory (bytes):", result.memory_peak_bytes)
else:
    result = None
    print(
        "Project exported but not deployed. Set RUN_HARDWARE_DEPLOYMENT=True when J15 is in SE."
    )

9. Inspect what the exporter generated#

After export, partition.json describes the generated execution plan: the nodes assigned to each region, the split point, the tensor C types and the transfer sizes. The HP wrapper calls hp_model_forward(), copies the boundary tensor to shared memory and requests HE execution. The HE project contains the generated implementation of the second region.

[ ]:

export_dir = artifact.path
manifest_path = export_dir / "heterogeneous" / "partition.json"
if manifest_path.is_file():
    manifest = json.loads(manifest_path.read_text())
    print(json.dumps(manifest, indent=2))
    for relative in [
        "dnn/src/forward.cpp",
        "heterogeneous/he/dnn/src",
        "multicore/he/heterogeneous_model.h",
        "apss/out/alif-tiny-image-devkit-e7.cramfs-xip",
    ]:
        print(relative, "->", (export_dir / relative).exists())
else:
    print("Manifest not generated; rerun the public export cell in section 7.")

10. Reading the communication logs#

The expected M55 startup handshake is:

M55-HP: M55-HE ready
M55-HP: PING sent
M55-HP: PONG received
DUAL_CORE_TEST: PASS

Only after this test does inference start. On the first inference, HP reports entering/completing its graph; HE consumes the shared boundary and returns the output. The raw MHU protocol between HP and HE is independent from Linux.

The A32 application separately opens the Linux RPMsg character device and requests the m55_hp_mhu0 endpoint. The current M55 firmware uses a raw-MHU protocol for HP/HE graph execution and does not advertise the corresponding Linux RPMsg service. Consequently, the A32 application exercises the Linux communication path but does not execute a third graph region. If you want to interact with your A32 core using Linux, you must change the jumpers J15 to U2 instead of U4.

11. From this example to a real A32 + M55 application#

To make the A32 perform useful work, extend the design in layers:

  1. Placement: introduce an A32 execution region instead of assuming exactly (M55_HP, M55_HE). Choose boundaries using measured compute/transfer cost, not only schedule midpoint.

  2. Export: generate an A32/Linux implementation for its region (for example C++ with an appropriate CPU backend) and package it in the Yocto recipe.

  3. Transport ABI: define tensor descriptors containing shape, dtype, byte count, sequence number and status. Replace the fixed implicit payload with negotiated buffers.

  4. Linux service: make M55-HP advertise the RPMsg endpoint expected by aidge-apss-pingpong; turn the A32 test program into a worker that sends/receives real tensors.

  5. Synchronization and safety: add timeouts, cache maintenance, memory barriers, ownership rules and error recovery. Shared memory content alone is not synchronization; MHU/RPMsg messages transfer ownership.

  6. General graphs: support multiple crossing tensors, residual branches, dynamic sizes and transfers larger than 4096 bytes. The current exporter deliberately rejects these cases early.

  7. Validation: compare the distributed output with a host reference before optimizing overlap or asynchronous execution.

The tiny graph in this notebook is useful precisely because each layer can be changed and verified independently before attempting a residual network such as ResNet-8.

12. Troubleshooting checklist#

  • SETools cannot connect: both J15 jumpers must be on SE; close serial monitors; verify the selected /dev/ttyACM* and Docker device access.

  • No UART output after flashing: move both J15 jumpers to U4 and reset/power-cycle.

  • Missing ``apss/`` or ``multicore/``: install/publish the current AKE7 BSP, not an older package.

  • Boundary exceeds 4096 bytes: choose another cut, reduce the feature tensor or extend the shared-memory transport.

  • No valid cut: the adapted graph has zero or multiple tensors crossing every possible midpoint; inspect branches/residuals and provide a placement that yields one crossing tensor.

  • First APSS build is slow: this is expected; preserve the aidge-alif-apss-work Docker volume for incremental rebuilds.

  • Notebook becomes very slow or its kernel is killed during BitBake: verify that Alif log streaming is line-buffered and keeps only a bounded diagnostic tail. Character-by-character streaming creates excessive Jupyter protocol traffic. Restart the kernel after updating aidge_deploy, and clear the failed cell output before running again.

  • ``A32_M55_TEST`` does not pass: the current raw-MHU M55 firmware does not expose the Linux RPMsg service; this is a known architectural limitation, separate from DUAL_CORE_TEST: PASS.

  • ``Trying to change a Node name … already used by another Node``: both regions generated the same scheduler name, such as _0_FC_0. Ensure that the AKE7 exporter namespaces completed regions before exporting the next one, then rerun the deployment cell.