Aidge backend OpenCV API#

Introduction #

The OpenCV backend provides its own operators and is particularly useful for data pre-processing. It is possible to create pre-processing pipelines using the same graph IR as Aidge core.

The OpenCV backend, alongside its database drivers, serves as a drop-in replacement for Torch datasets and data loaders. You can use either with Aidge, depending on your front-end and backend preferences (Torch mostly relies on Python Imaging Library (PIL) instead of OpenCV). Performance is similar for both, with a slight advantage for the Aidge OpenCV backend.

Operators specific to the OpenCV backend are usually suffixed by Transformation.

Warning

In accordance with OpenCV, color images are loaded in BGR format. Gray images are not automatically converted to color images (images are loaded as is).

To ensure that all images share the same RGB format (for example), add the following operator in the transforms pipeline: aidge_backend_opencv.ColorSpaceTransformation ( aidge_backend_opencv.colorspace.rgb ).

Note

OpenCV uses the HWC format when loading multi-channel images. This is the expected format for the Transformation operators.

Data in HWC are automatically converted to CHW, which is Aidge default format, in the aidge_core.DataProvider at the end of the transforms pipeline.

Example:

aidge_database = aidge_backend_opencv.CIFAR10(data_path="/local/DATABASE/cifar-10-batches-bin", train=True)
aidge_dataprovider = aidge_core.DataProvider(aidge_database,
                                          backend=BACKEND,
                                          batch_size=BATCH_SIZE,
                                          shuffle=True,
                                          drop_last=True)

aidge_dataprovider.transforms(0, aidge_core.sequential([
      aidge_backend_opencv.FlipTransformation(True, True)
   ]))

About ROI interface #

This backend also includes a ROI handling and transformation mechanism. The goal is to allow easy and efficient data augmentation for objects detector training. It will be possible to apply the same pre-processing pipeline to images, pixel-wise labels and ROI labels seamlessly.

Example:

import aidge_core
import aidge_backend_opencv as aicv
import numpy as np

# Image pre-processing flow
preproc = aidge_core.sequential([
   aicv.RandomFlipTransformation(0.5, 0.5)
])

a = aidge_core.Tensor(np.zeros([32, 32], dtype=np.float32))
a.to_backend("opencv")

scheduler = aidge_core.SequentialScheduler(preproc)
scheduler.forward(forward_dims=False, data=[a])[0]

# ROI pre-processing flow
preproc_roi = aicv.ROIGraph(preproc)

r = aicv.CircularROI(1, (1, 2), 3.0)
rois = aicv.ROIs([32, 32])
rois.data = [r]

scheduler = aidge_core.SequentialScheduler(preproc_roi)
out = scheduler.forward(forward_dims=False, data=[rois])[0]

print(out.data[0])

Synchronization between different modalities flow (image and ROI for example) is done with shared Producer_Op for random transformations. The ROI flow can be automatically constructed from the image flow with the ROIGraph method: preproc_roi = aicv.ROIGraph(preproc). The shared Producer_Op output value is set by a forward hook applied to the image flow Producer operator.

        %%{init: {'flowchart': { 'curve': 'monotoneY'}, 'fontFamily': 'Verdana' } }%%
flowchart TB

   subgraph shared operator
      Producer_0("<em>Producer#0</em>"):::producerCls_rootCls
      Producer_roi_0("<em>Producer#0</em>"):::producerCls
   end
   subgraph shared operator
      Producer_1("<em>Producer#1</em>"):::producerCls_rootCls
      Producer_roi_1("<em>Producer#1</em>"):::producerCls
   end
   FlipTransformation_0("<em>FlipTransformation#0</em>")
   Producer_0-->|"0 [1] boolean<br/>&darr;<br/>1"|FlipTransformation_0
   Producer_1-->|"0 [1] boolean<br/>&darr;<br/>2"|FlipTransformation_0
   input0((in#0)):::inputCls--->|" [32, 32] float32<br/>&darr;<br/>0"|FlipTransformation_0
   FlipTransformation_0--->|"0 [32, 32] float32<br/>&darr;"|output0((out#0)):::outputCls
   classDef inputCls fill:#afa
   classDef outputCls fill:#ffa
   classDef externalCls fill:#ccc
   classDef producerCls fill:#ccf
   classDef genericCls fill:#f9f9ff,stroke-width:1px,stroke-dasharray: 5 5
   classDef metaCls stroke-width:5px
   classDef rootCls stroke:#f00
   classDef producerCls_rootCls stroke:#f00,fill:#ccf
   classDef genericCls_rootCls stroke:#f00,fill:#f9f9ff,stroke-width:1px,stroke-dasharray: 5 5
   classDef metaCls_rootCls stroke:#f00,stroke-width:5px

   FlipTransformation_ROI_0("<em>FlipTransformation_ROI#0</em>")
   Producer_roi_1-->|"0 [1] boolean<br/>&darr;<br/>1"|FlipTransformation_ROI_0
   Producer_roi_0-->|"0 [1] boolean<br/>&darr;<br/>2"|FlipTransformation_ROI_0
   input_roi0((in#0)):::inputCls--->|"&darr;<br/>0"|FlipTransformation_ROI_0
   FlipTransformation_ROI_0--->|"0<br/>&darr;"|output_roi0((out#0)):::outputCls

Utils #

Tensor Aidge::cvMatToTensor(const cv::Mat &mat)#

Instantiate an aidge tensor with backend “opencv” from an opencv matrix.

Parameters:: mat – the cv::mat to instantiate the tensor from
Returns:: Tensor aidge tensor

void Aidge::cvMatToTensor(const cv::Mat &mat, Tensor &tensor)#

Aidge backend OpenCV API#

Introduction #

About ROI interface #

Predefined operators #

AffineTransformation #

CentroidCropTransformation #

ChannelExtractionTransformation #

ColorSpaceTransformation #

CompressionNoiseTransformation #

CropTransformation #

FlipTransformation #

PadCropTransformation #

RangeAffineTransformation #

RescaleTransformation #

SliceExtractionTransformation #

TrimTransformation #

Predefined operators acting on ROI #

CropTransformation_ROI #

FlipTransformation_ROI #

PadCropTransformation_ROI #

RescaleTransformation_ROI #

SliceExtractionTransformation_ROI #

Databases #

MNIST #

CIFAR10 #

CIFAR100 #

Directory #

ILSVRC2012_Directory #

Utils #

This Page