Data#

Tensor#

A Tensor is a multi-dimensional array defined by its dimensions, its datatype and its precision.

A Tensor can be used to represent:

  • A raw input of a DNN, such as an image or a time serie;

  • A label associated to a raw input;

  • A processed input or label computed through a DNN;

  • A parameter of a DNN, such as a weight.

AIDGE can define Tensor having the following datatype and precision:

Datatype

Precision

Float

64, 32, 16

Int

64, 32, 16, 8, 7, 6, 5, 4, 3, 2, 1

UInt

64, 32, 16, 8, 7, 6, 5, 4, 3, 2, 1
../../_images/TensorRegistry.PNG

Data Structure#

To load Data for training or testing the DNN, AIDGE uses a structure based on 3 objects, Stimuli, Database and DataProvider. Each object provide functionnalities in order to load and preprocess data for neural network inference :

  • Stimuli object : determines the appropriate loading function for a particular data format;

  • Database object : handle both data multimodality and data pre-processing;

  • DataProvider object : compose batch in a Tensor;

The three objects are combined in the following structure :

../../_images/Database.PNG

An item a collection of data with different modalities. For example an item for MNIST database is the collection of a digit and its associated label.

Stimuli#

The Stimuli object determines the appropriate loading function associated to the a given data format. The Registrar mecanisms allow to choose load function according to the data format.

Moreover, different loading function for different data format will have different tier library dependancies. Each loading function are implemented in the module corresponding to its tier library dependancies. For example, the loading function of an image in an opencv matrix is implemented in the backend_opencv module.

Database#

The Database is an abstract object representing a map from a key to a data. All databases should inherit from this class.

Concretly, a Database class is an object representing a map from a key to a tuple of Stimuli. This map is reprensented by the variable mStimulis. Each Stimuli correspond to a data modality. Each database can have its own data modalities. For example the MNIST database have two modalities :

  • Data modality 1 - The image of a digit;

  • Data modality 2 - The label, the number of the digit;

Each data modality may have a different pre-processing, hence each data modality have an associated pre-processing sequential graph. The data pre-processing are reprensented by the variables mDataTransformation and mSchedulers.

The get_item(index) function operates in the following manner :

  • Load the data by calling the stimuli load function;

  • Apply the preprocessing;

  • Return the tensors of the pre-processed data;

DataProvider#

The purpose of the DataProvider is to provide the DNN model with the input Data read by the Database. The DataProvider can generate a Tensor Batch, which takes the form of a Tensor with a dimension added to concatenate the various input Tensor.

For example a 2D image RGB 64x64 px have the following dimension (using CHW format): (3,64,64). A batch size of 32 will return a Tensor of size (using NCHW format) (32,3,64,64).

The DataProvider calls the get_item(index) function to compose the batch one item at a time.

Data partitioning#

Data partioning is achieved by instanciating several database objects. For example one database for the training data and one database for the test data.

Also an existing database can be randomly split into two non-overlapping database instances with the function splitDatabases (under developpement). For example, this function is usefull to create a validation data database from the training database.