VirtualLab's CI/CD

The dAIEdge-VLab’s CI/CD pipeline is the main pipeline that is used to run the different scripts. This pipeline is triggered by the dAIEdge-VLab’s user interface or by the functional test pipeline. The pipeline is composed of different stages that are executed in a specific order. Furthermore, it provides several environment variables.

What files does the pipeline provide ?

The CI/CD gives you access to the model and the dataset preprocessed (if available). Here is a quick look at these two main input file :

• Model : The model is a pre-trained model given by the user. The format of this model is fixed by you here (e.g. onnx, .tflite). Note that you should be able to determin the input and output size of the model uesing only this file.
• Dataset : If the $BENCHMARK_TYPE variable is set to TYPE2, a binary file of the dataset preprocessed will be provided. This binary file contains the raw input data that must be used to perfrom the benchmark. The binary file should be composed of INPUT_SIZE_BYTES * NB_SAMPLES bytes. You should ensure the validity of the dataset yourself. You may throw an error if the size is not vaild.

Both of these models are automatically placed at the root folder of the target repository. You may access them uesing the environment variables $MODEL_FILENAME and $DATASET_FILENAME.

Here is an example in python to leverage the use of the environment variables :

import os

model_path = os.environ['MODEL_FILENAME'] # If the work directory is the root foler.

Execution

You can have a look at the complete main pipeline of the dAIEdge-VLab in the file here.

The main stage of the dAIEdge-VLab’s CI/CD pipeline is the build-deploy stage. This stage does the following:

1. Clones the target source code using the environments variables defined in the Docker Image.
2. Downloads the model from the package registry at the location target_src/. This directory is the root directory of the target source code.
3. Downloads the dataset from the package registry at the location target_src/ if needed. This directory is the root directory of the target source code.
4. Clean all the files and environments variables that are not needed for the target to run.
5. Runs the target script AI_Support if the error.log is empty.
6. Runs the target script AI_Build if the error.log is empty.
7. Runs the target script AI_Deploy if the error.log is empty.
8. Runs the target script AI_Manager if the error.log is empty.
9. Finalizes the report automatically. It adds the known values to some keys.
10. Prints any errors that occurred during the execution of the scripts.
11. Saves the output of the AI_Manager script in the target_src/AI_Manager/out directory alongside the error.log and user.log files if they exist.

Environment variables

This section describes the environment variables that are usefull for the integration of the target.

Variables describing the requested job

Variables for the dAIEdge-VLab