VirtualLab's CI/CD

dAIEdge-VLab

VirtualLab's CI/CD

The dAIEdge-VLab’s CI/CD pipeline is the main pipeline that is used to run the different scripts. This pipeline is triggered by the dAIEdge-VLab’s user interface or by the functional test pipeline. The pipeline is composed of different stages that are executed in a specific order.

Stages

The main stage of the dAIEdge-VLab’s CI/CD pipeline is the build-deploy stage. This stage does the following:

Clones the target source code using the environments variables defined in the Docker Image.
Clean all the files and environments variables that are not needed for the target to run.
Downloads the model from the package registry at the location target_src/. This directory is the root directory of the target source code.
Runs the target script AI_Support.
Runs the target script AI_Build.
Runs the target script AI_Deploy.
Runs the target script AI_Manager.
Prints any errors that occurred during the execution of the scripts.
Saves the output of the AI_Manager script in the target_src/AI_Manager/out directory alongside the error.log and user.log files if they exist.

Here is the exact CI/CD configuration for the build-deploy stage:

.gitlab-ci.yml

build-deploy:
  stage: build-deploy
  tags:
    - $TARGET
    - $RUNTIME
  rules:
    - if: $EXECUTE_FUNCTIONAL_TEST_PIPELINE == "true"
      when: never
    - when: always
  before_script:
    - echo "Installing dependencies"
    - if ! command -v jq &> /dev/null; then apt update && apt install -y jq; else echo "jq is already installed."; fi
    - export FUNCTIONS_PATH=`pwd`/src
  script:
    - echo "🚀 Start pipeline V1.1"
    # Set error.log path and exit functions scripts
    - export WORKSPACE_PATH=`pwd`
    - export LOG_PATH=`pwd`
    - export FUNCTIONS_PATH=`pwd`/src
    # Create a root folder to run target script. Important to respect path of all files
    - mkdir target_src
    # Clone target source into target_src
    - ./src/init.sh target_src
    # Remove virtuallab sources
    - ./src/clean.sh
    - rm ./src/clean.sh
    # Remove virtuallab env
    - unset test_bot_api_token
    # Move to target_src
    - cd target_src
    # Get model from package registry
    - 'wget --header "JOB-TOKEN: ${CI_JOB_TOKEN}" "${CI_API_V4_URL}/projects/${CI_PROJECT_ID}/packages/generic/Model/v1.0.0/${MODEL_FILENAME}"'
    # Run target scripts
    - if [ ! -s "$LOG_PATH/error.log" ]; then bash ./AI_Support/support.sh; else echo "⏩ support.sh was not executed due to previous errors."; fi
    - if [ ! -s "$LOG_PATH/error.log" ]; then bash ./AI_Build/build.sh; else echo "⏩ build.sh was not executed due to previous errors."; fi
    - if [ ! -s "$LOG_PATH/error.log" ]; then bash ./AI_Deploy/deploy.sh; else echo "⏩ deploy.sh was not executed due to previous errors."; fi
    - if [ ! -s "$LOG_PATH/error.log" ]; then bash ./AI_Manager/manager.sh; else echo "⏩ manager.sh was not executed due to previous errors."; fi

    # Finalize the report
    - if [ ! -s "$LOG_PATH/error.log" ]; then bash ../src/report_finalizer.sh; else echo "⏩ report_finalizer.sh was skipped."; fi
    # Print any errors
    - |
      echo "Print error.log :"
      cat $LOG_PATH/error.log      
    # Print any user log
    - |
      echo "Print user.log :"
      cat $LOG_PATH/user.log      


  inherit:
    variables:
      - MODEL_FILENAME
      - NB_INFERENCE
      - TARGET
      - RUNTIME
      - EXECUTE_FUNCTIONAL_TEST_PIPELINE
      - VIRTUALLAB_PIPELINE_NAME
      - VERSION_TAG

  variables:
      FUNCTIONS_PATH: "../src/"
  
  environment: $TARGET
  resource_group: $TARGET
  artifacts:
    when: always
    paths:
      - ./target_src/AI_Manager/out
      - ./error.log
      - ./user.log

⚠️

Security : If you find any security issues or data leaking regarding the currently stated dAIEdge-VLab implementation, please report it to the maintainers.

dAIEdge-VLab Troubleshooting