On Device training Report Keys

This page describes all the keys that are used in the on device training report. The report output is a json file that contains all the keys even if they are not used.

Report available keys

General information keys

• version (type: str): This field is used to keep track of the report version. Currently, this field is not used.
• date (type: str) : Date and time of the generation of the report. Format : YY-MM-DD hh:mm:ss
• target (type: str): Target name
• target_id (type: str): Unique ID of the target
• version_tag (type: str): Version of the target used for the benchmark.
• train_engine (type: str): Train engine used to run the model.
• model_file_name (type: str): Name of the model used for the benchmark.
• benchmark_type (type: str): This field contains the type of benchmark from which the measures from. (TYPE1 : random data, TYPE2 : dataset preprocessed, TYPE3: raw dataset).

Hardware specific keys

• ram_size (type: int): Size in bytes of the available RAM of the system.
• ram_peak_usage (type: float): Percentage of the peak RAM usage.
• flash_size (type int): Size in bytes of the available FLASH.
• flash_usage (type: float): Percentage of the FLASH usage. Might not be relevant for Linux targets.
• load_cpu (type: float): Percentage of the CPU usage.
• load_accelerator (type: float): Percentage of the accelerator (GPU, NPU, …) usage.
• temperature (type: float): Temperature of the system (?) in °C.
• ambiant_temperature (type: float): Temperature in °C of the room where the system is.
• power_consumption (type: float): Power consumption of the system in Watt.
• enery_efficiency (type: float): Energy efficiency in Operation Per Watt (OPW).
• FLOPs (type: float): Operations per seconds

Hardware agnostic keys

• model_size (type: int): Size in bytes of the model on the target (after any compression / optimization performed by the engine).
• nb_parameters_model (type: int): Number of parameters of the model.
• loss_train (type: json):
  • min : (type: float): min loss recorded
  • max : (type: float): max loss recorded
  • <element list> (type: json): Each epoch results
    • epoch_index (type : int): Number of the training cycle
    • loss (type: float): New accuracy after training
    • time (type: flaot): Time took for the training cycle
• loss_test (type: json):
  • min : (type: float): min loss recorded
  • max : (type: float): max loss recorded
  • <element list> (type: json): Each epoch results
    • epoch_index (type : int): Number of the training cycle
    • loss (type: float): New accuracy after training
    • time (type: flaot): Time took for the training cycle
• odt_config (type: json): This key contains the configuration used for the on device training.
  • train_dataset(type: json):
    • nb_samples (type: int): Number of samples in the train dataset.
    • input (type: json):
      • shape (type: list): Shape of the input data.
      • dtype (type: str): Data type of the input data.
      • file_name (type: str): Name of the train dataset file.
    • output (type: json):
      • shape (type: list): Shape of the output data.
      • dtype (type: str): Data type of the output data.
      • file_name (type: str): Name of the train dataset labels file.
  • test_dataset(type: json):
    • nb_samples (type: int): Number of samples in the test dataset.
    • input (type: json):
      • shape (type: list): Shape of the input data.
      • dtype (type: str): Data type of the input data.
      • file_name (type: str): Name of the test dataset file.
    • output (type: json):
      • shape (type: list): Shape of the output data.
      • dtype (type: str): Data type of the output data.
      • file_name (type: str): Name of the test dataset labels file.
  • learning_parameters (type: json): This key contains the learning parameters used for the training.
    • epochs (type: int): Number of epochs used for the training.
    • batch_size (type: int): Batch size used for the training.
    • learning_rate (type: float): Learning rate used for the training.
    • optimizer (type: str): Optimizer used for the training.
    • loss_function (type: str): Loss function used for the training.

Report generation

The generated report should contain all the previously described keys. If you do not have a value for a given key, replace the value of this key by the keyword null. Example :

...
"ram_size": 4096,
"ram_peak": 0.7,
"flash_size": null,
...

Keys automatically overrided

The dAIEdge-VLab CI/CD automatically override some keys of the report generated by the Target. These keys are overrided to ensure their proper value.