Choose an inference engine
- Select an inference engine that that can be use on your specifice hardware board and that allow performance profiling.
According to your target you may need to:
- Install the inference engine on it.
- Build the inference engine in order to use it on the target.
Benchmarking tools
These are examples benchmarking tools provided by inference engine :
- See Profiling tool for OnnxRuntime.
- See Performance measurement for TFLite.
For MCU targets, the benchmark tool often need to be build from scratch.
⚠️
Warning: Note that some benchmark tools may not allow you to access the raw output of the model nor providing your own input data. This will limit your target to support only
TYPE1 benchmark type. Consider carefully the use of such tools.