I'm writing this post to announce the release of cONNXr, a project I have been working on since last year.
It's an ONNX backend written in pure C that allows running inference for machine learning models defined in .onnx.
It has zero dependencies and it's meant to work in any embedded device.
After having a working proof of concept by the end of 2019, the project was forgotten for some time. But some months later, with the lockdown and the pandemic I had some spare time to continue working on it. And this is the result. Just so you know, it's an unfinished project and not meant for production usage, so please use with care.
There are tons of unimplemented operators but some models are already supported out of the box:
- mnist
- tiny YOLO v2
- super resolution
- mobilenet v2
I got the idea of cONNXr while working at Ericsson in Sweden. We developed an algorithm for Link Adaptation in LTE networks that used neural networks.
But well, since everything was done "in the edge", more particularly in Ericsson Digital Units where you can't just install Tensorflow or Keras,
we had to implement everything from scratch. The model wasn't very complicated but it took us some time to develop the inference backend from scratch.
And by the way, this feature was planned to be showcased during the Mobile World Congress 2020 but was canceled due to the pandemic.
While facing this problem, I also discovered the open standard for machine learning ONNX:
ONNX is an open format built to represent machine learning models. ONNX defines a common set of operators - the building blocks of machine learning and deep learning models - and a common file format to enable AI developers to use models with a variety of frameworks, tools, runtimes, and compilers.
So I decided to connect both ONNX and my experience writing the backend for inference and cONNXr was born.
The c stands for C programming language, ONNX is self-explanatory, and r for runtime.
The project is organized in a modular way and while many operators are missing, the architecture allows anyone to implement a new one without having much context about the project.
Each operator is defined in a black box fashion with a clearly defined set of inputs/outputs. Newly implemented operators, must respect the ONNX standard.
Thanks also to the open-source contributors that helped during the last months.
The project is licensed under MIT License, but well, do whatever you want with it.