Machine learning (ML) is becoming increasingly important for microcontrollers because it enables intelligent, autonomous decision-making in embedded systems. The many applications of the Internet of Things (IoT) — often called “smart devices” — only become smart thanks to ML.
Microcontrollers are commonly used in high-end computing devices where data is processed locally rather than being sent to a centralized server. ML on microcontrollers enables real-time data processing at the source, reducing latency and bandwidth requirements.
Before the emergence of concrete machine learning frameworks in the microcontroller space, ML models were used on tiny machines, manually optimizing algorithms and models for the specific constraints of microcontroller devices. Since then, ML has become much more customized, especially for microcontrollers. Customization is based on specific ML learning models for specific niche applications in the embedded space.
TensorFlow Lite was the first and most significant framework for using ML on microcontrollers. TensorFlow was released in 2015, and TensorFlow Lite followed soon after. It is specifically aimed at mobile and embedded devices.
“TensorFlow Lite for Microcontrollers” is designed to run ML models on microcontrollers and other devices with just kilobytes of memory. It provided pre-trained tools and models that could be easily deployed. This has opened the door to a wider range of developers and applications in Tiny Machine Learning (TinyML, the practice of deploying ML models to embedded or extremely resource-constrained devices).
Today, many other frameworks enable the deployment of machine learning models on microcontrollers. For an embedded engineer, it is essential to be familiar with at least one of these frameworks as ML is the future of embedded devices. After web servers and clouds, most other artificial intelligence (AI) applications will emerge from embedded devices. AI will also likely experience a shift from cloud computing to edge computing.
In this article, we will discuss different frameworks that can be used to deploy ML models on microcontrollers.
Why use ML in microcontrollers?
Machine learning is important for embedded devices, especially IoT. Everyday electronics can become “smarter” by directly integrating ML models into microcontrollers. This means they do not require connection to an external processor or cloud service for functions such as speech recognition, gesture detection, predictive maintenance and others.
Running ML models directly on microcontrollers enables real-time processing with minimal latency. This is essential for applications including safety-critical systems and user-interactive devices that require quick reactions.
Because microcontrollers are designed to be highly power efficient, which is vital for battery-operated devices, running ML models on these devices can lead to significant power savings compared to sending data to and from a cloud server for processing.
ML on microcontrollers also solves two major challenges in the path of smart devices – security and privacy:
- Reducing device dependency on a server or cloud service
- Eliminating the need to transmit data over the Internet increases security and privacy.
Since the data is processed locally on a secure device, there are fewer chances of a data breach. Local data processing further eliminates the need for constant connectivity and the bandwidth that comes with it. This is especially useful in places where Internet access is spotty or non-existent.
Embedding machine learning in microcontrollers is more cost-effective than setting up a cloud-based infrastructure for data processing. This also opens up possibilities for new applications and innovations, especially in areas where traditional ML deployment is not viable due to size, power, or connectivity limitations.
The main ML frameworks
TensorFlow Lite for microcontrollers is no longer the only framework for deploying machine learning on microcontrollers. There are several, as TinyML is one of the fastest changing fields. Some of the most significant of them are mentioned below.
1. TensorFlow Lite for Microcontrollers
2. Edge Boost
3. MicroML
4. Cubo.AI
5. TinyMLPerf
6. TinyNN
TensorFlow Lite for Microcontrollers is the oldest framework for deploying ML models on microcontrollers, which can be found in everything from wearables to IoT devices. It's an extension of TensorFlow Lite, Google's lightweight solution for mobile and embedded devices. Its runtime can fit into just 16 KB for some microcontroller models, making it ideal for battery-powered or energy-harvesting devices.
TensorFlow Lite can run on bare-metal microcontrollers without the need for a full operating system, providing a set of pre-trained models optimized for performance on resource-constrained devices. It supports multiple microcontroller architectures and is easily integrated into existing hardware configurations.
However, implementing ML on such constrained devices presents challenges. The main issues are memory limitations, power constraints and the need for model optimization. TensorFlow Lite addresses this through model quantization (reducing the precision of numbers in the model), pruning (removing parts of the model that are less critical), and efficient use of hardware accelerators when available. Some notable use cases include wearable devices, smart home devices, predictive maintenance in IIoT, crop monitoring, and real-time health monitoring.
Edge Boost is an all-in-one platform that allows developers to collect data, train a model in the cloud, and then deploy it to a microcontroller. It has integrated TensorFlow Lite for microcontrollers and offers an easy-to-use interface, making it suitable for those new to the field. It is designed for edge ML, providing tools and capabilities to collect, process, and analyze real-world data and then develop, train, and deploy ML models directly to edge devices. These devices range from small microcontrollers on sensors to more powerful single-board computers.
This platform allows developers to collect data from multiple sources, including existing sensors and datasets. They can then easily design, train, and validate ML models on this data through a dashboard. The platform also features automatic optimization techniques to ensure models are lightweight yet effective for edge deployment. Once a model is trained, developers can deploy it to a target edge device from the platform.
Edge Impulse supports an extensive list of hardware and architectures. Developers can test their models in real time and monitor their performance after deployment, enabling continuous improvement and optimization. More sophisticated AI algorithms, broader hardware support, and deeper integration with cloud services for cutting-edge hybrid cloud solutions are expected soon.
MicroML is a lightweight machine learning framework that runs directly on microcontrollers with no operating system. It enables on-device processing, which is critical for applications that require real-time decision making, such as autonomous vehicles or medical devices.
MicroML is at the forefront of a computing paradigm shift, enabling ML to work even in constrained environments. By directly implementing intelligent decision-making on microcontrollers, MicroML is driving new niches of machine learning. With advances in model compression techniques and the development of more powerful microcontrollers, the reach and effectiveness of MicroML applications will only increase. Furthermore, as IoT continues to grow, so will the demand for intelligent edge computing solutions like MicroML. Although this is a niche solution compared to others, it is suitable for a variety of applications.
Cubo.AI , developed by STMicroelectronics, is part of the STM32Cube software platform, a comprehensive suite of software for STM32 microcontrollers and microprocessors. It allows developers to convert pre-trained neural networks into C code optimized for STM32 microcontrollers.
Cube.AI takes neural network models, typically developed and trained on powerful computing resources, and converts them into a format optimized for STM32 microcontrollers. It supports models trained on popular ML frameworks such as TensorFlow and Keras, providing flexibility to developers. The tool optimizes models to run on microcontrollers' limited resources (memory and processing power) without compromising performance. Cube.AI could support more types of neural networks and better optimization algorithms and will offer broader compatibility with emerging ML frameworks in the future.
TinyNN is a small, efficient neural network framework for microcontrollers designed to be lightweight and easily portable. TinyNN is designed to be minimalist, requiring minimal memory and processing power, making it ideal for low-cost, low-power microcontrollers. It focuses on simplified neural network models that are powerful enough for a variety of tasks, but simple enough to run on microcontrollers. It emphasizes efficiency in memory usage and computational requirements, which is crucial for battery-operated or power-sensitive applications.
TinyNN's capabilities are expected to increase as developments in neural network design, model compression methods, and energy-efficient computing open up new applications for microcontrollers.
TinyMLPerf is not a framework, but an important benchmarking tool. It is part of the MLPerf suite for benchmarking TinyML solutions, measuring and comparing the performance of ML models on microcontroller systems. This is crucial for determining which frameworks or tools to use.
While MLPerf focuses on a wide range of ML platforms and applications, TinyMLPerf specifically targets low-power, resource-constrained microcontrollers commonly used in IoT and embedded systems. Offers a set of standardized tests, making it easy to compare different ML solutions in terms of efficiency, speed, and size. These benchmarks can be run on real hardware, reflecting real-world performance.
Benchmarks typically work by running a series of predefined ML tasks and measuring important parameters like execution time, power consumption, and model size. The results show how a given configuration performs under typical TinyML workloads.
TinyMLPerf covers tasks relevant to TinyML, such as image recognition, audio processing, and sensor data analysis. It allows developers and researchers to objectively compare the performance of different ML models and hardware platforms in a TinyML context. It also helps identify the strengths and weaknesses of different approaches, guiding future improvements.
Conclusion
TensorFlow Lite for Microcontrollers was the first framework for deploying machine learning on microcontrollers. With the changes in the AI landscape, other notable frameworks have emerged. For example, Edge Impulse is an all-in-one platform where developers can integrate TensorFlow Lite ML solutions with various hardware platforms and evaluate their performance in real time.
Furthermore, MicroML has emerged as a solution to implement ML models directly on microcontrollers. Cube.AI is optimizing many ML models in Tensor and Keras for microcontrollers. Additionally, TinyNN has introduced neural networks to microcontroller platforms. And TinyMLPerf evaluates the real-world performance of ML models and tools on various microcontroller systems.
