Machine learning (ML) and artificial intelligence (AI) are no longer limited to high-end servers or cloud platforms. Thanks to new developments in integrated circuits (IC) and software technology, it is possible to implement ML algorithms and deep learning neural networks on small controllers and microcomputers. And these edge-mounted embedded devices should no longer rely on a remote server or cloud to derive insights from sensor data or user input.
Software frameworks like TinyML are evolving as microcontroller-specific ML solutions, and conventional deep learning frameworks can be implemented on powerful microcomputers. This offers several advantages.
For one, devices don't rely on network connectivity or the availability of a cloud service to add AI to the system. Secondly, microcontrollers and microcomputers are much more energy efficient than a web-based AI services server.
Microcontrollers can now also perform specific AI tasks while running in a coin cell. Compared to a high-end workstation that typically costs thousands of dollars, an ML-capable microcontroller costs less than a few hundred dollars. Another benefit of implementing AI on the edge device is to protect user data privacy and reduce the chances of cyber attacks and hackers. Microcontrollers are ubiquitous and can be deployed on a large scale for machine learning tasks.
There are several microcontrollers and single-board microcomputers available that can be used to develop AI-enabled embedded applications. Let's explore the main platforms.
NVIDIA Jetson Nano Developer Kit
The NVIDIA Jetson Nano Developer Kit is one of the most flexible standalone machines that can be used to deploy AI software at the edge. This microcomputer can run multiple neural networks in parallel.
Based on the quad-core ARM A57 processor and 128-core NVIDIA Maxwell GPU, the Jetson Nano can deliver 472 GFLOPS of computer performance. It also houses an integrated 4GB 64-bit LPDDR4 RAM at 1600 MHz. There are two power modes, 5W/10W with 5V DC input, and the board costs just $99.
It's easy to get started by simply inserting a MicroSD card containing the system image. The card is programmed for AI applications using the NVIDIA JetPack SDK. The latest SDK provides a complete Linux environment based on Ubuntu 18.04. The SDK also provides accelerated graphics with support for NVIDIA CUDA Toolkit 10.0 and GPU-accelerated libraries such as Tensor RT5 and cuDNN 7.3.
Supported ML frameworks include PyTorch, Keras, TensorFlow, Caffe/Caffee2, MXNet, and others. The SDK also supports OpenCV for computer vision and ROS for robotics applications.
What's more, this developer kit includes a DeepStream SDK that offers a complete streaming analytics toolkit for AI-based video and image processing. NVIDIA can process up to 8 full-motion HD video streams in real time. Jetson Nano is an excellent platform for deploying AI-based inference workloads for applications such as image classification, segmentation, object detection and localization, video enhancement, pose estimation, and speech processing.
The board features Gigabit Ethernet, HDMI 2.0, Display Port 1.3, MIPI CSI-2 camera interface, four USB 3.0 ports, a MicroSD card slot, and a 40-pin GPIO connector. The board can receive power via a 5V DC cylindrical plug adapter or Micro USB port. Power consumption can be set to just 5 Watts. The camera connector is compatible with 8MP IMX219 and Raspberry Pi camera module V2.
Jetson Nano is capable of inferring multiple deep neural network (DNN) models with real-time computer vision. Even neural networks can be retained locally through transfer learning. This board is an excellent choice for a variety of applications, including IoT with intelligent edge analytics, multi-sensor autonomous robots, video analytics, image recognition, and gesture recognition.
Google Coral Developer Council
Google Developer Board Coral is currently the most energy-efficient development board for edge ML inference. Based on NXP i.MX 8M SoC (including Quad-core Cortex A53 and Cortex-M4F), with integrated GC7000 Lite graphics GPU and Google Edge TPU, Coral Dev can deliver 4 TOPS computing performance. Furthermore, the card consumes only 0.5 Watt per TOPS. RAM is 1 or 4 GB LPDDR.
This development board requires a 5V DC supply for operation and its GPIO is 3V3 compatible. There's also 8GB onboard eMMC memory and a MicroSD card slot. The board costs $149.99.
One advantage that Coral Dev offers is that it combines ML with wireless connectivity. Along with Gigabit Ethernet, Coral Dev includes WiFi (802.11b/g/n/ac 2.4/5GHz) and Bluetooth 4.2. Another advantage is its removable system-on-module (SoM) that can scale efficiently between prototyping and production. The biggest drawback is that it only supports TensorFlow Lite and no other deep learning frameworks. However, this board runs a derivative of Debian Linux and supports several popular Linux tools, which is useful.
Coral Dev is ideal if you plan to deploy ML inference using TensorFlow Lite with minimal time to market. The card delivers impressive computational performance and can run the latest MobileNet v2 mobile vision models at 400 FPS. Users can do almost anything in Coral Dev that is within the scope of the TensorFlow Lite framework.
Raspberry Pi
Raspberry Pi was a single-board computer so popular that it sold out in 2021 due to its huge demand in the manufacturing industry. The latest Raspberry Pi 4 Model B is based on the powerful Broadcom BCM2711 Quad-core 64-bit Cortex A72 SoC clocked at 1.5 GHz. This board also features Broadcom VideoCore VI GPU and 1/2/4 GB of LPDDR4 RAM and can provide computing performance of 13.5 to 32 GFLOPS.
The cost of the card starts at $35 for a 1GB RAM model. The 2GB and 4GB RAM models cost $45 and $55, respectively. The Coral USB Accelerator costs $59.99 and can be added to the board to improve its ML capabilities.
Raspberry Pi supports TensorFlow and TensorFlow Lite. It can deploy various ML applications such as image classification, speech recognition, object detection and localization, facial recognition, and gesture recognition. The board is also capable of implementing various deep learning networks.
You can add the Coral USB accelerator to improve ML capabilities, but the cost increases. Compared to other development boards, Raspberry Pi has a weaker GPU. However, the RPi is a single-board computer that almost all manufacturers have.
Raspberry Pi 4B is an excellent board for learning and experimenting with ML and deep learning networks.
The Rock Pi 4
Rocha Pi 4 is often considered an upgrade to the Raspberry Pi 4B and is ideal for more complex projects. Launched in 2021, Rock Pi 4 has a better processor and GPU for implementing ML tasks. The board officially runs the Android operating system (Android 12) and supports conventional AI stack with GPU acceleration.
The Dual Cortex A72 at 1.8 GHz with Quad-core Cortex A53 at 1.4 GHz is at the center of the board. The integrated Mali T860MP4 GPU supports up to OpenGL v3.2 and OpenCL, DX11 and Vulkan 1.0.
The Rock Pi has a 64-bit dual-channel LPDDR4 RAM with options for 1/2/4 GB variants. The board has several memory options, including MicroSD, eMMC, PCIe and USB3. A MicroSD card of up to 128 GB can be used. The eMMC module is optional, with 8/16/32/64/128 GB of storage. The M.2 connector supports up to 2T M.2 NVME SSD.
The cost of the Rock 4A/4B starts at $49 for the 1GB RAM variant. The 2GB and 4GB RAM models cost $59 and $75, respectively. There are also other Rock 4 models available, including Rock 4A/4B Plus, Rock 4 SE and Rock 4C Plus. All models come with built-in WiFi5 and Bluetooth 5.
The Rock 4A/4B Plus is superior to the Rock 4A/4B in many specifications. The 4A models do not have built-in WiFi and Bluetooth. The board's other notable features include a 40-pin GPIO connector, integrated RTC, four USB 3.0 ports, a MIPI CSI connector, a MIPI DSI connector, and a full-sized HDMI 2.0 port. A dedicated hardware NPU accelerator will soon be added to the board.
Rock 4 is a better alternative for TensorFlow and TensorFlow Lite-based AI applications, but it is a newer option with only a little community support at the moment. Many accessories are still incompatible with the Rock 4.
The Sipeed MAIX GO
Sipeed MAIX GO is a good budget option for beginners interested in AI and ML. The board includes a 2.8-inch LCD and an OV2640 camera with an M12 lens. Based on the 64-bit dual-core RISC-V CPU clocked at 400 MHz, the MAIX GO features a 64 KPU (Neural Network Processor) with a width of 576 bits and an APU (Audio Processing Unit). It can support up to eight microphones with a sampling rate of 192 KHz.
The integrated RAM is 8 MB high-speed SRAM that can be overclocked up to 800 MHz. The board also features JTAG and UART and a lithium battery management chip.
Sipeed MAIX GO has a MicroPython port, supporting TensorFlow, Tiny-Yolo, FreeRTOS, Mobile-Net V1 and other frameworks. MAIX GO is an excellent development board for experimenting with basic AI stacks without purchasing additional components and accessories. Users can experience image classification, speech processing, and intelligent sensor data analysis with the board.
One disadvantage of MAIX GO is the lack of proper documentation and library support compared to other development boards. But at just $40, including a built-in camera and LCD, the MAIX GO is still worth considering.
Final thoughts
In previous years, the Intel Neural Stick 2 and BeagleBone AI would be on this list, but they have been discontinued. Choosing a board depends on your needs.
Beginners who already have a Raspberry Pi 4B can consider the AI stack based on the TensorFlow and TensorFlow Lite frameworks. But if you're looking for superior ML performance and features, the Rock 4 is an alternative to the RPi. For a tight budget, MAIX GO is an affordable option. Coral Dev is the fastest development solution.
For versatility around ML, AI, and deep learning, the NVIDIA Jetson Nano is ideal. For high throughput, try TensorFlow AI inference edge appliances. Or consider more than one frame to add to your experience.
