Raspberry Pi unveils Hailo-powered AI Kit to make the model 5 smarter

Raspberry Pi has created a machine-learning addition for its single board computer that features the Hailo-8L AI accelerator.

This AI Kit is an M.2 Hat board with a preinstalled M.2 stick that carries the Hailo-8L accelerator, marking the first time Raspberry Pi has ever sold a first-party accessory for AI. The Hailo-8L is a cut down version of the Hailo-8 chip announced earlier this year, and offers 13 TOPS of performance at INT8 rather than the 26 TOPS of the full-fat version.

For comparison, Qualcomm, AMD, and Intel have recently announced SoCs packing NPUs at 45, 50, and 48 TOPS respectively.

So while the Hailo-8L is therefore not Earth's mightiest accelerator it is – like all Raspberry Pi products – keenly priced at $70, plus ~$80 for its Raspberry Pi 5 host. The combo of an Arm-compatible Pi and the accelerator will therefore be well below the cost of just an NPU-equipped SoC.

Plus, the Hailo-8L has remarkably low power consumption, Hailo's CTO Avi Baum told The Register. The chip consumes about a watt of power for every three TOPS, capping out at roughly five watts, and usually uses just a watt or two for a "typical" workload – such as processing 60 FPS video in real time.

Relying on a separate accelerator rather than an integrated NPU is an intentional design choice, Raspberry Pi CEO Eben Upton told The Register. The computer designer already started to focus on a disaggregated architecture with the Raspberry Pi 5, which has the CPU and GPU cores on a 16nm chip while I/O functions have been moved to a cheaper 40nm chip. An NPU integrated into the 16nm silicon slab would have been expensive.

Upton observed that including the Hailo-8L or some other accelerator on the Raspberry Pi 5s main board wouldn't have been ideal, as it would have made it more expensive. By offering the AI Kit, users can still buy the Raspberry Pi 5 for a lower price, and add the Hailo-8L if they are keen on exploring machine learning. The modular approach also meant rapid development: Upton revealed the AI add-on moved from concept to final product in just six months.

Out of the cloud

Although the AI Kit represents a big performance leap for the Raspberry Pi 5, the single-board machines already have history in the field.

Upton pointed out that long before AI became the Next Big Thing, Raspberry Pi users were already using the technology for things like object detection. The CEO mentioned one particular instance of a Japanese cucumber farm using a Raspberry Pi 3 to sort thorny cucumbers with about a 70 percent accuracy rate.

However, that cucumber sorter relied on AI processing in the cloud. The debut of this kit means some AI workloads can run on an Pi – reducing reliance on connectivity.

"It's a good host for smaller models, and more optimized and quantized large language models (LLMs) are running at tokens per second on the Raspberry Pi 5," Upton told The Register, before noting that some work will still need to be run in the cloud. LLaMa 3 70B is tough to run locally on a PC unless you have an impressive setup.

While there aren't a ton of use cases right now, Upton expects the lower entry price of Raspberry Pi 5 AI to create an "exponential increase" in AI applications and solutions.

When asked whether the kit represented a long-term partnership between Raspberry Pi and Hailo and if anything other AI hardware was in the works, Upton replied that he would be "disappointed if this is the last Raspberry Pi-Hailo collaboration." ®

PS: Raspberry Pi is targeting 260 to 280 pence per share when it debuts on the stock market this month, making it worth up to £540 million ($690 million).