Graphic processor maker Nvidia joined the rarefied short list of companies valued at $1 trillion-plus earlier this year thanks in no small part to the explosion of interest in, and development of, generative AI and machine learning (ML) applications in Silicon Valley. After all, Nvidia’s GPUs are the preferred — and in some cases, necessary — hardware for training large foundational AI models like OpenAI’s hit GPTs, so much so that a shortage of Nvidia GPUs was the top gossip of the Valley recently.
Now, however, another company, SiMa.ai, is introducing a new product that it says will continue to help it beat Nvidia’s chip performance for ML in one important, fast-growing category: edge devices.
“We’re the first company to be better than Nvidia on both performance and power at the edge category,” said Krishna Rangasayee, founder and CEO of San Jose, California-based SiMa.ai, in an exclusive video call interview with VentureBeat, citing the two companies’ respective MLPerf benchmark scores.
The new product is called Palette Edgematic, and is a no-code, drag-and-drop software platform for deploying machine learning models quickly, reliably, by non-specialists on edge devices out in the field, using SiMa.ai’s existing Palette software on its own proprietary MLSoC silicon chips (manufactured to spec by leading supplier Taiwan Semiconductor, TMSC).
Palette Edgematic makes it “accessible for anybody without an ML background to be able to deploy very complicated systems,” Rangasayee.
The bleeding edge
Edge devices are the sensor arrays and monitoring computer systems put out in the field on heavy industrial equipment, oil and gas plants, solar panels and wind turbines, manufacturing plants, even military hardware like drones, to allow them to understand their environment and conditions, detect and alert people to maintenance needs before they cause downtime, and improve uptime, performance, and cost-savings.
By necessity of the physically challenging conditions in which they operate, computers on these devices have traditionally been too simple and hardware-limited to run the resource-heavy processes required for machine learning applications.
Yes, some of the processing can be shifted to the cloud, but not all of it — and in fact, for mission critical equipment and operations like those in energy and the military, on-prem or on-device processing power is key.
“The core problem that we observed [was that] we had seen AI and ML scale in the cloud, we had seen AI and ML scale well in the mobile platforms too, but everything that I call the embedded edge… that’s kind of really being left behind,” said Rangasayee.
In order to achieve highly performant ML operations out in the field on edge devices, Rangasayee and his colleagues at SiMa.ai had to do more than just design great software and hardware. They had to make it efficient as well, keeping in mind the low-power requirements of many of their potential customers and the environments they were working in.
“Our thesis has remained the same: you need the performance of the cloud with the power efficiency of the edge,” Rangasayee explained. “You need to really bring world class ease of use.”
Conquering a giant
While Rangasayee and his colleagues have nothing but respect for Nvidia and what it has achieved (Rangasayee called Nvidia’s CUDA software “phenomenal”), and believe the two companies can co-exist serving different market segments, there’s little denying that they want their potential customers to switch from Nvidia to SiMa.ai hardware and software.
“If you’re using an NVIDIA PCI Express card, you could swap that out,” Rangasayee noted. “You could plug it in, and you fundamentally are up and running. So the ability to switch over is pretty easy, and particularly the software being this easy to use. It really makes us a good viable alternative for everybody.”
In fact, Rangasayee believes that many people working with edge devices have simply defaulted to Nvidia chips and software because until recently, there were no commercially viable, affordable alternatives with comparable performance.
“Though Nvidia is not the right choice for the edge, they’ve been considered that so far because of the software strength that they have,” Rangasayee contended. “And they haven’t really had viable competition.”
Real-world use cases
At the end of the day, the founder and CEO thinks that the ease-of-use and performance achieved by SiMa.ai’s Palette Edgematic will win out for this specific use case — and already is, with some customers.
Rangasayee showed VentureBeat’s journalist and author of this article demoes of military drone footage captured and analyzed rapidly using an application created with Palette Edgematic, boosting the video captured from a paltry 3 frames-per-second up to 60 frames-per-second thanks to SiMa.ai’s hardware and software combo.
He also showed how a Palette Edgematic user could easily drag and drop — and tweak, if they so wanted — full ML code modules and applications onto their edge device from a trusted library of open source AI models.
Rangasayee also showed how autonomous vehicle developers could use Palette Edgematic to simply drag and drop the necessary ML code to create a data pipeline from the vehicle’s sensors to its onboard computers for rapid processing.
“You pick your ML models, you drag and drop, you construct your computer vision pipeline, and say ‘hey, here’s my perception engine. Here’s my SLAM engine. Here’s my semantic segmentation.’ And you click a button, that all runs on a device. And in minutes, you get the data back and you’re done. Now, this would normally take nine months.”
Rangasayee said that using Palette Edgematic, a customer could reduce the number of developers they had working on this kind of formerly intensive project from 75 people to about two, and achieve the same results, faster, using far fewer resources.
Little wonder one of his favorite taglines is “from months to minutes.”
Where SiMa.ai and Palette Edgematic go from here
SiMa.ai views its announcement of Palette Edgematic as merely the beginning of its mission to make deploying ML on the edge easy and reliable for nontechnical users.
“Every quarter we continue to add more ML models more computer vision pipeline libraries and keep making this more and more accessible and robust,” Rangasayee said. “That’s the journey ahead of us.”
He likened Palette Edgematic to the introduction of the Apple iPhone compared to using a BlackBerry or Nokia, which were far less intuitive, a fitting comparison on the same day as an Apple product event.
“Every high school student should be able to create complicated computer vision pipelines,” Rangasayee said. “We’re starting with computer vision. We have bigger ambitions for the company beyond that space.”