We want to hear from you! Take our quick AI survey and share your insights on the current state of AI, how you’re implementing it, and what you expect to see in the future. Learn More
AI chip-maker SambaNova Systems has announced a significant achievement with its Samba-CoE v0.2 Large Language Model (LLM).
This model, operating at an impressive 330 tokens per second, outperforms several notable models from competitors such as the brand new DBRX from Databricks released just yesterday, MistralAI’s Mixtral-8x7B, and Grok-1 by Elon Musk’s xAI, among others.
What makes this achievement particularly notable is the efficiency of the model—it achieves these speeds without compromising on precision, and it requires only 8 sockets to operate instead of alternatives requiring 576 sockets and operating at lower bit rates.
Indeed, in our tests of the LLM, it produced responses to our inputs blindingly quickly, clocking in at 330.42 tokens in one second for a 425-word answer about the Milky Way galaxy.
Countdown to VB Transform 2024
Join enterprise leaders in San Francisco from July 9 to 11 for our flagship AI event. Connect with peers, explore the opportunities and challenges of Generative AI, and learn how to integrate AI applications into your industry. Register Now
A question about quantum computing yielded a similarly robust and fast response at a whopping 332.56 tokens delivered in one second.
Efficiency advancements
SambaNova’s emphasis on using a smaller number of sockets while maintaining high bit rates suggests a significant advancement in computing efficiency and model performance.
It is also teasing the upcoming release of Samba-CoE v0.3 in partnership with LeptonAI, indicating ongoing progress and innovation.
Additionally, SambaNova Systems highlights that the foundation of these advancements is built on open-source models from Samba-1 and the Sambaverse, employing a unique approach to ensembling and model merging.
This methodology not only underpins the current version but also suggests a scalable and innovative approach to future developments.
The comparison with other models like GoogleAI’s Gemma-7B, MistralAI’s Mixtral-8x7B, Meta’s llama2-70B, Alibaba Group’s Qwen-72B, TIIuae’s Falcon-180B, and BigScience’s BLOOM-176B, showcases Samba-CoE v0.2’s competitive edge in the field.
This announcement is likely to stir interest in the AI and machine learning communities, prompting discussions around efficiency, performance, and the future of AI model development.
Background on SambaNova
SambaNova Systems was founded in Palo Alto, California in 2017 by three co-founders: Kunle Olukotun, Rodrigo Liang, and Christopher Ré.
Initially focusing on the creation of custom AI hardware chips, SambaNova’s ambition quickly expanded, encompassing a broader suite of offerings including machine learning services and a comprehensive enterprise AI training, development and deployment platform known as the SambaNova Suite in early 2023, and earlier this year, a 1-trillion-parameter AI model, Samba-1, made from 50 smaller models in a “Composition of Experts.”
This evolution from a hardware-centric start-up to a full-service AI innovator reflects the founders’ commitment to enabling scalable, accessible AI technologies.
As SambaNova carves its niche within AI, it also positions itself as a formidable contender to established giants like Nvidia, raising a $676 million Series D at a valuation of over $5 billion in 2021.
Today, the company competes with other dedicated AI chip startups such as Groq in addition to stalwarts like Nvidia.