SambaNova, the AI chip startup headquartered in Palo Alto, has introduced a groundbreaking semiconductor, the SN40L chip. This innovation promises to revolutionize enterprise solutions by enabling high-performance computing applications, specifically the SambaNova Suite LLM platform, to train models faster while significantly reducing overall costs.
The SN40L chip boasts the capability to support a model with a staggering 5 trillion parameters and a sequence length exceeding 256k on a single system node.
It emerges as a formidable alternative to NVIDIA, a leader in the chip industry, which recently unveiled its formidable GH200 chip, set to become one of the most powerful options on the market, featuring 480 GB of CPU RAM and 96 GB of GPU RAM.
A ‘Game Changer’ for Enterprise AI
This chip’s potential finds its true expression in powering the SambaNova Suite, an enterprise application designed to aid both private and government organizations in the development and deployment of generative AI applications. The synergy between the SambaNova Suite and the SN40L chip is described as a ‘game changer’ by the company.
Rodrigo Liang, the co-founder and CEO of SambaNova, emphasizes this technological milestone: “We’re now able to offer these two capabilities within one chip – the ability to address more memory, with the smartest compute core – enabling organizations to capitalize on the promise of pervasive AI, with their own LLMs to rival GPT-4 and beyond.”
Addressing the Chip Shortage
Taiwan Semiconductor Manufacturing Company, which is also a supplier of NVIDIA’s AI chips, will take on the manufacturing of SambaNova’s chips.
The current semiconductor shortage has left many businesses waiting in anticipation for NVIDIA’s coveted chips, vital for running and training massive AI applications and models. To address this void, companies like SambaNova, AMD, and Intel have stepped up.
Kunle Olukotun, co-founder of SambaNova Systems, shares insights into industry trends: “We’ve started to see a trend towards smaller models, but bigger is still better, and bigger models will start to become more modular.”
Raising Over $1 Billion in Venture Funding
Unlike NVIDIA, SambaNova has a distinctive approach to chip distribution. Rather than selling its chips as standalone products, it incorporates them into its custom-built tech stack, which also features the SambaNova Suite.
This approach poses additional challenges, particularly with NVIDIA entering the full-stack AI-as-a-service market with its DGX Cloud offering, as pointed out by Dylan Patel, chief analyst at the consultancy SemiAnalysis: “The chip is a significant step forward. I don’t believe the chip will change the landscape.”
Olukotun concludes by highlighting the SN40L’s significance for customers: “Customers are requesting an LLM with the power of a trillion-parameter model like GPT-4, but they also want the benefits of owning a model fine-tuned on their data.
With the new SN40L, our most advanced AI chip to date, integrated into a full stack LLM platform, we’re giving customers the key to running the largest LLMs with higher performance for training and inference, without sacrificing model accuracy.”