As Nvidia maintains its commanding 90% market share in AI computing with a $3 trillion valuation, a new wave of chip startups is positioning itself to capture a slice of the rapidly evolving AI hardware market in 2025. The competitive landscape is shifting as AI workloads transition from training to inference computing, creating opportunities for specialized challengers.
Thomas Sohmers, CEO of chip startup Positron AI, notes that in 2024, the majority of AI compute spending shifted to inference—the computation needed to produce responses to user queries—and this trend is expected to continue on an exponential growth curve. This shift is driven by the emergence of reasoning models like OpenAI’s o1 and o3, and Google’s Gemini 2.0 Flash Thinking, which use more compute-intensive strategies during inference to improve results.
Three major Nvidia challengers—Groq, SambaNova Systems, and Positron AI—shared their strategies and expectations for 2025 with Business Insider. Mark Heaps, chief technology evangelist at Groq, emphasized execution as the key priority, revealing that the entire company has forgone holiday breaks to build systems and meet overwhelming customer demand. He described their sales funnel as “carbonated and bubbling over” with unprecedented customer interest.
Rodrigo Liang, CEO of SambaNova Systems, is focusing on the critical shift from training to inference workloads, positioning his company’s technology to help enterprises scale efficiently and sustainably. SambaNova claims its architecture consumes 10 times less power than competitors, addressing both economic and environmental concerns around AI deployment.
The startups face significant challenges, particularly customer hesitation to move away from established vendors. Heaps acknowledged the “no one ever got fired for buying from the incumbent” mentality but believes customers are recognizing the difficulty of obtaining Nvidia chips and the superior performance alternatives can offer.
These companies are betting that inference-time computing and chain-of-thought reasoning strategies will benefit all inference chip providers, not just Nvidia. As AI models increasingly “think more” before answering to improve response quality, the associated time and monetary costs create opportunities for hardware optimized specifically for these workloads.
Key Quotes
In 2024, the majority of AI compute spend shifted to inference. This will continue to grow on what looks like an exponential curve.
Thomas Sohmers, CEO of Positron AI, explains the fundamental market shift that’s creating opportunities for Nvidia challengers. This transition from training-focused to inference-focused computing represents a structural change in AI workload patterns.
Right now, everybody at Groq has decided not to take a holiday break this year. Everyone is executing and building the systems. I tell everyone our funnel right now is carbonated and bubbling over. It’s unbelievable, the amount of customer interest.
Mark Heaps, Groq’s chief technology evangelist, reveals the intense demand his company is experiencing and their all-hands-on-deck approach to capturing market opportunity. This demonstrates the urgency startups feel to capitalize on the current market dynamics.
Our architecture consumes 10 times less power, making it possible for enterprises to deploy AI systems that meet their goals without blowing past their power budgets or carbon targets.
Rodrigo Liang, CEO of SambaNova Systems, highlights a critical differentiator as power consumption becomes a major constraint for AI deployment. This efficiency advantage could be decisive as data centers face energy limitations.
No one ever got fired for buying from — insert incumbent. But we know that it’s starting to boil up. People are realizing it’s hard for them to get chips from the incumbent, and it’s also not as performant as Groq is.
Mark Heaps acknowledges the biggest challenge facing Nvidia alternatives—customer risk aversion—while noting that supply constraints and performance advantages are beginning to shift purchasing decisions.
Our Take
The timing of this competitive push is strategic and potentially transformative. Nvidia’s dominance has been built largely on training workloads, where CUDA software ecosystem and raw GPU power created insurmountable advantages. However, inference represents a different game with different optimization targets—latency, throughput, power efficiency, and cost per token.
What’s particularly notable is the convergence of market forces: reasoning models dramatically increasing inference compute requirements, Nvidia supply constraints creating frustrated customers, and growing power limitations forcing efficiency considerations. These startups aren’t just offering “me too” products; they’re architecting solutions for a fundamentally different workload profile.
The 10x power efficiency claim from SambaNova, if validated at scale, could be a game-changer as data centers hit power walls. Similarly, Groq’s focus on execution speed addresses the latency sensitivity of real-time AI applications. The question isn’t whether these companies can build competitive chips—it’s whether they can overcome Nvidia’s ecosystem advantages and customer inertia before their window closes.
Why This Matters
This development represents a pivotal moment in the AI hardware industry, where Nvidia’s near-monopoly may finally face meaningful competition. The shift from training to inference workloads is fundamentally reshaping AI computing requirements, potentially opening the market to specialized solutions that can optimize for speed, cost, or power efficiency.
The emergence of reasoning models that require significantly more computational resources during inference creates both challenges and opportunities. While this increases overall compute demand—benefiting all chip makers—it also rewards companies that can deliver superior performance or efficiency for these specific workloads.
For enterprises deploying AI, this competition could mean more choices, better pricing, and specialized solutions tailored to their needs. The power efficiency claims from companies like SambaNova are particularly significant as data centers struggle with energy constraints and sustainability goals.
The broader implication is that as AI applications mature and diversify, the hardware market will likely fragment into specialized niches rather than remaining dominated by a single player. This could accelerate AI adoption by making deployment more accessible and sustainable for organizations of all sizes.
Recommended Reading
For those interested in learning more about artificial intelligence, machine learning, and effective AI communication, here are some excellent resources:
Recommended Reading
Related Stories
- Jensen Huang: TSMC Helped Fix Design Flaw with Nvidia’s Blackwell AI Chip
- Sam Altman’s Bold AI Predictions: AGI, Jobs, and the Future by 2025
- OpenAI’s Valuation Soars as AI Race Heats Up
- EnCharge AI Secures $100M Series B to Revolutionize Energy-Efficient AI Chips
- Pitch Deck: TensorWave raises $10M to build safer AI compute chips for Nvidia and AMD
Source: https://www.businessinsider.com/chip-startups-nvidia-market-share-2025-2024-12