D2D: AI Factories Are Weird
Nvidia's true core competency, GTC and more
In this issue we take a look at Nvidia’s GTC conference, their claims about AI Factories and recognize their real strength is the company’s ability to take risks even when those result in failure.
Just a reminder that we operate our newsletters on a paid model. Paid subscribers will get three newsletters a month including our China Deep Tech notes. Paid subscribers will also get early access to the newsletter as well other benefits coming soon. Please subscribe and support our work.
Highlights from our Blog
Nvidia remains the center of attention this month.
In anticipation of Nvidia’s GTC developer conference, we looked back at their history to see if we could decode what has made them so successful. People tend to attribute their success to a single-minded focus on investing in AI for almost a decade. We found the opposite is true. Over the years, they have taken big risks, many of which ended in failure. Their success stems from that willingness to take chances, something few other companies can manage.
For us, the highlights of GTC included the high degree of strategic flexibility. Nvidia has built the company around providing complete solutions - from graphics cards 30 years ago to giant DGX AI systems today, but are apparently willing to sell just individual components as well. Their software moat remains formidable. If you are looking for cracks in their armor, they admitted that their advantage in AI Inference rests on their NV Link networking layer, which is not a major weakness, but at least opens the door a bit for the competition.
One thing that did not sit right with us was all of Nvidia’s talk about “AI Factories” - GPU-centric data centers, independent of the Internet giants. The industry has tried this model before (aka PaaS) and it could not withstand the inertia of software ecosystems and the hyperscalers’ capital stores. Ultimately, these AI Factories base their value on their ability to acquire GPUs at a time when those are scarce, but semis are cyclical...
With all that said about Nvidia’s strengths, what does that mean for the broader data century industry? We think Nvidia’s leadership in cloud semis is here to stay. However, within that umbrella there are multiple ways for this to play out. Nvidia could just be the first among many as a silicon provider for the Cloud, it could be the dominant provider, and if you want to dream big, it could even reshape the economics of the hyperscalers.
We have been watching the emergence of the category of handheld gaming devices - small enough to be very portable, but powered by PC-grade silicon. Of course, the missing piece for that category is software. So we were intrigued to learn of Playtron which has a lightweight OS that focuses solely on running games. This is a tough business to crack into, but is definitely something the industry needs.
If you like this content, you should check out our podcast The Circuit
Semis, Hardware and Deep Tech
Our friends at Expedera landed an investment from autos semis maker Indie. First and foremost, check out Expedera, they are doing really interesting work providing IP for NPU cores. But also interesting to think of what Indie needs AI for, and the expansion of machine learning into corners of compute that we do not typically think of as needing AI. Indie is building an assisted driving (ADAS) stack, but they are attacking the problem from below. Instead of designing big processors to fit in cars, Indie is working their way up from much more prosaic parts. Interesting to think how AI could play out in low level systems
The new Nvidia Blackwell GPUs are power hungry. This is going to be a major topic in coming years. Data centers are already power constrained in most geographies, and the rise of machine learning is going to exacerbate the problem.
Some strong developments from Qualcomm on the software front. As they prepare to launch their PC CPUs into the market in coming months, they have been demonstrating their work in building up the software ecosystem for Snapdragon. This includes native support for games like Baldur’s Gate 3 among other titles, as well as native support for the Chrome browser. Getting big names like this on board goes a long way.
TSMC’s chairman and CSO penned an article in IEEE Spectrum tying the rise of AI to advances in semiconductor manufacturing. Not a surprising take for these two, but interesting to see their view on ways in which transistor density (aka Moore’s Law) can continue into the 2030’s.
Silicon Photonics are slowly, slowly coming to a server near you. This is an important trend, but still requires a lot of development.
Networking and Wireless
Short sale research firm Hindenburg is out with a report calling out Equinix’s accounting practices. Hindenburg has a good, but not perfect, track record, and we cannot vouch for their findings. That being said, we have long had a sense that things at Equinix are not quite right. For such an important part of digital infrastructure, their position in the market has always struck as more financial engineering that technical excellence. We agree with Hindenburg that AI is not really the major growth driver management presents it as for the company.
Software and the Cloud
Amazon and Microsoft are both in the news with some massive capex plans. For instance, Amazon says they will spend $150 billion over the next 15 years to expand their data center AI footprint. And Microsoft is reportedly planning to spend $100 billion with OpenAI for a ‘supercomputer’. These are big numbers, even for these companies.
There is a shortage of COBOL programmers needed to maintain all the critical legacy COBOL systems out there. Generative AI can sort-of write code. Can it solve the shortage? There is hope, but we still have a ways to go.
Google says smartphones need more memory to run AI models. There is a lot going on in this item. First, it is a good time to be memory supplier. Second, the dynamics around AI inference are confusing, with vendors looking to promote AI features on phones, but no one is quite clear what that means and so cannot say what is required. Google itself went back and forth over which of its Pixel phones would get its new Gemini Nano AI model. Which leads to the third element of this story - the recurring theme of dysfunction at Google. Did the Gemini team not communicate with the Pixel team until after the Gemini announcement? And of all companies, Google should be leading the charge for Inference at the Edge. They would really like to have phones handle basic queries, it will save Google billions in capex, but there does not seem to be any of the strategic cohesion required to pull all the parts together.
Diversions
Can we get kids off smartphones? A topic we think will resonate with many, but sadly, no easy answers.
Image by Microsoft Co-Pilot
Thank you for reading D2D Newsletter. This post is public so feel free to share it.