D2D: LLM and some AI Pragmatism
Huawei's new magical chip, a search for US semis policy and more on Arm's IPO
Just a reminder that we operate our newsletters on a paid model. Paid subscribers will get three newsletters a month including our China Deep Tech notes and audio recordings of our posts. Paid subscribers will also get early access to the newsletter as well other benefits coming soon. Please subscribe and support our work.
Highlights from our Blog
The biggest news in semis right now is the mystery surrounding Huawei’s new Mate 60 phone and the quasi-mystical Kirin 9000 chip that powers it. The chip has been torn down and benchmarked but many questions remain. Cut off from TSMC by the US Government, Huawei seems to have gotten SMIC to produce the chip using its 7nm process, which is apparently good enough for the Kirin 9000 to deliver competitive performance. Huawei once had the best design team in mobile, so this performance is not surprising. That being said, this is just one chip, and it is doubtful they can maintain the competitiveness of their roadmap as all their competitors can still tap into TSMC via Qualcomm and Mediatek. It is also now looks very likely that the Chinese government heavily subsidized SMIC’s operations, as they are pushing their DUV systems to the utmost, and likely incurring meaningful losses on low yields for the chip. So for the moment, this chip looks likely to be a one-time achievement.
And just as everyone is questioning the effectiveness of US sanctions on China and Huawei, we ask the question what are the goals of US Semis Policy? Do we want to gain access to onshore advanced semis manufacturing processes? That seems solved for the moment in the deserts of Arizona. Do we want to catch up with trailing edge capacity? Texas Instruments and ADI seem to be doing that fairly well already, without any government help. We think the focus should be on what comes next.
Arm released more materials for their upcoming IPO as they embarked on the road show for that deal. These paint a better picture of how the company plans to grow, and emphasized the sizable installed base of software developed on top of their chips. These were encouraging, but leave open some big questions that lead us to struggle to justify the newly-revealed target valuation of $52 billion. Questions like how do they increase their share of industry economics, and what is going to happen with Arm China which their CFO touched on with a positive spin (why do CFOs always get tasked with delicately touching on difficult topics?). Arm is an important company for the industry, but the IPO is likely to be challenging.
If you like this content, you should check out our podcast The Circuit
Noteworthy Items
Semis and Hardware
Why are GPUs well-suited to deep learning? This is the best answer to this question we have seen yet. The author provides a very good analogy comparing GPUs and CPUs to moving truck versus Ferrari, that is the most useful mental model we have found for describing the difference between the two. He makes the point that most people (ourselves included) usually say GPUs are better because they are highly parallelized, but a better description is to say that GPUs have better memory bandwidth, while CPUs are optimized for latency. An important distinction as memory is clearly the biggest bottleneck to LLM deployments right now.
A look at the trade-offs between using specialized and general purpose chips. Mathematically, this is a fairly straightforward analysis, but the slowdown of Moore’s Law and a rapidly changing software landscape (aka AI) have radically altered the equation.
An academic paper reviewing Samsung’s Exynos line of mobile applications processors. This is pretty dense, and does not answer questions about its future, but worth reviewing for insight how Samsung has struggled to keep pace in this corner of the market.
DigiTimes ask the reasonable question - why is TSMC struggling to build a fab in the US, while Samsung has been doing just that fairly well for years. Working in Samsung’s favor was that they acquired their way into the US, while TSMC is building from scratch.
A handy guide to all the sub-sectors within semiconductors.
The Next Platform thinks that driving hardware performance gains is going to get much harder. If only there was someone investing to address this problem…
Networking and Wireless
Another report confirming what is now glaringly obvious, the 5G build out is largely complete, at least for this phase, and the big telecom operators are all shifting their spend elsewhere.
LoRa is an unsung workhorse of wireless standards. A good solution for low data-rate devices. And now we have a new world record for a LoRa signal - 830 miles across the Atlantic Ocean.
Software and the Cloud
The math behind LLMs. A very deep dive into how the latest AI systems work. This gets very deep in the weeds, but is important for anyone trying to work through how all this works.
Measuring software developer productivity is one of the most important and most challenging management problems in the world today. It is hard to do well, and many aspects of coding resemble artisanship more than manufacturing, despite years of companies working towards that end. McKinsey recently published a report on the subject, not surprisingly, many developers did not agree with their approach.
Ivan Pepeinjak is a serious networking engineer. We have followed him for years for his highly pragmatic take on networking topics, cutting through all the hype (remember software defined networks?). He has been reading up on AI and LLMs and has a solidly realistic take on generative AI, with a link from systems professional Bruce Davie’s detailed thoughts on the subject. As Davie’s says and Pepinjak highlights “Having a model for language is different from having a model of the world.” LLMs are important but need to be kept in perspective.
Our favorite gaming VCs have a good piece looking at the brewing battle for portable gaming consoles. A new wave of handheld gaming devices is coming to market in the next two years, and Konvoy compares the situation to the console wars of past decades. We plan to revisit this topic soon.
Diversions
A map of the world’s river basins and watersheds .