D2D: The Rapidly Changing Market for AI Semis

May 22, 2023

Become a Paid Member of the D2D Community

Just a reminder that we will soon be shifting our newsletters to a paid model. Paid subscribers will get three newsletters a month including our China Deep Tech notes. Paid subscribers will also get early access to the newsletter as well other benefits coming soon.

Highlights from Our Blog

The market for AI semis is changing very quickly. The headlines are all about GPUs. There are good reasons to use GPUs for the math behind neural networks and transformers. Training AI models rely heavily on Nvidia’s best, but the market is more than GPUs. There is big demand for AI accelerators (aka NPUs), and let’s not forgot that a lot of AI work will be done on CPUs. It is also important to remember that the market for inference semis is much bigger, and this is really two markets - inference in the cloud and inference at the edge. And those segments face their own set of dynamics.

That all being said, Nvidia remains the king. Largely on the back of its CUDA software stack, which remains such an important competitive barrier that maybe we should think about Nvidia as a software stock.

Global Foundries reported a disappointing set of quarterly results and launched some major personnel changes. They are still very well positioned to become a major player in the analog foundry segment, but the clock is ticking.

We have been spending a lot of time looking at analog chips. In particular, we have been on a quest to understand the intellectual property (IP) and software requirements for these products. There is likely no equivalent to Arm in the analog space, but there is a big opportunity to help analog companies add a bit of digital logic to their products.

Semis and Electronics

As much as it seems like everyone is designing their own chips, delivering on this consistently is challenging and expensive. Chinese handset maker Oppo is reportedly giving up its roll your own plans. Noteworthy, but recall that Oppo is part of the BBK Group, whose other members are still plowing ahead with internal designs, which offers an insurance policy should Oppo need to change course in the future.

There were a host of major processor announcements last week. Ampere launched a 192-core CPU. Meta unveiled their long-awaited Inference chip. And Qualcomm unveiled their AI at the edge plan. We will likely write up each of these in more depth soon. Worth pointing out that Qualcomm is often an afterthought in AI conversations, but they are just targeting a very different part of the market, and they seem to have a pretty well thought-out plan here. Not least, their fairly robust software solution which they call the Qualcomm AI Stack, but we are always going to think of as Quda.

Everyone is waiting for Apple’s AR announcement next month. The bill of materials (BOM) for the device allegedly leaked in China last week. There is absolutely no way to judge the accuracy of this list until the iFixit teardown hits. We will say this looks directionally about right, with a total BOM of $1,600 (including a battery pack?). Bring the popcorn, we are entering the silly season for what may, or may not, be the most important electronics announcement in many years. On a related note, Apple is reportedly going to manufacture its own micro LED screens.

The big system challenge for AI is not doing the math, it is moving all the data around - on-chip and between chips. The folks at Marvell have an interesting angle on this in the form of optical interconnects via Next Platform.

A new set of benchmarks for leading CPUs is out. Intel looks surprisingly good on this narrow set of tasks, but the real takeaway is that the CPU space is incredibly competitive.

On that note, India now reportedly has a highly competitive CPU for HPC/supercomputing. This space lends itself to highly specialized solutions, which are not readily commercialized for the broader market. Still it is worth noting that the every major semis company has a large office in India, but this is the first time that we are aware of a team in India launching a CPU. We expect there will be many more.

With all the talk of on-shoring semiconductor manufacturing back to the US, an important question will be is there a sufficient labor force to make that possible.

Intel announced a partnership with Arm. Intel wants to be in the foundry business, so they are going to need to support Arm-based chips. This announcement is not surprising, but there are a lot of devils in the details.

The team at Tiny Tape Out is gamifying semis development.

Software and the Cloud

The hardest executive challenge in technology today is managing the costs of software development. This is critical not only for software companies, but semis companies as well as all enterprises. There is no easy solution to the problem, but this is a good way to think about it.

Here is a primer on the math behind neural networks and transformers in particular, a major topic in AI today.

Wireless and Networking

Spanish data regulators have fined the GSMA for improper handling of the facial recognization data they collected for entry passes to Mobile World Congress. Posted here because we know a lot of readers likely have their faces stored in the GSMA’s system, ourselves included.

Last year, we noted that European mobile operator Vodafone was in the process of scaling up its software team. At the time, we noted this is hard for any company, especially a telco. Vodafone’s new CEO (and former CFO) apparently agrees and looks likely to scale back the efforts.

Ericsson is being accused of trying to install a Divide and Conquer policy in the inner workings of the Open RAN initiative. As we have noted, we have seen this movie many times before.

An interesting case study for Private Mobile Networks. John Deere built a sizable network for its manufacturing facility. Among US manufacturers, John Deere has built a reputation for being fairly technology forward in its thinking, and we think they are probably the first among many of their peers to go down the private network path.

Other Things We Found Interesting

A project to map the human immune system. Tracking things like this are one of things that keep us energized and excited about working in tech.