D2D: The AI Party Has Been Postponed CES 2024

Edge vs. Cloud Inference, XR Resurfaces, Auto Semis and much more

CES is a big show (we may have mentioned this in the past (and here and here). There is so much going on that it is impossible to pick out trends consistent across the whole event. When we hear people try to sum up the show what we hear is just a reflection of their views going into the show. We were reminded of this last week when we met three people who had never been to CES (or even Las Vegas) before. Given that we are closing in on 20 shows, our big theme from the show is that we are getting too old for this.

More seriously, we will not even attempt to pick out a theme for the show and instead just jump into our most noteworthy findings. The threads we picked up include:

• Not as much AI as we expected, and that absence is meaningful in some counter-intuitive ways

• The battle between conducting AI Inference on the Edge versus in the Cloud has entered early skirmishes

• XR is making some meaningful advances

• There are some big changes in how the auto industry is positioning itself for consumers 

• X Tech - Food Tech, Age Tech, Golf Tech and the rest of the weird and wonderful

The Big AI Party Is Not Cancelled, Just Postponed

We went into CES expecting to be completely inundated with “AI”, all marketing guns blazing. What we got was much more nuanced. 

True, there was an endless number of “AI powered” devices that have nothing to do with AI. Al backpacks, AI office chairs and AI toothbrushes to name just three of many. Those are just gadgets (barely) that have a tiny bit of software in them. A decade ago we started calling them “smart”, but apparently the memo went around and we are now supposed to call them AI.  But that is just part of the normal silliness of CES marketing and every marketer’s impossible dream of standing out out from the cacophony,

Much more important was the lack of AI commentary from the big vendors. We fully expected lines of PCs and smartphones enabled with transformer-based AI accelerators, but we got none of that. The big, usual suspects had oddly little to say on the subject. 

We think there are two causes of this. First, Intel, AMD and Qualcomm have held recent AI-heavy events of their own. The way that tech marketing works now, there is far less return in launching a product at CES, better to host your own event than try to compete with everyone else.

Second, and more telling, no one is ready for any of that AI enablement. Chat GPT is barely a year old, so there is very little silicon tuned for that type of model available yet. Qualcomm is probably the furthest along, but Intel and AMD are still a cycle or so away from having chips on the market that can make any meaningful dent with large language models for consumer devices. Moreover, even if they all had their chips ready, what would anyone do with it? The software side of consumer AI is just not there yet. We can argue whether it ever will be there 

Cloud and Enterprise AI inference is a different story, but the consumer side needs Microsoft, Google and most especially Apple to show us something interesting. 

There were a few interesting ideas out there. For instance, Rabbit launched the Rabbit R1 a voice assistant. However, much like the Humane pin from a few months back, we felt this was much more of a concept than a product we would want to own. These devices are really just natural language processors, it is novel to be able to have a conversation with an electronic gadget, but true utility is not clear. Our sense is that these devices have not really thought through how a big a change these devices require for human interaction. Not all things are made better by speaking to them, sometimes having a screen to look at and a keyboard to type is better. There may be a world someday in which we speak to our computers like we are in Star Trek, but for right now a web page or app is still better for most uses.

Begun the Inference Wars Have

As much as the tech world is excited by “AI”, for makers of semiconductors the excitement is much more tempered. Everyone recognizes that there is a big opportunity out there, but there is considerable trepidation as to who will actually share in those benefits. One year into the large language model (LLM) revolution and the biggest beneficiary is Nvidia, by far. So far that there are many who worry that Nvidia will come to dominate not only AI but all of compute. That is the extreme view, but there are many reasons to think that Nvidia will not leave much room for everyone else. 

Nvidia now captures 70%+ share of data center compute spend. They are also set to dominate the market for AI training semis for the foreseeable future. Hence the shift in most semis messaging to talk about the market for AI Inference. This is going to be a much larger market than training, but how exactly that happens and who will participate remains unclear. 

First and foremost, there is the very big question as to where inference will take place - in the Cloud in some data center or at the Edge on user devices. For some time, we have operated on the assumption that the only way the economics of AI inference pencil out is for much of the workload to end up being done at the edge, meaning the end user pays the ‘capex’. Inference is going to be very expensive and even the deep pockets of the hyperscalers will struggle to afford the necessary build out. Beyond that, there are reasons around security, privacy and “data sovereignty” which lend themselves to edge inference. 

At CES, we had that view seriously challenged several times. For instance, we met with privately-held Groq. They are a new-model company that designed their own inference chip (the LPU), but monetize through software providing low-cost, high-speed inference via their PaaS platform. Their contention is that their approach is sufficiently superior to the alternatives that it bends the cost curve enough to make cloud inference economical and edge inference superfluous. Of course, we have many questions about Groq - how good is there performance really and can they compete with the hyperscalers when it comes to providing cloud services. That being said, they have a company and an approach that merits more attention.

Separately, we spoke to several companies making large moving objects who felt fairly strongly that cloud inference worked for most of their needs. For instance, we spent a lot of time in the John Deere booth. Deere is probably the most advanced industrial company when it comes to compute and autonomy. We spoke with someone from their software team about the edge vs cloud debate. According to him, much of the Deere software stack can be run perfectly well in the Cloud. Some tasks like vehicle safety (i.e. do not run over farm workers) probably need to be run on the vehicle, but many other tasks, including navigation, run perfectly well for them from the cloud. (They also run a robust cellular service to provide that low latency connectivity.) It was clear from our conversation with him that as a software designer he felt much larger compute constraints in the cloud than at the edge, implying much smaller uplift for edge silicon. 

Admittedly, these are anecdotal data points from the far end of the capabilities spectrum. Most other companies will have to contend with very different environments. Nonetheless, they give us pause in to reconsider our assumptions about how this will play out.

Ultimately, this debate will be settled on the software side. Can developers continue to bring out models and tools which run on the edge? The answer to this question is important, and the stakes are huge. Companies like AMD, Intel and Qualcomm are all counting on a big uplift for edge inference.

XR is Still a Thing

Years ago CES was consumed by virtual reality (VR) gear. For a couple shows, the companies making goggles and related gear seemed to be everywhere. And people spoke hopefully about solving some of the big technical hurdles for Augmented Reality (AR). All of which raised expectations that we were on the cusp of the Mixed Reality (XR) world.

Then winter came. Sales of the first VR goggles were uninspiring. The technical problems around AR persisted. Everyone moved on to other hot, new things, and XR largely disappeared from CES.

This year that changed. Obviously, the pending launch of Apple’s Vision Pro is a major catalyst in rekindling interest in the space, but there have been other developments,. 

Notably, the software ecosystem has advanced. We are starting to see companies displaying new game titles specifically created with VR in mind. There are also new accessory and component companies coming to the fore. We saw several companies displaying some combination of XR hardware and software again in several areas on the show floor. We did not meet Wearable Devices personally, but heard some intriguing stories about their wristbands which can do neural monitoring and capture intent, which could be an important I/O tool for XR. And there was Ixana whom we discussed last year which was demoing their audio and video solutions which dramatically lower power requirements for XR peripherals.

The category is not out of the woods yet. A lot depends on Apple and the Vision Pro. The price tag, buying model, user interface and availability of the Vision Pro remain important problems. We also think the software ecosystem for the device will grow in fits and starts as developers balance catching the Vision Pro wave with the realities of a very small target audience for the foreseeable future. That being said, we see signs that the rest of the segment is moving forward. The Vision Pro is a catalyst for speeding things up, but does not have to be a break-out hit for the category to grow.

And then there is the question of AR. Connected glasses which can layer data over the real world hold the potential to become our next major device. The technical problems are still huge. But even here, there are signs of progress. The announcement which non-attendees asked us about the most, by far, were the transparent LED monitors from LG and Samsung. We saw these first hand and came away suitably impressed. Samsung demo’ed their screens providing a wall of glass that overlaid a live soccer match. LG’s demo was more of live decoration for their massive booth. But it is easy to see how these are going to be much more important for AR. Mark these down as steps in the right direction, but still a long way to go.

Finally, there is the critical question as to software for all of this. As noted, Apple is likely to be the driving force in building this ecosystem, but we did see some tantalizing ideas at the show.

For example, probably our favorite app was in the Sony Booth. They have developed a system with Hawkeye Innovations, Pulselive and Beyond Sports, that converts video images of sports matches into the digital realm. Viewers can then review the gameplay from any angle they choose. This runs the gamut from low-res diagrams of movement which can be used by coaches to full-blown digital avatars allowing users to insert their own images into the gameplay. 

We have no idea when this will get deployed into VR, but we can easily see it being a hit. 

Autos Make a Move

There were big moves in the Auto portion of CES this year, some literal. The Las Vegas Convention Center (LVCC) which hosts much of the show is in the process of revamping its spaces. Where once all the auto-related exhibitors clustered in the North Hall, this year the big auto OEMs and their Tier 1 suppliers moved to the newish West Hall, leaving the old North Hall to the Tier 2 suppliers and after-market parts dealers.

As much as this is a trivial logistical move, we can interpret it as part of something bigger. The auto makers have always been an odd fit for the Consumer Electronics Show, but they are waking up to the reality that they are now hosts of what is likely to be the next major consumer compute platform. They have done away with demos of speed and torque, and now are all solidly focusing on apps, AI and autonomy. It now makes more sense for these OEMs to have booths alongside Qualcomm and Mobileye rather than next to providers of tricked-out underbody lights and the latest shock absorbers.

Mobileye, Qualcomm, Daimler and Hyundai are the central aisle

Not surprisingly there was a lot of talk about autonomy this year. That side of autos is still inching forward but remains a hot topic for big shows. In fairness, some companies are making real progress here, just not the ones we usually think of.

True, “Level 5” autonomy for cars remains distant, but for vehicles with a more limited range of operations it is much closer. Specifically - construction. The makers of Bobcat mini-bakchoes and Hyundai both had demos of construction gear in their “auto” booths. That being said, both of these vehicles are not quite ready for production.

However, tech pioneer John Deere does have autonomous equipment in use. The entranceway to their booth had a live feed from an autonomous vehicle tilling a cotton field in Texas. We have written about Deere in the past and will likely do so more in the future.

And Everything Else

One of the best parts of CES is the unexpected - all the weird and wild devices that people have come up with. This makes it very hard to cover the show coherently, but also makes the event much more fun.

These devices run the gamut from small gadgets like the Clicks iPhone keyboard case to small research labs who have their own transparent TVs on display.

Of course, everyone tries to figure out some coherent way to position all these companies. The Sands Exhibition hall is full of this kind of X-Tech; where X can be any sub-category: heatlhtech; sleeptech; agtech; sportstech'; and many others.

One of the largest exhibitors on this floor was the AARP’s “Age-Tech” booth. We have no idea why the senior advocacy and lobbying group has a massive booth at CES, but we were equally surprised when mattress companies and the US Postal Service showed up in past years. By the same token, we were surprised when John Deere first showed up, and that turned out fairly well.

And then there is the just plain odd. Like this Korean company’s “AI” urine analysis systems for home toilets.

And let’s not forget “merch”. CES has its own gear now, we did not buy anything but have to admit an official CES ball cap has its appeal.