Rabbit AI kicks off LLM hardware
It's new, it's shiny, it's not a phone
The big news from this past week: Rabbit AI has kicked off the AI-hardware space with its AI-infused pocket gadget, the Rabbit R1. If you’re asking what an “AI-infused pocket gadget” actually is, you’re…not alone. I watched founder Jesse Lyu’s 25 minute keynote from last week’s CES 2024 event in Vegas (https://twitter.com/rabbit_hmi/status/1744781083831574824), and went from skeptical to impressed and back down to skeptical.
The basics: it’s a device, with a single button and a camera and some fancy AI. Here’s a picture.
Rabbit AI’s pitch is that “large language models” (LLMs, the actual workhorse underneath ChatGPT) are good at understanding things, but bad at actually doing things, and the Rabbit R1 is meant to bridge this gap (e.g., going from “I want to go on a vacation in Paris” to actually having flights, a hotel, and an itinerary). People have been really excited by the “AI agents” space where LLMs are actually allowed to do stuff (AutoGPT, gpt-engineer, etc. were all attempts at this), but to my knowledge this is the first real attempt at integrating AI into hardware after the LLM craze of the past year.
My issue is that I’m not really sure what problem this solves? Lyu starts off the event talking about how app-driven devices are bad user experiences. I’m not sure users really have a problem with lots of apps. He’s very clear that this isn’t meant to replace your phone, but it also comes with a 4G LTE slot, a camera, and can connect to Spotify, Uber, and more, so I don’t know, it’s starting to look a little like a phone replacement to me. And if it’s not a phone replacement, I don’t know how many people outside of the Silicon Valley hard tech crew you can convince to walk around with an additional device just because (hello, Google Glass).
It’s always hard to tell how much a demo reflects reality, but there’s some impressive stuff here. Getting AI to function on small devices is notoriously hard. Either your device needs to talk wirelessly to a computer far away (this is basically what happens when you use ChatGPT) and you sit around waiting for it to talk back, or you need to cram your model and all of its bits and pieces into a pretty small device and you sit around waiting for your tiny device to run this big complicated model. Rabbit AI calls their model a “Large Action Model,” which feels a bit like a rebrand of the large language and large vision models that are already around, but maybe they have some secret sauce or something. There are definitely ways to make models run faster on custom hardware, but that stuff is just…really hard, so kudos to them if they’ve pulled it off.
I’ll just come out and say that I am pretty sure that they didn’t manage to actually fit a high performance language model onto a small device, and that they’re making calls to some external model so there’ll be some latency there. I also feel like the anti-app crusade creates a very oppressive form of ecosystem lock-in, which is probably bad for healthy competition. Remember when Apple Maps was awful so we all downloaded Google Maps and then Apple Maps actually got better [2]?
One quick aside: I feel like the fact that LLMs have mostly been good at understanding stuff and not good at doing stuff is probably more of a feature than a bug? If you’re worried about AI turning very powerful and destroying the world, you should probably be glad that no one is letting these models do much stuff just yet. But I’m fully expecting Microsoft and Apple [1] to start integrating AI more closely with their internal hardware over the next few years, so I’m not sure Rabbit AI’s phone-but-not-a-phone will be able to survive. Time will tell.
The engineer in me is saying “this is a cool thing someone did so I really don’t care what its purpose is, because we should do cool things because they are cool.” I think it’s worth saying that after a 30 minute marketing video with lots of animations and flashy demos I still have no idea why I would want something like this besides the it-factor. But if Rabbit AI just ends up being a kick in the pants to Apple and Google about bringing their on-device AI on par with everything we’ve seen over the last year, that would still be a better exit than 90% of other startups, so best of luck to them.
[1] In fairness, I’ve also been expecting Apple to make Siri not be terrible for a while now, and I’m still waiting. This isn’t really a glib point,
[2] I didn’t actually go back to Apple Maps, but I’m informed it’s not bad these days.