Summary

• At the recent AI Day 2022, Tesla unveiled its Optimus robot, presented the inner workings of its self-driving and AI technologies, and briefly captured the attention of everyone working in AI and robotics.

• Reactions from media and experts poured in during the aftermath of the event; some praised the speed of the humanoid hardware development, while others criticized the lack of anything truly novel.

• Regardless of actual product development, this AI Day worked wonders for Tesla in terms of publicity, which the company hopes will help them recruit the necessary talent for its AI ambitions.

What Happened

Tesla recently hosted its second AI Day to present the company’s progress in AI and to recruit engineers in robotics, AI, and hardware. Perhaps what the public and media looked forward to the most is Tesla’s work-in-progress humanoid robot, Optimus, which at last year’s AI day was demo’ed as a person dressed up in a spandex suit. We’ll give a quick summary of what went down at AI day and some commentary, some reactions from the press and experts, and our take on the event.

Optimus Bot

Tesla first showed a working prototype that was able to walk and wave its arms on stage. Afterward, they also showed the newer in-development prototype design of the robot, which they say is designed for mass-manufacturing. This prototype couldn’t walk on its own, so Tesla engineers rolled out the robot on a platform:

This demonstrates Tesla is still in the early days of developing the necessary hardware platform. Keep in mind, however, that giving a live humanoid demonstration is much riskier and more challenging than showing demo videos, so the live humanoid walking demo is more impressive than it looks.

Speaking of demo videos, Tesla did show a few clips where the robot went beyond locomotion and manipulated objects, including carrying a box and “doing” an industrial insertion task:

I put “doing” in quotes because the video cuts right before where the insertion would’ve happened. It’s likely for this particular demo the robot was not actually capable of completing the task all the way, and the video is mostly to show the arm/body coordination as well as the vision system:

Compared to other humanoid robot companies, Tesla does have an advantage in building and deploying advanced computer vision systems, and the Optimus team no doubt benefits tremendously from using existing software and tools from the self-driving team. This stands in contrast to the action side of Optimus, with which Tesla cannot easily bootstrap from the self-driving side. What the company demonstrated was a high-level pipeline where a motion capture system was used to record full-body demonstrations, which was then used to help optimize for the appropriate robot actions.

This is a far less scalable method of data collection than taking advantage of Tesla’s fleet of vehicles for self-driving. Data quality also matters more here as we need to learn to perform tasks that interact with the physical world, not just perception. We covered why getting such robot data is hard in a previous editorial (see below), and it remains unclear how Tesla’s approach here fundamentally improves upon anything. I guess we’ll have to wait until the next AI Day to find out.

Last Week in AI

Getting Data for Robot AI — Hard, but Possible

This is the third of a series on the challenges and opportunities of applying AI to robotics, specifically in the setting of autonomous service robots that can assist humans in everyday tasks. See the overview here. TL;DR Deep-learning-driven AI needs a lot of data. Robot AIs need data of robots interacting with the world, which is a lot harder to get th…

Read more

a year ago · Jacky Liang

Full Self-Driving

Elon is famous for repeatedly promising that full self-driving is just around the corner and then failing to deliver on that promise (so much so that there is now a class-action lawsuit). It’s clear that FSD is still not ready for prime time, but there has been steady and continuous progress. Tesla revealed that FSD beta is currently deployed on 160k vehicles, an astonishing number (certainly more than its competitors - I doubt Waymo has 160k cars).

If the answer to self-driving is collecting data and learning at larger and large scales (which it very well may not be, or it might be necessary but not sufficient), then Tesla is certainly in a good position. Deploying any AI model to 160k customers is no small accomplishment. However, the company did not give a concrete timeline on when FSD can be expected to come out of beta. The presentations also didn’t report the rate of human interventions during FSD beta. This is understandable as it is probably a trade secret, but some vague plot of how the intervention rate is going down with each FSD update and how far the system still has to go before an acceptable level is reached would’ve been nice.

The FSD presentations instead focused on the technical approach, from perception to chips to data labeling and more. The perception model that runs on the car is able to predict full 3D occupancy and motion flows of the environment around the car, among other things, all within 10 milliseconds. This is quite impressive and speaks a lot about Tesla’s tight integration of AI software with its custom hardware.

The Reactions

IEEE Spectrum summarized many robotics experts’ reactions in What Robotics Experts Think of Tesla’s Optimus Robot. Reactions are unsurprisingly diverse, but there is a broad agreement that Tesla has achieved impressive progress on the hardware in just a year, but that nothing it has done is very original:

Looking at the Tesla Bot as a roboticist, I am impressed by what the engineers achieved for this prototype in a year. However, the behaviors demonstrated are less impressive than that of Honda’s Asimo from 20 years ago - Georgia Chalvatzaki, Assistant Professor at Technische Universität Darmstadt

Optimus reveal: Mind blown with the velocity of the team and the very sleek hardware design elements. Yet to see autonomy. Surprised Tesla went full Boston Dynamics mode with classical control/planning when it’s been around for a while… -Keerthana Gopalakrishnan, Roboticist at Google Brain

What was most impressive to me was what the Optimus team was able to accomplish in such a short period of time. If you are in this field, you would agree, too. The prototype they have created will serve as an excellent beginning platform for them to learn from and to build upon. -Dennis Hong, Professor at UCLA

Our Perspective

While big tech companies are known to host large-scale press events from time to time, Tesla’s AI Day is not quite the same. As Elon tweeted, the event was highly technical and geared toward recruiting. Having attended AI Day in person, I’ll roughly divide the evening into 3 parts: a career fair, a bunch of tech talks, and a Q&A session. This departs from the popular coverage on the event, which seem to overtly comment on the “product” angle of AI Day (e.g. treating the Optimus demo as a product announcement), which to in-person attendants was clearly not the focus.

The career fair: What’s not seen in the live stream are the rows of recruiting booths around the stage/seating area. Each booth is assigned a specific team at Tesla and staffed with engineers from that team. This is similar to a normal career fair, except it’s for the different teams of one company instead of multiple companies. I enjoyed this setup quite a bit as one can delve into deeper and more specific conversations about the tech stack at Tesla, in contrast to the more generic recruiting experience and big career fairs.

The tech talks: Admittedly this part could’ve used some improvements, but I’m not sure what the better alternative would’ve been. A two and half hour tech talk is just way too long. This is ok from a retrospective online viewing perspective, since one can just jump to the portion of the video they’re specifically interested in. But for the live audience member, sitting through everything was…rough. Those curious about humanoid motors might not care so much about FSD data labeling, and those trying to understand 3D computer vision might’ve been quite frustrated with the extended discussion in chip design. Nonetheless, I was impressed with Tesla’s openness with their technical approach (I don’t recall seeing this level of technical detail from other AI companies’ public presentations). This helps them both to showcase the breadth of Tesla AI and to garner the attention of potential job applicants.

The Q&A session: It was pretty cool to see Elon and the Tesla team open up the floor for a general, unmoderated Q&A session. It took about an hour, and people asked everything from incoming FSD updates to AGI. Of course this was not without its drawbacks - the unmoderated nature meant question topics swung wildly from one area to the next, and tired audience members leaving the presentation area was quite distracting. Still, the openness and willingness to engage with the audience was impressive, and it makes a lot of sense in the context of a recruiting event.

In conclusion, Tesla successfully drew the attention of the AI world with its AI Day 2022. The company managed to gather a highly technical and specific audience (engineers and researchers in robotics/AI) into what was essentially a one-company career fair. The reveal of the Optimus robot and the ensuing hype/media coverage helped further Tesla’s publicity goals, but the event worked even better for recruiting, its main purpose.