Everyone’s missing the point of the Tesla Vision vs. LiDAR Wile E Coyote video

A video came out last week comparing two approaches to autonomous vehicles: cameras and LiDAR. The video was fun, as YouTube videos are wont to be, but the fallout from it has been anything but fun, with pretty much everyone missing the point of the video in the first place.

The video was posted by YouTuber Mark Rober, who typically does science & engineering related stunts. It was essentially a comparison test between Tesla’s camera-only autopilot/FSD system and LiDAR systems, with the LiDAR vehicle running Luminar’s system.

The experiment tested whether the cars could react to seeing a child in the road in six circumstances: standing, running into the road by surprise, fog, rain, bright lights, and standing behind a comical Wile E. Coyote style wall with a picture of a road painted on it.

Clearly, one of these things is not like the others. Five of the tests gave us potentially meaningful results about the world around us, and the sixth was just for fun.

Advertisement – scroll for more content

The test results showed the LiDAR doing better overall, primarily due to its better performance in fog and rain. But each vehicle produced impressive results on some of the tests – like the child jumping out in front of the car and the bright lights tests, both of which seemed quite difficult (the latter especially for a vision system).

But even in the rain and fog tests, these were quite biblical levels of rain and fog. For more realistic light fog or lighter rain, the cameras likely would have fared better.

There are a few other downsides of vision-only, such as that it can have trouble looking into lights (though it did well in the bright light test), and Tesla has in the past had a hard time with crossing trucks or overpasses being hard to distinguish from billboards, both of which can be solved with the ranging functions of LiDAR or radar.

So all told, these results track with the technical limitations of cameras when compared to LiDAR. Since cameras are passive and LiDAR is active (sending out laser pulses to reflect off of objects), LiDAR is able to “see through” certain things that cameras can’t.

And this is a debate which EV fans have heard plenty about – it’s the fundamental difference between Tesla’s approach and the approach of just about everyone else. Tesla is going vision-only, but most other companies are using a hybrid approach with some mix of vision, LiDAR, radar, ultrasonics, etc.

Tesla actually did used to have sensors other than vision, as early Tesla cars had radar in addition to cameras. But CEO Elon Musk directed the company to remove radar (over the objection of engineers) because he figures if humans can drive with two eyes and no lasers, cameras should be able to do the same. (He isn’t alone, though – Andrej Karpathy, Tesla’s former head of AI and a well respected person in the field, agrees that vision-only is the right approach).

So the tests showed us that LiDAR has some capability that vision doesn’t, but we already knew that. What are the benefits of vision-only?

First, there are clear advantages on cost and complexity, because you need less sensing equipment. LiDAR has been expensive, though costs are dropping rapidly, so this may be less of a factor going forward.

Also, LiDAR sensors used to be huge spinning rigs attached to vehicles, but now they often take the form of a “taxi bump” that looks a bit like a taxi light on the top of the car, just above the windshield – but this still does restrict the design of a vehicle and a lot of people don’t like the look.

Second, vision-only could potentially make for a simpler software solution because you don’t have to reconcile the input from multiple sensing methods to figure out the reality in front of you.

This is something that held Tesla back in the early days of vision + radar, because there were a lot of false positives and negatives from weird situations (e.g. curved metal objects like soda cans could look bigger than they should, stationary vehicles were hard to distinguish, etc.). While the data was more robust because there were multiple sensing methods, it was proving itself harder to interpret.

And, while it’s not an inherent benefit of vision-only, the specific benefit for Tesla is that the company has a LOT of vision data it can use for training. This is a big advantage that it has over every other company by several orders of magnitude, since millions of Teslas have been driving around collecting data for years now, whereas companies like Waymo only have a few hundred cars.

So, we know a bit about the differences in technology, their strengths and weaknesses, and the long-time industry debate that motivated this test. Nothing seems all that unreasonable about what we’ve heard so far, and the test turned out about as expected. There’s still an open question over what the best path forward is, though the general consensus is that more sensing data is better than less, and that Tesla is making a risky move with its vision-only system.

So, why so much drama?

Okay, well, it’s the internet. So that’s reason number one. Everyone else here is chasing the same thing Rober chases: views. And so that’s probably the only thing we need to say, alright, article over, moving on.

…. But no, really. The actual drama is over the differentiation between “Autopilot” and “Full Self-Driving,” and over the behavior of Teslas when activating or deactivating the system, specifically on the headline “Wile E. Coyote” test.

Most discussion has focused on this particular test, because, well, it’s the most fun one. Rober is one of the most popular YouTubers on the planet, after all, so he should know a thing or two about how to make a compelling video (and the intro sentence of the video is quite a doozy):

As Rober said in the very first line of the video, he had his Tesla on Autopilot, not Full Self-Driving, during this test.

Some criticism has focused on the title of the video, which is “Can You Fool A Self Driving Car?”, suggesting that the test would use Tesla’s “Self-Driving” system.

These are two separate systems, and FSD is more sophisticated than Autopilot. However, Autopilot has long colloquially been referred to as self-driving (often to the chagrin of Tesla defenders), and while Tesla does refer to FSD as “self-driving,” it very much isn’t. Both of the systems are classed as “level 2,” which means the driver is still responsible for the vehicle at all times, even though FSD can be activated in more situations than Autopilot. And many more Teslas have Autopilot than FSD, so it makes sense to test the more common one.

Luminar’s LiDAR can be “self-driving,” insofar as there are level 3+ systems that use Luminar’s sensing technology (such as Mercedes’ DRIVE PILOT).

So the title is not technically incorrect, does use similar colloquialisms in both cases, and is, after all, a youtube video, and we’re all hopefully aware of how YouTube titles need to be crafted to fit Google’s algorithm and hopefully can get beyond the title and into the literal first frame of the video for the more accurate description of what’s happening here.

And we’ve covered a final criticism before, which is a screenshot showing that Rober didn’t have the system active in the video. This is previously-documented as “normal” Autopilot behavior, where the system turns itself off about a second before a definite crash. The screenshots were taken during this second. Rober also responded mentioning that the video used different takes to keep it compelling, and posted the full uncut footage on Twitter.

Another criticism focuses on the subsequent stock surge seen by Luminar (LAZR). The company’s stock went up from 5.05 to 8.35 over the course of the week after the video, a rise of 65%. This has raised some eyebrows, but I expect that the main explanation here is that prior to the video, only pretty dedicated EV/self-driving folks knew about Luminar, and now it’s been exposed to people associated with the most traded stock on the planet for several years running, TSLA. This is naturally going to drive a ton of volume to a small stock (with ~0.03% of TSLA’s market cap).

We’ve also seen others trying to recreate the video, with more success for the Tesla.

But these criticisms focus mainly on the Wile E. Coyote test, which everyone acknowledges is not a realistic situation. That test was for the youtube video – the real meat of it was the other 5 tests that actually could happen in the real world.

And even on those 5 tests, people are getting overexcited about the differences shown. The fog and water were both significantly heavier than what would most often be experienced in real life. In more “real world” weather circumstances, a camera may have worked plenty well enough (assuming the cameras aren’t obscured by water or condensation – which is certainly an issue). And if the inclement weather is as bad as shown in the video – maybe it’s time to stay home (or, uh, head straight to the hurricane evacuation center).

All in all, it felt like a fun test for a YouTube video, which described technology in a simple way to a crowd that hadn’t heard about it, was generally accurate about the strengths and weaknesses of the compared systems, but just overstated a lot of things “for content.”

There’s a discussion to be had there about content requiring more and more extreme stunts these days to be compelling, but the level of the reaction has gone well overboard. But then, that’s to be expected for anything on the internet, especially about Tesla.

And the discussion over which approach is correct will continue – companies like Luminar think that LiDAR is superior, and Tesla thinks cameras are enough. Time will tell who’s right, but most professionals in the field tend to place their bets on the former, rather than the latter.

Charge your electric vehicle at home using rooftop solar panels. Find a reliable and competitively priced solar installer near you on EnergySage, for free. They have pre-vetted installers competing for your business, ensuring high-quality solutions and 20-30% savings. It’s free, with no sales calls until you choose an installer. Compare personalized solar quotes online and receive guidance from unbiased Energy Advisers. Get started here. – ad*