All language subtitles for AI.Confidential.with.Hannah.Fry.S01E02.Death.by.Driverless.Car.1080p.HEVC.x265-MeGusta[EZTVx.to].en

af Afrikaans
ak Akan
sq Albanian
am Amharic
ar Arabic
hy Armenian
az Azerbaijani
eu Basque
be Belarusian
bem Bemba
bn Bengali
bh Bihari
bs Bosnian
br Breton
bg Bulgarian
km Cambodian
ca Catalan
ceb Cebuano
chr Cherokee
ny Chichewa
zh-CN Chinese (Simplified)
zh-TW Chinese (Traditional)
co Corsican
hr Croatian
cs Czech
nl Dutch
eo Esperanto
et Estonian
ee Ewe
fo Faroese
tl Filipino
fi Finnish
fy Frisian
gaa Ga
gl Galician
ka Georgian
de German
el Greek
gn Guarani
gu Gujarati
ht Haitian Creole
ha Hausa
haw Hawaiian
iw Hebrew
hi Hindi
hmn Hmong
hu Hungarian
is Icelandic
ig Igbo
id Indonesian
ia Interlingua
ga Irish
it Italian
ja Japanese
jw Javanese
kn Kannada
kk Kazakh
rw Kinyarwanda
rn Kirundi
kg Kongo
ko Korean
kri Krio (Sierra Leone)
ku Kurdish
ckb Kurdish (Soranรฎ)
ky Kyrgyz
lo Laothian
la Latin
lv Latvian
ln Lingala
lt Lithuanian
loz Lozi
lg Luganda
ach Luo
lb Luxembourgish
mk Macedonian
mg Malagasy
ms Malay
ml Malayalam
mt Maltese
mi Maori
mr Marathi
mfe Mauritian Creole
mo Moldavian
mn Mongolian
my Myanmar (Burmese)
sr-ME Montenegrin
ne Nepali
pcm Nigerian Pidgin
nso Northern Sotho
no Norwegian
nn Norwegian (Nynorsk)
oc Occitan
or Oriya
om Oromo
ps Pashto
fa Persian
pl Polish
pt-BR Portuguese (Brazil)
pt Portuguese (Portugal)
pa Punjabi
qu Quechua
rm Romansh
nyn Runyakitara
ru Russian
sm Samoan
gd Scots Gaelic
sr Serbian
sh Serbo-Croatian
st Sesotho
tn Setswana
crs Seychellois Creole
sn Shona
sd Sindhi
si Sinhalese
sk Slovak
sl Slovenian
so Somali
es Spanish
es-419 Spanish (Latin American)
su Sundanese
sw Swahili
sv Swedish
tg Tajik
ta Tamil
tt Tatar
te Telugu
th Thai
ti Tigrinya
to Tonga
lua Tshiluba
tum Tumbuka
tr Turkish
tk Turkmen
tw Twi
ug Uighur
uk Ukrainian
ur Urdu
uz Uzbek
vi Vietnamese
cy Welsh
wo Wolof
xh Xhosa
yi Yiddish
yo Yoruba
zu Zulu

Original subtitles

SIREN WAILS

A car crashes in the middle of the night.

A woman is run down.

SOBBING

It's the first death of its kind in the world...

..a pedestrian killed by a driverless car.

Although the vehicle was driving itself,

using artificial intelligence,

there was a human operator behind the wheel.

For the first time ever, the decision had to be made -

who was to blame for this crash?

Was it the human or the AI?

Artificial intelligence - a machine beyond the mind of man.

For decades, scientists have dreamed of creating incredible machines

that could talk like us, learn like us, think like us.

But what we didn't imagine is the impact they would have on us.

In this series, I'm exploring what happens

when AI collides with human lives,

unearthing stories far stranger than we could ever have imagined.

I'm Professor Hannah Fry. I'm a mathematician,

and I've spent my career examining the ways

technology can transform our future.

Here in Phoenix, a multi-billion-dollar industry

has the potential to save millions of lives.

That is weird.

SHE CHUCKLES

We're on the cusp of a self-driving revolution,

and driverless taxis are now a reality

in several countries around the world.

All right, request it.

You hail them on an app.

Ooh! Love my initials on top.

This is my first time in one of these.

Worked with Google for years.

Never been in a driverless car.

Imagine.

Here it is!

Where's it's going to stop?

Where's it going to stop?

There?

Ooh!

Oh, that was quite well done.

Here, look - HF on the top.

No-one in the car.

Start the ride.

Go!

SHE GASPS AND LAUGHS

It's quite nerve-racking!

AUTOMATED VOICE: We'll do all the driving,

so please don't touch the steering wheel or pedals during your ride.

It's green, but it slowed down.

Oh, it's turning right. That's why.

I sort of don't trust it. I don't trust it.

Ooh, we're changing lanes.

SHE LAUGHS

It's not timid, is it?

The technology needed for a car to drive itself

is nothing short of miraculous.

The car uses a combination of sensors

to understand what's going on around it.

First, cameras - great at identifying road signs,

traffic lights, pedestrians and other cars,

but they struggle in bad weather.

So, many driverless cars also have radar,

often built into the front bumper,

which sends out radio waves that bounce off objects

and measure what comes back.

It works well over long distances, but not so well up close.

That's why some driverless cars also use lidar,

the spinning cylinders you sometimes see on the roof and side of the car.

It's like radar, but with lasers,

and is able to build up a detailed 3D picture of nearby objects.

Precise but easily confused

by reflective surfaces like windows and shiny buildings.

This is sort of inside the computer's brain, as it were.

So you're seeing a view of what it thinks is around you.

The AI takes these three imperfect systems,

pieces together what's actually out there,

tries to predict what will happen next

and decides how the car should steer, brake and accelerate.

When I was writing about these things ten years ago,

they were a research project.

They were understanding the environment,

advancing the engineering, kind of tweaking the software.

It was only very, very recently that they've become

commercially available, that just anybody could hail one.

And soon they're even coming to the UK,

starting with the busy streets of London.

It is good at sticking to the speed limit, I'll tell you that.

I think there is actually quite a lot to look forward to

from this future with driverless cars.

Humans make terrible drivers.

And these things, they're not...

They're not falling asleep at the wheel.

They're not drink-driving. They're not getting road rage.

Like, fine. Maybe they're not going to be perfect,

but there is a lot of scope

for roads to be much safer than they currently are.

And I think that is a goal that is worth pursuing.

But the story of how we got to this point

is marked by tragedy and loss.

There was a pedestrian walking a bicycle.

Once the pedestrian got into the lane of traffic,

the vehicle struck the pedestrian.

It was a self-driving vehicle.

It was in the autonomous mode at the time.

In Arizona, in 2018, for the first time,

someone was struck and fatally injured

by a driverless car.

NEWSREEL: Rafaela Vasquez was the human backup driver

in the self-driving Uber vehicle that hit and killed Elaine Herzberg.

After years of avoiding the limelight,

the woman at the centre of the story,

Rafaela Vasquez, had agreed to meet me.

Hey! Hi, how are you? Good. How are you doing?

Do you want to come in? Yeah, thank you.

You haven't spoken on camera about this before, have you? No.

How are you feeling about it? I don't know.

I have a whole plethora of emotions going through me,

but my biggest issue going through this

was not being able to rebut anything or even defend myself.

It's like getting a chance to speak for yourself?

Yeah, cos I've been dealing with it now for, what, seven years?

So this is you? Mm-hm. Yes.

And that's to get you in and out of the buildings.

Rafaela got a job at Uber in 2017, not long after the company

had decided to develop their own self-driving cars.

I like a lot of technology stuff.

Self-driving vehicles were starting to emerge.

I knew that Phoenix was becoming a hotbed for it,

so I would see autonomous vehicles roaming around.

I'm interested in that, so I went and applied.

I passed everything instantly.

So, I was excited. Mmm.

Her job was to ride in the autonomous cars

as Uber began testing their new technology

on Arizona's public roads.

There was a lot of engineers, coders,

but everybody was super nice.

And you have this chance to see the new technology from the inside.

Not just see it, I'm testing it before it even comes out. Yeah.

I loved my job. I absolutely loved my job.

When you were in the cars in those early days,

how good did you think they were at driving?

They were better than what I thought they were going to be.

And it also depends. Like, Uber vehicles for the longest time

had problems with overreacting with things on the side of the road.

And some builds are great,

but then they do another update to try to fix something else

and it makes something else go haywire.

And then you just don't know.

So much has happened to you since that, right?

Yeah.

During its first year of testing in Phoenix,

Uber assigned two operators to every self-driving car.

All right, and we're engaging.

One rode in the passenger seat to track the car's performance.

On the laptop, I can monitor a lot of the prediction software.

So I can see, like, where the car is going.

Any unexpected behaviour had to be logged on a computer.

The second operator sat behind the wheel

and kept their eyes on the road.

They were expected to take control if the AI in the car malfunctioned.

But a year into testing, Uber changed the set-up.

Now one operator had to watch the road

AND monitor the car's actions.

OK. Come on, then.

This decision to switch to a single human in the car

came just a few months before Rafaela's crash.

Even riding in the car, I still get nervous. Oh, really?

Yeah. That's why I'm hesitating,

because I'm just trying to not have a panic attack. I understand.

OK.

Oh, sorry. No, don't apologise, please.

There is no pressure. Like... No, I want to do this.

Are you sure? Yeah. OK, here we go.

I don't want my driving to unnerve you.

No, I'm just worried you're going to get pulled over

cos you're not driving like a typical person.

I'm driving like a granny?

Well, you're driving like an autonomous vehicle, actually.

Oh, really? Right at the speed limit.

THEY LAUGH

Each operator was allocated a route

which the car would drive again and again.

It could be monotonous work.

Like, the most boring one I hated was this one

through this little neighbourhood. You come out, you go here.

Then this is all like 20 miles an hour down here, down here,

and then, boom, back.

And that's all it is. Round and round...

For eight hours. ..with the car driving itself. Mm-hm.

Humans are not good at paying attention when things get boring.

And with only one operator monitoring a repetitive route,

there was a danger of getting distracted.

But when it wasn't quiet on the streets,

there could be a lot for one person to do.

God, there's lots of students out, isn't there?

Yes, and this is what we're testing. Yeah.

And you can't predict what somebody's going to do.

So I would always take it out of autonomous mode during these areas.

Right. That's why two people was important.

But then all of the sudden, they changed it.

But there are several screens

that you're continually having to look at.

Yes. We were supposed to push buttons and enter codes

any time something happened.

So, you're trying to put something into the iPad

while you're supposed to be monitoring the road.

Yeah.

On the night of March 18th 2018, Rafaela was on her usual test route.

The car was in autonomous mode

and had been running for 19 minutes without incident.

At 9.58pm, it turned right onto Mill Avenue.

At the same time, a pedestrian began walking across Mill Avenue,

pushing a bicycle by her side.

Rafaela was looking down as the vehicle approached the woman.

The car should have detected her, but it didn't.

By the time Rafaela looked up and slammed on the brakes,

it was too late.

The vehicle struck the woman at 39mph.

The pedestrian killed was Elaine Herzberg.

She was often seen on her bike in the area.

She never gave up. She always helped people.

She was funny, funny, funny, funny.

If you were her friend, you felt true love from her.

She didn't deserve what happened to her.

So this is the park... That's the park. Right.

I did... That venue, that theatre, everybody crossed there,

including the homeless people,

cos the homeless people would come here to the park.

That's where she was going.

This was the first time Rafaela had returned to the site of the crash.

Do you see that sign on that post, that light? Yeah, yeah.

She was over there. Oh, gosh.

And she's just screaming the whole time. Yeah.

RAFAELA EXHALES

I...

The screaming, it...

It was just terrible to hear it. And then...

But then what was worse was when it stopped.

And then the ambulance showed up and then they said she'd passed away,

and then I lost it.

Somebody died.

Following the collision, the police launched a criminal investigation

into the artificial intelligence car

and its human operator.

Building a system that can drive a car involves much more

than turning a wheel and pressing pedals.

You also need to teach it to do things that humans do instinctively,

like spotting pedestrians and recognising road signs.

We do that without even thinking,

but training a machine to do it has proved incredibly difficult.

So back in the early days of AI, the only real form of intelligence

that we knew about was human intelligence.

And so people looked to the human brain for some inspiration

about how to build an electronic brain.

And the thing about the human brain is that it is made up

by these billions and billions of neurones

that are connected together.

And as you think,

you are essentially sending these little electrical impulses,

these little bursts that can be big or small,

through this network in your brain.

In the 1950s, people started trying to construct

a much simpler computer version

where a network of artificial neurones

would pass signals between each other.

This concept became what we now call a neural network.

But rather than all of your neurones being kind of intermeshed together,

they appear in layers, like a sort of hierarchy.

Decades later, people realised you could use these neural networks

to recognise images - like a stop sign.

Here's a simplified version of how it works.

Each neurone in the hierarchy has its own job.

At the bottom, they're just looking at a single pixel each.

And as you go further up the network,

things get more sophisticated.

Maybe there'll be a neurone up there

that is checking to see if there's some red.

Maybe there's another one up there that's checking to see

if there's an octagonal shape

that stands out from the background behind it.

And all of this information, all of these signals get sent up,

so you eventually get right to the very top,

to the big boss,

who makes a decision based on all of the information

that has flowed through the network,

to finally decide whether it thinks it's a stop sign, yes or no.

The extraordinary thing about these networks

is that if you only show it one picture,

it'll just be guessing whether it's a stop sign or not.

But show it thousands and tell it when it's right or wrong

and the AI learns to recognise the sign itself,

adjusting its network every time it makes a mistake.

In the neural networks you'll find in a car,

they'll be classifying not just stop signs,

but pedestrians, vehicles, lampposts, road markings.

They are gigantic and gigantically complex,

but the principle is still the same.

This is a machine built through trial and error.

But if errors happen in the real world, on our roads,

the consequences can be fatal.

THUD, TYRES SKID

In April 2019, a Tesla Model S failed to stop at a stop sign

and ploughed through a T junction.

MAN: I'm here. The 911's here. How are you, sir?

OFFICER: OK.

The driver of the Tesla, 42-year-old George McGee,

had been on his phone when it fell out of his hand.

He bent down to pick it up,

leaving Tesla's Autopilot to drive the car.

Check his back. Check his back.

But something went wrong.

The Tesla hit 26-year-old Dillon Angulo's truck at 62mph.

Remarkably, Dillon survived the accident.

But he wasn't alone that night.

This picture was actually the day of the crash.

And...

..we were going fishing,

and we stopped to get bait at the bait store and...

At the time of the accident, Dillon had been with his new girlfriend,

22-year-old Naibel Benavides.

Can I see her?

Oh...

The impact from the Tesla killed Naibel instantly.

She's so gorgeous.

Naibel always had this peace and happiness to her.

And just being around her would just...

It would rub off on you, you know?

I was going to meet her mom the next day.

You know, we were going to catch the fish,

and then I was going to cook lunch for her mom the next day.

And... Was that the first time you were going to meet her mum?

Yeah, that was going to be the first time. Mmm.

And unfortunately, you know, the first time that...

..I have to meet her mom were under these circumstances.

Oh, my gosh.

I'm so sorry this happened to you.

Right away, they started doing an investigation into the accident.

And I finally get my hands on the police body cam video...

..and in that police body cam video, the driver, yeah, he's like,

"I was driving. I was on the phone. I had the car on Autopilot Cruise."

I started to do research and I had no idea this existed, you know?

This thing called Autopilot, where the cars can drive themselves,

you know? And...

And right then and there, I was like...

.."This guy was relying on this car to drive itself.

"This is why this happened to us."

Autopilot is Tesla's advanced

driver assist function,

highlighted in their slick promotional videos

that sell a vision of technological sophistication,

safety and convenience,

showing that it can steer, brake and change lanes on real roads

in the real world.

Tesla Car next year will probably be

90% capable of Autopilot,

like so 90% of your miles could be on auto.

Other car companies will follow.

Elon Musk has even tweeted

that his cars can completely drive themselves,

but they can't.

The driver's manual says a human still has to be

in full control of the car.

This thing is psychotic. Oh, my God! Yeah, it's turning.

I'm not involved in this. Watch the road.

What do you think Autopilot...?! Whoa, Jesus, that was scary.

Wait, there's a double yellow line. Am I on the right side of the road?

No, you are not! Get the fuck over! Are you serious? Yes!

Look! Don't fucking trust this thing.

And there's serious safety concerns over the Autopilot feature.

REPORTER: This Tesla crashed into a highway divider in California,

killing the driver.

Another slammed into a parked fire truck in Utah.

Both had the Autopilot feature on.

This is the bend right before

the intersection where the accident happened.

And at night, from right here,

you could already see the red light blinking from right here.

Oh, yeah. I see it.

This is where her body ended up laying.

You know, we'd pulled over to look at the stars.

It's crazy how far her body flew, you know?

That's how fast the car was going straight through this intersection.

Six months after the fatal collision,

the man behind the wheel of the Tesla, George McGee,

was charged with careless driving, which he didn't contest.

But Dillon wanted Tesla to also be held accountable.

These car manufacturers, they need to do a better job

with designing these cars.

We want justice. Naibel's family and I, we want... We want justice.

These cars were allowed on the road

before they were ready to be on the road.

They were not safe.

And they were advertised as these cars that can drive themselves.

Tesla offered Dillon and Naibel's family an undisclosed sum

to draw a line under the case.

But they refused and a court date was set.

Tesla would now face a jury trial over its Autopilot system.

NEWSREADER: First At Five, the video showing the moments

before an Uber self-driving car crashes.

Backup driver Rafaela Vasquez looks down for approximately four seconds.

Back in 2018, the fallout from Rafaela's crash

had left the whole self-driving industry hanging in the balance.

If there's no human driver,

who bears the responsibility when a car makes a misstep?

Is there enough safety built in before we deploy these vehicles?

Is there any suggestion at the moment

that Uber has done something wrong? We don't know.

Under intense media and political scrutiny,

Tempe's police investigation left no stone unturned.

So the police have sent over all of their digital evidence.

Erm...

And I have not yet watched this. Let's have a look.

The actual car in the police compound being analysed.

Oh, my gosh, look at that.

And then there's also one that says "Seize phones."

They're knocking on someone's door. It's the police department.

They've obviously blurred everyone's faces.

Hey. Hey, Rafaela.

We're coming up to do follow up regarding the accident,

because we have a seizure warrant for your cellphone.

They've got a warrant for her phone.

Which one... Which number did you want? OK, Rafaela.

How many phones do you have? What? How many phones do you have?

I have my work phone and then my personal phone.

But my work phone is the one I had at work.

So the warrant's saying we need both phones.

Well, we need all the phones. How many?

All of your phones.

Why are they taking her phones?

Thank you, Rafaela.

Hi! Hi, good morning.

Kasey Marsland was the lead detective on the case.

He agreed to show me the footage he'd obtained from Uber.

So this is the footage with all three of the camera views.

When I first hit play,

we can see the driver looking down multiple times. Mmm.

And there didn't seem to be a logical explanation

as to why she was looking down.

So we're coming up to the crash now, are we? Yes.

And there.

Can you go back a few frames in this? Of course.

I just want to see what's happening immediately before. Sure.

So looking down, looking down, looking down,

looking down, looking down, looking down,

looking up... Oh. Oh, wow. Yes. HANNAH GASPS

What does she say she was looking at?

Initially, there was a statement

that it had to do with the iPad in the centre of the vehicle.

And does that broadly stack up?

Were Uber requiring people to look at screens while they were driving?

So the... Yes, that was... One of the purposes of her job,

was to monitor the vehicle.

And if there was anything abnormal,

she would need to interact with that screen.

I mean, they're conflicting instructions, right,

that, like... Yes. ..they're supposed to be looking at screens

but simultaneously monitoring the road.

Yeah, it definitely presents a bit of a challenge.

Yeah. However, when we of course looked at what the screen

was actually showing at the time,

it didn't show that there was any sort of alerts

or any sort of interaction with the screen.

We looked into the phone.

We saw the apps that were installed on her personal phone.

We noticed Hulu was an active app on her phone,

and when we wrote a search warrant to the company,

they responded with a printout of the activity on the account.

And that's what gave us probably the most important information

at the time. And what did it say?

So this was the information that was provided from Hulu.

And it shows that the show The Voice was being streamed

to her personal phone.

So what was your take on seeing all of this?

Essentially, this goes to the beginning of her shift.

She started her shift by driving the vehicle out of the garage,

and it was at that time, before she even got onto the roadway,

that she had set up and started streaming the Hulu.

And so I think that the conscious decision to provide yourself

with a likely distraction before even getting onto the roadway

is an absolutely reckless decision to be made.

So I found... I found a couple of clips around the time.

And lots of them are not on your side.

Some of them were terrible, especially at first.

Take a look at this. This is Hulu records.

It turns out she was streaming that singing competition The Voice

at the time when she should have been watching the road.

Rafaela Vasquez was riding in autonomous mode at the time,

but she was distracted and looking down for more than 30%

of the nearly 22 minutes before the crash.

Right there - they're right.

I wasn't distracted, I was doing my job.

I had other things to do other than operate the vehicle,

cos we went to one person.

So I had duties assigned.

Like, in the video, you see me look away. I am looking away.

But if you time them, and even the police did,

I never looked away for more than five seconds.

Why? Cos that's how we were trained.

We were trained to look, boom,

iPad five seconds, then look back, boom.

Over here, five seconds, look back, boom.

Our cellphones, we have to Bluetooth them in to the vehicle.

I wasn't watching TV, I was listening to it.

But the police said I was watching -

that means that's a definitive statement.

You just told the public

that you have evidence that I was watching something.

That's bullshit, because you don't.

You have evidence that I was Bluetooth streaming something,

but, hello, streaming, Bluetooth - you can listen to stuff.

People do it all the time with music.

Everyone automatically assumed - because the police said it -

that I was watching Hulu, and because of that, I was negligent,

therefore, I was unable to prevent the accident,

therefore the charge.

The investigation lasted for two and a half years

before Rafaela was finally charged.

Where were you when the indictment came through?

I was at home. My attorneys informed me.

They just told me that I was going to be indicted

for negligent homicide.

I said, "What?" And they said, "Negligent homicide."

Negligent homicide?! I said, "Homicide?"

I said, "So, murder?"

That's just me, how I interpret that. So I now...

I'm like, "Are you k..." I didn't know what to say or do.

Now I'm in shock.

If found guilty, Rafaela faced up to eight years in prison.

With conflicting accounts from both sides,

I decided to track down the official accident report.

The report concluded that the crash was probably caused

by Rafaela's failure to monitor the driving environment,

but it was also critical of Uber,

uncovering troubling flaws in the design of its AI programming.

This table is particularly interesting,

cos this is like seeing inside the brain

of what the driverless car was thinking at the time.

The car actually notices that there's something there

5.6 seconds before the collision,

which is plenty of time to start braking or turning.

And initially it's radar

that detects there's something in the distance.

It predicts it's a vehicle, though.

And then 0.4 seconds later,

the lidar kicks in and picks up that there's something there.

The lidar doesn't know what it is, though.

It just classifies it as "other", something unknown.

Now the lidar keeps changing its mind.

So a full second later,

changes its mind, decides it's a vehicle,

then 0.3 of a second later, changes its mind again,

and then it keeps switching -

vehicle, other, vehicle, other.

The computer within this driverless car is not connecting the dots.

It's not seeing this as one object whose classification keeps changing,

whose path is slowly moving over time.

Instead, it is seeing this as brand-new objects every single time.

In fact, it's only 1.2 seconds before impact

that it finally decides to settle that it's a bicycle.

Obviously way too late to actually do anything about it.

Terrifying.

You design a system that fails to track the path of an object

until it's decided what it is?

If you've got something that you're going to collide with,

I don't care what it is! I care about that it's going to collide.

That's insane.

"Although the system sensed the pedestrian nearly six seconds

"before the impact, the system never classified her as a pedestrian

"or predicted correctly her goal.

"The system design did not include

"a consideration for jaywalking pedestrians."

It wasn't designed to recognise people

unless they were on a crosswalk.

I think this is pretty damning, actually.

What was this thing doing on the roads?

The report made me want to know more about what was happening

inside Uber around the time of Rafaela's crash in 2018.

After some digging, I found Robbie Miller...

Robbie, hi!

..an operations manager who'd been at Uber

at the same time as Rafaela.

He had quit on safety grounds

just a few days before the fatal collision.

I'd been working in the self-driving space

for four or five years at this point.

I was running the self-driving truck fleet for Uber.

Eventually, there was a move to combine the operations

of the testing for self-driving cars and self-driving trucks.

How did you feel about that?

I was not at all comfortable with it.

The self-driving cars were having significant issues.

There's a lot of really scary incidences that are occurring -

near misses, near collisions.

We would have, you know, an incident

where the car was driving on the sidewalk...in broad daylight.

And you realise, like, this...

They are headed on a path

where someone is going to get seriously injured or worse.

And so I...

I gave notice.

There's this overarching fear, I would say, at Uber,

that Waymo is about to release their self-driving cars.

And it's...

I think it's very scary for Uber's leadership to not have a response.

So it's turned into a race? It is absolutely a race.

Waymo, owned by Google's parent company Alphabet,

was Uber's arch-rival.

Just five months after Waymo launched its first public trial,

Uber moved from two operators in the car to one.

You need to show to your investors, "Hey, we're making this progress."

An easy way to do that is just take someone out of the car.

And this is something I mentioned -

"You need that second person in the vehicle.

"You're not ready to take that person out of the vehicle."

Just a few days after I left, the crash occurred.

And... Where did you first hear about it?

I was driving

and I received a phone call from one of my former co-workers at Uber.

And I sat in the parking lot and cried for 15 minutes.

Wow.

I am very passionate about the technology.

I believe in the technology. I want the technology to succeed.

But there's...

There's a way to do it.

A few years after Rafaela's fatal crash,

Uber sold off its self-drive division,

leaving the path clear for Waymo.

Here you go! Waymo's just crashed.

AI technology - not working!

Why is this happening to me on a Monday? I'm in a Waymo car...

AUTOMATED VOICE: Connected to Rider Support.

..and... This call may be recorded for quality assurance.

..this car is just going in circles.

Hi there, Mike. This is Gab. I'm calling from Waymo support...

Yeah, I got a flight to catch.

Why is this thing going in a circle? I'm getting dizzy.

Waymo are now the dominant force in driverless taxis in the US,

with 2,500 of them on the road.

Experts have praised their safety record.

But some see these cars as symbols of the powerful tech elite

and their disconnection from the lives of ordinary people.

Hey! Hello.

And there's one group who've decided to take action.

Nice to meet you. It's nice to meet you. How are you all doing?

Doing well. Doing well.

They call themselves the Safe Street Rebels

and carry out their operations incognito.

What is it about the autonomous driving that you dislike, though?

They are heavily promoting themselves

as the future of public transit,

and we just fundamentally don't think

they're the future of public transit.

They're just a taxi where you don't talk to someone.

And when that car is not parked, it's still driving around, waiting.

50% of the miles they drive, there's nobody in the vehicle.

In addition, they cannot be ticketed

for any kind of moving violation in the city.

We have videos of them driving 40mph

on the wrong side of the road, and nobody can do anything about it.

They're completely immune.

What, they're above the kind of rules that would stand

for an Uber driver or a Lyft driver?

Yes, and if they can't do something about it, we can.

To express their frustration,

the group go around San Francisco disabling Waymos.

Let me understand the strategy, then.

So how easy are they to override?

LAUGHS: Pretty easy! Definitely pretty easy.

We're not allowed to show you how they disable these cars,

although it's not exactly high-tech.

So you're not putting them out of action permanently? No, no.

And we're not causing any damage to them either.

It's like... It's painter's tape,

so it doesn't leave any residue when you remove it.

There's one... There's one coming. Right.

There's one, there's one... Yeah. Check this one.

Someone go in front.

MAN: Fuck it up! Fuck it up!

CHEERING

Did you hear that?

There was people on the sidewalk who were, like, cheering them on.

The disabled car was now blocking the road.

CHIME AUTOMATED VOICE: Please step back.

So the one behind hasn't been taped but is stuck.

There's another one coming. Oh, my gosh!

Three of them...

..and they're just stuck.

And now there's another car behind, flashing.

But it has a human driver, so they can go around the problem.

Is it actually illegal, what you're doing?

We don't think it's illegal.

We can't find any law that it breaks.

It's not vandalism, so... It's hard to point any law we're breaking.

Are there not other ways that you could do this? I mean, can you...

I don't know, is there, like, not a more sort of democratic way?

It would be nice if we could vote,

if we had the opportunity to vote on them,

but we have not been presented with that opportunity yet.

Is this... Does it feel a bit like this is something

that has happened to you rather than with you?

I mean, absolutely.

Hey, yo, there's one coming the other side.

Other side, other side, other side. Other side! It's coming. Yeah.

Yeah. You want to get in front?

They are surprisingly easy to bully, aren't they?

The other thing that I think was really surprising

was just how many of them were going past, right?

And maybe it's the time of day, but, I mean...

..hardly any of them have people inside.

Yeah, it's empty.

They did say something that I hadn't considered before,

which is about how they are continually circling

when they don't have people inside them.

And that I hadn't considered, because of course,

there's a congestion aspect...

..but there's an environmental aspect, too.

I mean, they've gotta be powered.

Hmm. Interesting.

Interesting.

It happened fast.

The hearing for Rafaela Vasquez was scheduled

as a settlement conference,

but it quickly led to a plea deal.

In July 2023, after five years of legal wrangling,

Rafaela accepted a plea deal to avoid going to jail.

The agreement indicates that you wish to plead guilty to

the crime of endangerment.

The offence is non-dangerous, non-repetitive.

Is that the crime that you wish to plead guilty to? Yes.

She pled guilty to a reduced charge - endangerment -

with the guarantee it would be lowered to the least-serious

category of offence, following three years of probation.

Ah! Hey, how are you? Look who it is. Hi.

I wanted to meet Rafaela's lawyers, Al Morrison....

How are you? ..and Marci Kratter.

So good to see you.

Why did you take this case on?

Our job is to fight for the little guy - no offence -

but she was the little guy.

And the more we got into this case,

the more we realised just how one-sided it was.

Where do you think this came from, then? I mean, is this...

Is this the police, or is this from Uber?

Uber. Really?

They did whatever they could to make it not their fault.

And it's David and Goliath. Yeah.

We have this single person

up against a multimillion-dollar company with unlimited resources.

The problem was that there were so many aspects of the way

these cars were programmed

that didn't account for real-world situations.

The most obvious one is, the thing wasn't programmed

to deal with jaywalkers.

To save yourself in a campus, college campus,

to not programme vehicles to deal with jaywalking

I think is the height of negligence.

I was scared to go to trial.

If I lose, it's prison time, no ifs, ands or buts.

And the media already destroyed me out there.

That's always in the back of our minds as lawyers,

and certainly our clients' minds, that it's the exposure.

"What's going to happen to me if I go to trial and lose?"

But Marci and I felt like we had a great case,

but we also understood the risk was all hers.

Me not going to trial has no reflection on them.

These two saved my life.

They absolutely saved my life.

No criminal charges were ever brought against Uber.

Rafaela got dealt a really rough hand,

and this didn't play out fairly.

Maybe she was watching her phone.

Maybe she wasn't. Like, I...

Honestly, I don't know, but I also...

I don't think that that's the point of this.

The scandal here is that you have got this massive corporate,

multibillion-dollar project which is not prioritising

the safety of the people who are in the cars, or, like,

the members of the public who haven't even agreed

to participate in the experiment.

It is not enough to say that all of the responsibility lies with

the person who was in the car.

That's not enough. That's not enough for me.

Back in Miami, Dillon and Naibel's family were taking Tesla to court

over the fatal crash that had killed Naibel.

Nice to meet you. I'm Hannah. Adam. Great to see you. Great to meet you.

What's up, buddy?

SHE CHUCKLES

Dillon's lawyer, Adam Boumel,

brought me up to speed on the case.

This is a sort of situation where there's no precedent at all.

Does that make it quite difficult to build a legal case when you're,

I don't know, forging a new path? Extremely. Yeah?

I mean, we had to do it from the start.

This is creating the law.

Obviously, from day one, Tesla's position was,

this is 100% driver's fault.

The thing is that it's difficult to untangle the responsibility

in this case, because you're not saying that the driver had

no responsibility at all. No, of course not.

We 100% acknowledge that the driver had responsibility.

And our position from day one has been that this is a case

of shared responsibility.

Yes, the driver was at fault.

He was distracted, he was disengaged from the driving task.

But then you have to ask yourself,

why was he disengaged from the driving task?

And you realise that Tesla fostered this belief in him,

this trust of the system that was unwarranted.

He thought that the car would stop or swerve or do something

before ploughing into a parked vehicle.

Do you have the data from inside the car?

Do you know what the car was seeing?

So, we were able to finally get the data from inside the car,

showing what the Autopilot computer was detecting and processing.

The computer understood that the car was speeding

towards the end of the road.

It appreciated the stop sign.

It appreciated the blinking red light.

It appreciated Dillon's parked SUV.

So, it identified a lot of things, and it did nothing.

The car knew what was going on in front of it,

and didn't do nothing to warn the driver to stop the car.

So, this isn't a case of misinformation

inside the car's system.

The car has all of the information it needed in order

to avoid the collision.

But it was programmed not to behave in the way that the driver thought

it was programmed to behave.

Did the car behave as people expected and believed it should,

based off of Tesla's marketing and Elon Musk's statements,

in kind of hyping up the capabilities of this car?

When you take that backdrop and you put it against this case,

the car did nothing that anybody thought it would or should.

So, this has almost become a case of, like,

misadvertising or, like, misrepresentation, then?

They make the consumers and drivers believe that

the car is more capable,

creating false expectations in their drivers.

In the Uber crash, the blame had been laid

firmly on Rafaela's shoulders.

With Tesla, would this now be the first time the car itself

would be considered at fault?

Tonight, a Florida jury forcing Tesla to pay $243 million

to victims in a deadly 2019 crash,

the jury finding flaws in Tesla's self-driving software

were partly to blame.

We can be proud that we stood up,

and that we did everything in our power to help shine light

on what's going on.

Against all odds, Dillon emerged victorious in court,

forcing Tesla to take some responsibility -

for the first time ever -

for a crash involving its Autopilot system.

It was a shocking landmark verdict

that could well reshape the future of self-driving cars.

When they gave the verdict, how was it?

It was very emotional, you know?

I mean, like, me and my dad, we started hugging each other and,

you know, praying... that we would get justice.

You know, from the beginning, we knew this was

a joint liability case,

and the jury decided to hold Tesla accountable.

And I'm grateful for that.

And I'm grateful that people heard all the evidence,

and saw that Tesla made a mistake.

I really get the impression that this has never been

about money for you.

It was more important for us to shine light,

and to show the world that this technology is not safe.

People will come to me and, you know, tell me congratulations.

But I don't feel like it's something to celebrate.

Like... You're not walking away from this a winner.

I would say it's more just... justice was served. Yeah.

Tesla plan to appeal the ruling.

Meanwhile, in the UK,

British firm Wayve will launch their self-driving cars

in London later this year.

This feels like so much more high stakes than it did

when I was in Phoenix.

I mean, this is slightly wild for me, right?

Like, I've lived in London for 20 years,

and we're driving in a driverless car down Camden High Street.

THEY LAUGH

Like, this is...

This is honestly something I thought was much further in the future.

It never gets old.

Wayve CEO Alex Kendall points out that the AI in driverless cars

has come on leaps and bounds in the past few years.

And this is interesting. This person's body language was

sort of turning backwards and forwards away from the crossing.

So, that was noticeable, actually.

The car sort of changed its mind a couple of times.

It saw the person turn their body back and forth.

The AI has got quite good at learning that kind of intent.

So, it's like super, super fancy cruise control.

It's a completely different experience from cruise control.

Have I just undercut your product there?

You're comparing a floppy disk with a quantum computer.

Think of AI as the next evolution of tooling.

When you think about the wheel, the calculator, the computer -

different tools that, as humanity,

we've invented to push forward society... Mm-hm.

..I think intelligent machines will be that next evolution of this.

Whether you like it or not, driverless cars are coming.

I mean, they're here already.

And I still think there's lots of good to come from that.

But to get to this point, we had to go through that difficult phase.

We had to go through the learning - which, I mean, by definition,

learning involves making mistakes.

It's just that these mistakes, you know, these deaths,

these lives that were ruined,

I don't think that they were inevitable.

I don't think that they were an acceptable price to pay

for a new technology.

I think different choices could have been made.

I think there were different ways to balance the roll-out of

the new product with the safety of the public.

And I just hope now that these are lessons from the past,

you know, I hope that the steepest part of this learning curve

is now behind us.

SIRENS WAIL

The chief executive of one of the United States's largest

health insurance companies has been shot and killed in New York.

The alleged killer has been identified as Luigi Mangione.

The suspect had something to say about the insurance industry.

The insurance companies are relying on algorithms to make decisions

to deny patient care.

Stop denials with AI!

ALL: How many people have to die?

The anger and the upset is real.

ALL: Health care is a human right!

To discover more about AI and how it can shape our future, go to...

..or scan the QR code on the screen now.

Can't find what you're looking for?
Get subtitles in any language from opensubtitles.com, and translate them here.