All language subtitles for AI.Confidential.with.Hannah.Fry.S01E02.Death.by.Driverless.Car.1080p.HEVC.x265-MeGusta[EZTVx.to].en

SIREN WAILS

A car crashes
in the middle of the night.

A woman is run down.

SOBBING

It's the first death of its kind
in the world...

..a pedestrian killed by
a driverless car.

Although the vehicle
was driving itself,

using artificial intelligence,

there was a human operator
behind the wheel.

For the first time ever,
the decision had to be made -

who was to blame for this crash?

Was it the human or the AI?

Artificial intelligence -
a machine beyond the mind of man.

For decades, scientists have dreamed
of creating incredible machines

that could talk like us,
learn like us, think like us.

But what we didn't imagine
is the impact they would have on us.

In this series,
I'm exploring what happens

when AI collides with human lives,

unearthing stories far stranger than
we could ever have imagined.

I'm Professor Hannah Fry.
I'm a mathematician,

and I've spent my career
examining the ways

technology can transform our future.

Here in Phoenix,
a multi-billion-dollar industry

has the potential
to save millions of lives.

That is weird.

SHE CHUCKLES

We're on the cusp of
a self-driving revolution,

and driverless taxis
are now a reality

in several countries
around the world.

All right, request it.

You hail them on an app.

Ooh! Love my initials on top.

This is my first time
in one of these.

Worked with Google for years.

Never been in a driverless car.

Imagine.

Here it is!

Where's it's going to stop?

Where's it going to stop?

There?

Ooh!

Oh, that was quite well done.

Here, look - HF on the top.

No-one in the car.

Start the ride.

Go!

SHE GASPS AND LAUGHS

It's quite nerve-racking!

AUTOMATED VOICE: We'll do
all the driving,

so please don't touch the steering
wheel or pedals during your ride.

It's green, but it slowed down.

Oh, it's turning right.
That's why.

I sort of don't trust it.
I don't trust it.

Ooh, we're changing lanes.

SHE LAUGHS

It's not timid, is it?

The technology needed for a car
to drive itself

is nothing short of miraculous.

The car uses a combination
of sensors

to understand what's going on
around it.

First, cameras -
great at identifying road signs,

traffic lights, pedestrians
and other cars,

but they struggle in bad weather.

So, many driverless cars
also have radar,

often built into the front bumper,

which sends out radio waves
that bounce off objects

and measure what comes back.

It works well over long distances,
but not so well up close.

That's why some driverless cars
also use lidar,

the spinning cylinders you sometimes
see on the roof and side of the car.

It's like radar,
but with lasers,

and is able to build up a detailed
3D picture of nearby objects.

Precise but easily confused

by reflective surfaces
like windows and shiny buildings.

This is sort of inside
the computer's brain, as it were.

So you're seeing a view of
what it thinks is around you.

The AI takes these
three imperfect systems,

pieces together what's actually
out there,

tries to predict
what will happen next

and decides how the car should
steer, brake and accelerate.

When I was writing about
these things ten years ago,

they were a research project.

They were understanding
the environment,

advancing the engineering,
kind of tweaking the software.

It was only very, very recently
that they've become

commercially available,
that just anybody could hail one.

And soon they're even coming
to the UK,

starting with the busy streets
of London.

It is good at sticking to
the speed limit, I'll tell you that.

I think there is actually
quite a lot to look forward to

from this future
with driverless cars.

Humans make terrible drivers.

And these things, they're not...

They're not falling asleep
at the wheel.

They're not drink-driving.
They're not getting road rage.

Like, fine. Maybe they're not
going to be perfect,

but there is a lot of scope

for roads to be much safer than
they currently are.

And I think that is a goal
that is worth pursuing.

But the story of how we got
to this point

is marked by tragedy and loss.

There was a pedestrian
walking a bicycle.

Once the pedestrian got into
the lane of traffic,

the vehicle struck the pedestrian.

It was a self-driving vehicle.

It was in the autonomous mode
at the time.

In Arizona, in 2018,
for the first time,

someone was struck
and fatally injured

by a driverless car.

NEWSREEL: Rafaela Vasquez
was the human backup driver

in the self-driving Uber vehicle
that hit and killed Elaine Herzberg.

After years of
avoiding the limelight,

the woman at the centre
of the story,

Rafaela Vasquez,
had agreed to meet me.

Hey! Hi, how are you?
Good. How are you doing?

Do you want to come in?
Yeah, thank you.

You haven't spoken on camera
about this before, have you? No.

How are you feeling about it?
I don't know.

I have a whole plethora of emotions
going through me,

but my biggest issue
going through this

was not being able to rebut anything
or even defend myself.

It's like getting a chance
to speak for yourself?

Yeah, cos I've been dealing with it
now for, what, seven years?

So this is you? Mm-hm. Yes.

And that's to get you
in and out of the buildings.

Rafaela got a job at Uber in 2017,
not long after the company

had decided to develop
their own self-driving cars.

I like a lot of technology stuff.

Self-driving vehicles
were starting to emerge.

I knew that Phoenix was becoming
a hotbed for it,

so I would see autonomous vehicles
roaming around.

I'm interested in that,
so I went and applied.

I passed everything instantly.

So, I was excited. Mmm.

Her job was to ride
in the autonomous cars

as Uber began testing
their new technology

on Arizona's public roads.

There was a lot of
engineers, coders,

but everybody was super nice.

And you have this chance to see
the new technology from the inside.

Not just see it, I'm testing it
before it even comes out. Yeah.

I loved my job.
I absolutely loved my job.

When you were in the cars
in those early days,

how good did you think they were
at driving?

They were better than what I thought
they were going to be.

And it also depends. Like,
Uber vehicles for the longest time

had problems with overreacting
with things on the side of the road.

And some builds are great,

but then they do another update
to try to fix something else

and it makes something else
go haywire.

And then you just don't know.

So much has happened to you
since that, right?

Yeah.

During its first year of testing
in Phoenix,

Uber assigned two operators
to every self-driving car.

All right, and we're engaging.

One rode in the passenger seat
to track the car's performance.

On the laptop, I can monitor
a lot of the prediction software.

So I can see, like,
where the car is going.

Any unexpected behaviour
had to be logged on a computer.

The second operator
sat behind the wheel

and kept their eyes on the road.

They were expected to take control
if the AI in the car malfunctioned.

But a year into testing,
Uber changed the set-up.

Now one operator
had to watch the road

AND monitor the car's actions.

OK. Come on, then.

This decision to switch
to a single human in the car

came just a few months before
Rafaela's crash.

Even riding in the car,
I still get nervous. Oh, really?

Yeah. That's why I'm hesitating,

because I'm just trying to not have
a panic attack. I understand.

OK.

Oh, sorry.
No, don't apologise, please.

There is no pressure. Like...
No, I want to do this.

Are you sure? Yeah.
OK, here we go.

I don't want my driving
to unnerve you.

No, I'm just worried
you're going to get pulled over

cos you're not driving like
a typical person.

I'm driving like a granny?

Well, you're driving like
an autonomous vehicle, actually.

Oh, really?
Right at the speed limit.

THEY LAUGH

Each operator was allocated a route

which the car would drive
again and again.

It could be monotonous work.

Like, the most boring one I hated
was this one

through this little neighbourhood.
You come out, you go here.

Then this is all like 20 miles
an hour down here, down here,

and then, boom, back.

And that's all it is.
Round and round...

For eight hours. ..with the car
driving itself. Mm-hm.

Humans are not good at paying
attention when things get boring.

And with only one operator
monitoring a repetitive route,

there was a danger of
getting distracted.

But when it wasn't quiet
on the streets,

there could be a lot
for one person to do.

God, there's lots of students out,
isn't there?

Yes, and this is what we're testing.
Yeah.

And you can't predict
what somebody's going to do.

So I would always take it out of
autonomous mode during these areas.

Right.
That's why two people was important.

But then all of the sudden,
they changed it.

But there are several screens

that you're continually
having to look at.

Yes. We were supposed
to push buttons and enter codes

any time something happened.

So, you're trying to put something
into the iPad

while you're supposed to be
monitoring the road.

Yeah.

On the night of March 18th 2018,
Rafaela was on her usual test route.

The car was in autonomous mode

and had been running for 19 minutes
without incident.

At 9.58pm,
it turned right onto Mill Avenue.

At the same time, a pedestrian began
walking across Mill Avenue,

pushing a bicycle by her side.

Rafaela was looking down
as the vehicle approached the woman.

The car should have detected her,
but it didn't.

By the time Rafaela looked up
and slammed on the brakes,

it was too late.

The vehicle struck the woman
at 39mph.

The pedestrian killed
was Elaine Herzberg.

She was often seen on her bike
in the area.

She never gave up.
She always helped people.

She was funny, funny, funny, funny.

If you were her friend,
you felt true love from her.

She didn't deserve
what happened to her.

So this is the park...
That's the park. Right.

I did... That venue, that theatre,
everybody crossed there,

including the homeless people,

cos the homeless people
would come here to the park.

That's where she was going.

This was the first time Rafaela had
returned to the site of the crash.

Do you see that sign on that post,
that light? Yeah, yeah.

She was over there. Oh, gosh.

And she's just screaming
the whole time. Yeah.

RAFAELA EXHALES

I...

The screaming, it...

It was just terrible to hear it.
And then...

But then what was worse
was when it stopped.

And then the ambulance showed up and
then they said she'd passed away,

and then I lost it.

Somebody died.

Following the collision, the police
launched a criminal investigation

into the artificial intelligence car

and its human operator.

Building a system that can
drive a car involves much more

than turning a wheel
and pressing pedals.

You also need to teach it to do
things that humans do instinctively,

like spotting pedestrians
and recognising road signs.

We do that without even thinking,

but training a machine to do it
has proved incredibly difficult.

So back in the early days of AI,
the only real form of intelligence

that we knew about
was human intelligence.

And so people looked to
the human brain for some inspiration

about how to build
an electronic brain.

And the thing about the human brain
is that it is made up

by these billions and billions
of neurones

that are connected together.

And as you think,

you are essentially sending
these little electrical impulses,

these little bursts
that can be big or small,

through this network in your brain.

In the 1950s, people started
trying to construct

a much simpler computer version

where a network of
artificial neurones

would pass signals
between each other.

This concept became what we now call
a neural network.

But rather than all of your neurones
being kind of intermeshed together,

they appear in layers,
like a sort of hierarchy.

Decades later, people realised
you could use these neural networks

to recognise images -
like a stop sign.

Here's a simplified version
of how it works.

Each neurone in the hierarchy
has its own job.

At the bottom, they're just looking
at a single pixel each.

And as you go further up
the network,

things get more sophisticated.

Maybe there'll be a neurone up there

that is checking to see
if there's some red.

Maybe there's another one up there
that's checking to see

if there's an octagonal shape

that stands out from the background
behind it.

And all of this information,
all of these signals get sent up,

so you eventually get right to
the very top,

to the big boss,

who makes a decision based on
all of the information

that has flowed through the network,

to finally decide whether it thinks
it's a stop sign, yes or no.

The extraordinary thing
about these networks

is that if you only show it
one picture,

it'll just be guessing
whether it's a stop sign or not.

But show it thousands
and tell it when it's right or wrong

and the AI learns to recognise
the sign itself,

adjusting its network
every time it makes a mistake.

In the neural networks
you'll find in a car,

they'll be classifying
not just stop signs,

but pedestrians, vehicles,
lampposts, road markings.

They are gigantic
and gigantically complex,

but the principle is still the same.

This is a machine built
through trial and error.

But if errors happen
in the real world, on our roads,

the consequences can be fatal.

THUD, TYRES SKID

In April 2019, a Tesla Model S
failed to stop at a stop sign

and ploughed through a T junction.

MAN: I'm here. The 911's here.
How are you, sir?

OFFICER: OK.

The driver of the Tesla,
42-year-old George McGee,

had been on his phone
when it fell out of his hand.

He bent down to pick it up,

leaving Tesla's Autopilot
to drive the car.

Check his back. Check his back.

But something went wrong.

The Tesla hit 26-year-old
Dillon Angulo's truck at 62mph.

Remarkably, Dillon survived
the accident.

But he wasn't alone that night.

This picture was actually the day
of the crash.

And...

..we were going fishing,

and we stopped to get bait
at the bait store and...

At the time of the accident, Dillon
had been with his new girlfriend,

22-year-old Naibel Benavides.

Can I see her?

Oh...

The impact from the Tesla
killed Naibel instantly.

She's so gorgeous.

Naibel always had this peace
and happiness to her.

And just being around her
would just...

It would rub off on you, you know?

I was going to meet her mom
the next day.

You know,
we were going to catch the fish,

and then I was going to cook lunch
for her mom the next day.

And... Was that the first time
you were going to meet her mum?

Yeah, that was going to be
the first time. Mmm.

And unfortunately, you know,
the first time that...

..I have to meet her mom
were under these circumstances.

Oh, my gosh.

I'm so sorry this happened to you.

Right away, they started doing
an investigation into the accident.

And I finally get my hands
on the police body cam video...

..and in that police body cam video,
the driver, yeah, he's like,

"I was driving. I was on the phone.
I had the car on Autopilot Cruise."

I started to do research and I had
no idea this existed, you know?

This thing called Autopilot,
where the cars can drive themselves,

you know? And...

And right then and there,
I was like...

.."This guy was relying on this car
to drive itself.

"This is why this happened to us."

Autopilot is Tesla's advanced

driver assist function,

highlighted in their slick
promotional videos

that sell a vision of
technological sophistication,

safety and convenience,

showing that it can steer, brake
and change lanes on real roads

in the real world.

Tesla Car next year will probably be

90% capable of Autopilot,

like so 90% of your miles
could be on auto.

Other car companies will follow.

Elon Musk has even tweeted

that his cars can completely
drive themselves,

but they can't.

The driver's manual
says a human still has to be

in full control of the car.

This thing is psychotic.
Oh, my God! Yeah, it's turning.

I'm not involved in this.
Watch the road.

What do you think Autopilot...?!
Whoa, Jesus, that was scary.

Wait, there's a double yellow line.
Am I on the right side of the road?

No, you are not! Get the fuck over!
Are you serious? Yes!

Look! Don't fucking trust
this thing.

And there's serious safety concerns
over the Autopilot feature.

REPORTER: This Tesla crashed into
a highway divider in California,

killing the driver.

Another slammed into
a parked fire truck in Utah.

Both had the Autopilot feature on.

This is the bend right before

the intersection where
the accident happened.

And at night, from right here,

you could already see the red light
blinking from right here.

Oh, yeah. I see it.

This is where her body
ended up laying.

You know, we'd pulled over
to look at the stars.

It's crazy how far her body flew,
you know?

That's how fast the car was going
straight through this intersection.

Six months after
the fatal collision,

the man behind the wheel of
the Tesla, George McGee,

was charged with careless driving,
which he didn't contest.

But Dillon wanted Tesla
to also be held accountable.

These car manufacturers,
they need to do a better job

with designing these cars.

We want justice. Naibel's family
and I, we want... We want justice.

These cars were allowed on the road

before they were ready
to be on the road.

They were not safe.

And they were advertised as these
cars that can drive themselves.

Tesla offered Dillon and
Naibel's family an undisclosed sum

to draw a line under the case.

But they refused
and a court date was set.

Tesla would now face a jury trial
over its Autopilot system.

NEWSREADER: First At Five,
the video showing the moments

before an Uber
self-driving car crashes.

Backup driver Rafaela Vasquez looks
down for approximately four seconds.

Back in 2018, the fallout
from Rafaela's crash

had left the whole self-driving
industry hanging in the balance.

If there's no human driver,

who bears the responsibility
when a car makes a misstep?

Is there enough safety built in
before we deploy these vehicles?

Is there any suggestion
at the moment

that Uber has done something wrong?
We don't know.

Under intense media
and political scrutiny,

Tempe's police investigation
left no stone unturned.

So the police have sent over
all of their digital evidence.

Erm...

And I have not yet watched this.
Let's have a look.

The actual car in the police
compound being analysed.

Oh, my gosh, look at that.

And then there's also one
that says "Seize phones."

They're knocking on someone's door.
It's the police department.

They've obviously blurred
everyone's faces.

Hey. Hey, Rafaela.

We're coming up to do follow up
regarding the accident,

because we have a seizure warrant
for your cellphone.

They've got a warrant for her phone.

Which one... Which number
did you want? OK, Rafaela.

How many phones do you have?
What? How many phones do you have?

I have my work phone
and then my personal phone.

But my work phone is the one I had
at work.

So the warrant's
saying we need both phones.

Well, we need all the phones.
How many?

All of your phones.

Why are they taking her phones?

Thank you, Rafaela.

Hi! Hi, good morning.

Kasey Marsland was
the lead detective on the case.

He agreed to show me the footage
he'd obtained from Uber.

So this is the footage
with all three of the camera views.

When I first hit play,

we can see the driver
looking down multiple times. Mmm.

And there didn't seem to be
a logical explanation

as to why she was looking down.

So we're coming up to the crash now,
are we? Yes.

And there.

Can you go back a few frames
in this? Of course.

I just want to see what's happening
immediately before. Sure.

So looking down, looking down,
looking down,

looking down, looking down,
looking down,

looking up... Oh. Oh, wow. Yes.
HANNAH GASPS

What does she say
she was looking at?

Initially, there was a statement

that it had to do with the iPad
in the centre of the vehicle.

And does that broadly stack up?

Were Uber requiring people to look
at screens while they were driving?

So the... Yes, that was... One
of the purposes of her job,

was to monitor the vehicle.

And if there was anything abnormal,

she would need to interact
with that screen.

I mean, they're conflicting
instructions, right,

that, like... Yes. ..they're
supposed to be looking at screens

but simultaneously monitoring
the road.

Yeah, it definitely presents
a bit of a challenge.

Yeah. However, when we of course
looked at what the screen

was actually showing at the time,

it didn't show
that there was any sort of alerts

or any sort of interaction
with the screen.

We looked into the phone.

We saw the apps that were installed
on her personal phone.

We noticed Hulu was an active app
on her phone,

and when we wrote
a search warrant to the company,

they responded with a printout
of the activity on the account.

And that's what gave us probably
the most important information

at the time.
And what did it say?

So this was the information
that was provided from Hulu.

And it shows that the show The Voice
was being streamed

to her personal phone.

So what was your take
on seeing all of this?

Essentially, this goes to
the beginning of her shift.

She started her shift by driving
the vehicle out of the garage,

and it was at that time, before
she even got onto the roadway,

that she had set up
and started streaming the Hulu.

And so I think that the conscious
decision to provide yourself

with a likely distraction
before even getting onto the roadway

is an absolutely reckless decision
to be made.

So I found... I found a couple
of clips around the time.

And lots of them
are not on your side.

Some of them were terrible,
especially at first.

Take a look at this.
This is Hulu records.

It turns out she was streaming
that singing competition The Voice

at the time when she should have
been watching the road.

Rafaela Vasquez was riding
in autonomous mode at the time,

but she was distracted
and looking down for more than 30%

of the nearly 22 minutes
before the crash.

Right there - they're right.

I wasn't distracted,
I was doing my job.

I had other things to do
other than operate the vehicle,

cos we went to one person.

So I had duties assigned.

Like, in the video, you see me
look away. I am looking away.

But if you time them,
and even the police did,

I never looked away
for more than five seconds.

Why? Cos that's how we were trained.

We were trained to look, boom,

iPad five seconds,
then look back, boom.

Over here, five seconds,
look back, boom.

Our cellphones, we have to
Bluetooth them in to the vehicle.

I wasn't watching TV,
I was listening to it.

But the police said I was watching -

that means that's
a definitive statement.

You just told the public

that you have evidence
that I was watching something.

That's bullshit, because you don't.

You have evidence that I was
Bluetooth streaming something,

but, hello, streaming, Bluetooth -
you can listen to stuff.

People do it all the time
with music.

Everyone automatically assumed -
because the police said it -

that I was watching Hulu, and
because of that, I was negligent,

therefore, I was unable
to prevent the accident,

therefore the charge.

The investigation lasted
for two and a half years

before Rafaela was finally charged.

Where were you
when the indictment came through?

I was at home.
My attorneys informed me.

They just told me that
I was going to be indicted

for negligent homicide.

I said, "What?"
And they said, "Negligent homicide."

Negligent homicide?!
I said, "Homicide?"

I said, "So, murder?"

That's just me,
how I interpret that. So I now...

I'm like, "Are you k..."
I didn't know what to say or do.

Now I'm in shock.

If found guilty, Rafaela faced up
to eight years in prison.

With conflicting accounts
from both sides,

I decided to track down
the official accident report.

The report concluded that the crash
was probably caused

by Rafaela's failure to monitor
the driving environment,

but it was also critical of Uber,

uncovering troubling flaws
in the design of its AI programming.

This table
is particularly interesting,

cos this is like seeing
inside the brain

of what the driverless car
was thinking at the time.

The car actually notices that
there's something there

5.6 seconds before the collision,

which is plenty of time
to start braking or turning.

And initially it's radar

that detects there's something
in the distance.

It predicts it's a vehicle, though.

And then 0.4 seconds later,

the lidar kicks in and picks up
that there's something there.

The lidar doesn't know
what it is, though.

It just classifies it as "other",
something unknown.

Now the lidar
keeps changing its mind.

So a full second later,

changes its mind,
decides it's a vehicle,

then 0.3 of a second later,
changes its mind again,

and then it keeps switching -

vehicle, other, vehicle, other.

The computer within this driverless
car is not connecting the dots.

It's not seeing this as one object
whose classification keeps changing,

whose path is slowly moving
over time.

Instead, it is seeing this as
brand-new objects every single time.

In fact, it's only 1.2 seconds
before impact

that it finally decides to settle
that it's a bicycle.

Obviously way too late
to actually do anything about it.

Terrifying.

You design a system that fails
to track the path of an object

until it's decided what it is?

If you've got something that
you're going to collide with,

I don't care what it is! I care
about that it's going to collide.

That's insane.

"Although the system sensed
the pedestrian nearly six seconds

"before the impact, the system
never classified her as a pedestrian

"or predicted correctly her goal.

"The system design did not include

"a consideration
for jaywalking pedestrians."

It wasn't designed
to recognise people

unless they were on a crosswalk.

I think this is
pretty damning, actually.

What was this thing doing
on the roads?

The report made me want to know more
about what was happening

inside Uber around the time
of Rafaela's crash in 2018.

After some digging,
I found Robbie Miller...

Robbie, hi!

..an operations manager
who'd been at Uber

at the same time as Rafaela.

He had quit on safety grounds

just a few days before
the fatal collision.

I'd been working
in the self-driving space

for four or five years
at this point.

I was running the self-driving
truck fleet for Uber.

Eventually, there was a move
to combine the operations

of the testing for self-driving cars
and self-driving trucks.

How did you feel about that?

I was not at all comfortable
with it.

The self-driving cars
were having significant issues.

There's a lot of really scary
incidences that are occurring -

near misses, near collisions.

We would have, you know,
an incident

where the car was driving on
the sidewalk...in broad daylight.

And you realise, like, this...

They are headed on a path

where someone is going to get
seriously injured or worse.

And so I...

I gave notice.

There's this overarching fear,
I would say, at Uber,

that Waymo is about to release
their self-driving cars.

And it's...

I think it's very scary for Uber's
leadership to not have a response.

So it's turned into a race?
It is absolutely a race.

Waymo, owned by Google's
parent company Alphabet,

was Uber's arch-rival.

Just five months after Waymo
launched its first public trial,

Uber moved from two operators
in the car to one.

You need to show to your investors,
"Hey, we're making this progress."

An easy way to do that
is just take someone out of the car.

And this is something I mentioned -

"You need that second person
in the vehicle.

"You're not ready to take
that person out of the vehicle."

Just a few days after I left,
the crash occurred.

And... Where did you first hear
about it?

I was driving

and I received a phone call from
one of my former co-workers at Uber.

And I sat in the parking lot
and cried for 15 minutes.

Wow.

I am very passionate about
the technology.

I believe in the technology.
I want the technology to succeed.

But there's...

There's a way to do it.

A few years after
Rafaela's fatal crash,

Uber sold off
its self-drive division,

leaving the path clear for Waymo.

Here you go! Waymo's just crashed.

AI technology - not working!

Why is this happening to me
on a Monday? I'm in a Waymo car...

AUTOMATED VOICE: Connected to
Rider Support.

..and... This call may be recorded
for quality assurance.

..this car is just going in circles.

Hi there, Mike. This is Gab.
I'm calling from Waymo support...

Yeah, I got a flight to catch.

Why is this thing going in a circle?
I'm getting dizzy.

Waymo are now the dominant force
in driverless taxis in the US,

with 2,500 of them on the road.

Experts have praised
their safety record.

But some see these cars as symbols
of the powerful tech elite

and their disconnection from
the lives of ordinary people.

Hey! Hello.

And there's one group
who've decided to take action.

Nice to meet you. It's nice
to meet you. How are you all doing?

Doing well. Doing well.

They call themselves
the Safe Street Rebels

and carry out
their operations incognito.

What is it about the autonomous
driving that you dislike, though?

They are heavily
promoting themselves

as the future of public transit,

and we just fundamentally
don't think

they're the future of
public transit.

They're just a taxi
where you don't talk to someone.

And when that car is not parked,
it's still driving around, waiting.

50% of the miles they drive,
there's nobody in the vehicle.

In addition, they cannot be ticketed

for any kind of moving violation
in the city.

We have videos of them driving 40mph

on the wrong side of the road,
and nobody can do anything about it.

They're completely immune.

What, they're above
the kind of rules that would stand

for an Uber driver or a Lyft driver?

Yes, and if they can't do something
about it, we can.

To express their frustration,

the group go around San Francisco
disabling Waymos.

Let me understand
the strategy, then.

So how easy are they to override?

LAUGHS: Pretty easy!
Definitely pretty easy.

We're not allowed to show you
how they disable these cars,

although it's not exactly high-tech.

So you're not putting them
out of action permanently? No, no.

And we're not causing any damage
to them either.

It's like... It's painter's tape,

so it doesn't leave any residue
when you remove it.

There's one...
There's one coming. Right.

There's one, there's one...
Yeah. Check this one.

Someone go in front.

MAN: Fuck it up!
Fuck it up!

CHEERING

Did you hear that?

There was people on the sidewalk
who were, like, cheering them on.

The disabled car was now blocking
the road.

CHIME
AUTOMATED VOICE: Please step back.

So the one behind hasn't been taped
but is stuck.

There's another one coming.
Oh, my gosh!

Three of them...

..and they're just stuck.

And now there's another car
behind, flashing.

But it has a human driver,
so they can go around the problem.

Is it actually illegal,
what you're doing?

We don't think it's illegal.

We can't find any law
that it breaks.

It's not vandalism, so... It's hard
to point any law we're breaking.

Are there not other ways that you
could do this? I mean, can you...

I don't know, is there, like,
not a more sort of democratic way?

It would be nice if we could vote,

if we had the opportunity
to vote on them,

but we have not been presented
with that opportunity yet.

Is this... Does it feel a bit like
this is something

that has happened to you
rather than with you?

I mean, absolutely.

Hey, yo, there's one coming
the other side.

Other side, other side, other side.
Other side! It's coming. Yeah.

Yeah. You want to get in front?

They are surprisingly easy to bully,
aren't they?

The other thing that
I think was really surprising

was just how many of them
were going past, right?

And maybe it's the time of day,
but, I mean...

..hardly any of them
have people inside.

Yeah, it's empty.

They did say something
that I hadn't considered before,

which is about how they are
continually circling

when they don't have
people inside them.

And that I hadn't considered,
because of course,

there's a congestion aspect...

..but there's an
environmental aspect, too.

I mean, they've gotta be powered.

Hmm. Interesting.

Interesting.

It happened fast.

The hearing for
Rafaela Vasquez was scheduled

as a settlement conference,

but it quickly led to a plea deal.

In July 2023, after five years
of legal wrangling,

Rafaela accepted a plea deal
to avoid going to jail.

The agreement indicates
that you wish to plead guilty to

the crime of endangerment.

The offence is non-dangerous,
non-repetitive.

Is that the crime that you wish
to plead guilty to? Yes.

She pled guilty to
a reduced charge - endangerment -

with the guarantee it would be
lowered to the least-serious

category of offence, following
three years of probation.

Ah! Hey, how are you?
Look who it is. Hi.

I wanted to meet Rafaela's lawyers,
Al Morrison....

How are you?
..and Marci Kratter.

So good to see you.

Why did you take this case on?

Our job is to fight for
the little guy - no offence -

but she was the little guy.

And the more we got into this case,

the more we realised just
how one-sided it was.

Where do you think this came from,
then? I mean, is this...

Is this the police,
or is this from Uber?

Uber. Really?

They did whatever they could
to make it not their fault.

And it's David and Goliath. Yeah.

We have this single person

up against a multimillion-dollar
company with unlimited resources.

The problem was that there
were so many aspects of the way

these cars were programmed

that didn't account for
real-world situations.

The most obvious one is,
the thing wasn't programmed

to deal with jaywalkers.

To save yourself in a campus,
college campus,

to not programme vehicles
to deal with jaywalking

I think is the height of negligence.

I was scared to go to trial.

If I lose, it's prison time,
no ifs, ands or buts.

And the media already
destroyed me out there.

That's always in the back
of our minds as lawyers,

and certainly our clients' minds,
that it's the exposure.

"What's going to happen to me
if I go to trial and lose?"

But Marci and I felt like
we had a great case,

but we also understood
the risk was all hers.

Me not going to trial has
no reflection on them.

These two saved my life.

They absolutely saved my life.

No criminal charges were
ever brought against Uber.

Rafaela got dealt
a really rough hand,

and this didn't play out fairly.

Maybe she was watching her phone.

Maybe she wasn't. Like, I...

Honestly, I don't know,
but I also...

I don't think that
that's the point of this.

The scandal here is that you have
got this massive corporate,

multibillion-dollar project
which is not prioritising

the safety of the people who are
in the cars, or, like,

the members of the public
who haven't even agreed

to participate in the experiment.

It is not enough to say that all of
the responsibility lies with

the person who was in the car.

That's not enough.
That's not enough for me.

Back in Miami, Dillon and Naibel's
family were taking Tesla to court

over the fatal crash
that had killed Naibel.

Nice to meet you. I'm Hannah. Adam.
Great to see you. Great to meet you.

What's up, buddy?

SHE CHUCKLES

Dillon's lawyer, Adam Boumel,

brought me up to speed on the case.

This is a sort of situation
where there's no precedent at all.

Does that make it quite difficult
to build a legal case when you're,

I don't know, forging a new path?
Extremely. Yeah?

I mean, we had to do it
from the start.

This is creating the law.

Obviously, from day one,
Tesla's position was,

this is 100% driver's fault.

The thing is that it's difficult
to untangle the responsibility

in this case, because you're not
saying that the driver had

no responsibility at all.
No, of course not.

We 100% acknowledge that
the driver had responsibility.

And our position from day one
has been that this is a case

of shared responsibility.

Yes, the driver was at fault.

He was distracted, he was
disengaged from the driving task.

But then you have to ask yourself,

why was he disengaged from
the driving task?

And you realise that Tesla
fostered this belief in him,

this trust of the system
that was unwarranted.

He thought that the car would stop
or swerve or do something

before ploughing into
a parked vehicle.

Do you have the data from
inside the car?

Do you know what the car was seeing?

So, we were able to finally get
the data from inside the car,

showing what the Autopilot computer
was detecting and processing.

The computer understood that
the car was speeding

towards the end of the road.

It appreciated the stop sign.

It appreciated
the blinking red light.

It appreciated Dillon's parked SUV.

So, it identified a lot of things,
and it did nothing.

The car knew what was going on
in front of it,

and didn't do nothing to warn
the driver to stop the car.

So, this isn't a case
of misinformation

inside the car's system.

The car has all of the information
it needed in order

to avoid the collision.

But it was programmed not to behave
in the way that the driver thought

it was programmed to behave.

Did the car behave as people
expected and believed it should,

based off of Tesla's marketing
and Elon Musk's statements,

in kind of hyping up
the capabilities of this car?

When you take that backdrop
and you put it against this case,

the car did nothing that anybody
thought it would or should.

So, this has almost become
a case of, like,

misadvertising or, like,
misrepresentation, then?

They make the consumers and
drivers believe that

the car is more capable,

creating false expectations
in their drivers.

In the Uber crash,
the blame had been laid

firmly on Rafaela's shoulders.

With Tesla, would this now be
the first time the car itself

would be considered at fault?

Tonight, a Florida jury forcing
Tesla to pay $243 million

to victims in a deadly 2019 crash,

the jury finding flaws
in Tesla's self-driving software

were partly to blame.

We can be proud that we stood up,

and that we did everything in
our power to help shine light

on what's going on.

Against all odds,
Dillon emerged victorious in court,

forcing Tesla to take
some responsibility -

for the first time ever -

for a crash involving
its Autopilot system.

It was a shocking landmark verdict

that could well reshape
the future of self-driving cars.

When they gave the verdict,
how was it?

It was very emotional, you know?

I mean, like, me and my dad,
we started hugging each other and,

you know, praying...
that we would get justice.

You know, from the beginning,
we knew this was

a joint liability case,

and the jury decided to hold
Tesla accountable.

And I'm grateful for that.

And I'm grateful that people
heard all the evidence,

and saw that Tesla made a mistake.

I really get the impression
that this has never been

about money for you.

It was more important for us
to shine light,

and to show the world that
this technology is not safe.

People will come to me and,
you know, tell me congratulations.

But I don't feel like
it's something to celebrate.

Like... You're not walking away
from this a winner.

I would say it's more just...
justice was served. Yeah.

Tesla plan to appeal the ruling.

Meanwhile, in the UK,

British firm Wayve will launch
their self-driving cars

in London later this year.

This feels like so much more
high stakes than it did

when I was in Phoenix.

I mean, this is slightly
wild for me, right?

Like, I've lived in London
for 20 years,

and we're driving in a driverless
car down Camden High Street.

THEY LAUGH

Like, this is...

This is honestly something I thought
was much further in the future.

It never gets old.

Wayve CEO Alex Kendall points out
that the AI in driverless cars

has come on leaps and bounds
in the past few years.

And this is interesting.
This person's body language was

sort of turning backwards
and forwards away from the crossing.

So, that was noticeable, actually.

The car sort of changed
its mind a couple of times.

It saw the person turn their
body back and forth.

The AI has got quite good
at learning that kind of intent.

So, it's like super,
super fancy cruise control.

It's a completely different
experience from cruise control.

Have I just undercut
your product there?

You're comparing a floppy disk
with a quantum computer.

Think of AI as
the next evolution of tooling.

When you think about the wheel,
the calculator, the computer -

different tools that, as humanity,

we've invented to push
forward society... Mm-hm.

..I think intelligent machines
will be that next evolution of this.

Whether you like it or not,
driverless cars are coming.

I mean, they're here already.

And I still think there's lots
of good to come from that.

But to get to this point, we had
to go through that difficult phase.

We had to go through the learning -
which, I mean, by definition,

learning involves making mistakes.

It's just that these mistakes,
you know, these deaths,

these lives that were ruined,

I don't think that
they were inevitable.

I don't think that they were
an acceptable price to pay

for a new technology.

I think different choices
could have been made.

I think there were different ways
to balance the roll-out of

the new product with
the safety of the public.

And I just hope now that
these are lessons from the past,

you know, I hope that the steepest
part of this learning curve

is now behind us.

SIRENS WAIL

The chief executive of one
of the United States's largest

health insurance companies has been
shot and killed in New York.

The alleged killer has been
identified as Luigi Mangione.

The suspect had something to say
about the insurance industry.

The insurance companies are relying
on algorithms to make decisions

to deny patient care.

Stop denials with AI!

ALL: How many people have to die?

The anger and the upset is real.

ALL: Health care is a human right!

To discover more about AI and how it
can shape our future, go to...

..or scan the QR code on
the screen now.