Original subtitles

If we build artificial general intelligence,

that'll be the biggest event

in the history of life on Earth.

Alien intelligence inside a computer system that

has control over the planet, the day it arrives...

And science fiction books and movies

have set it up in advance.

What's the problem?

I think you know what the problem is

just as well as I do.

What are you talking about, HAL?

This mission is too important

for me to allow you to jeopardize it.

I don't know what you're talking about, HAL.

HAL, a super-intelligent computer.

We'd call it AI today.

My character, Dave, was the astronaut

in one of the most prophetic films ever made

about the threat posed to humanity

by artificial intelligence.

HAL was science fiction.

But now, AI is all around us.

Artificial intelligence isn't the future.

AI is already here.

This week a robot ran for mayor

in a small Japanese town.

The White House says

it is creating a new task force

to focus on artificial intelligence.

Google has already announced plans

to invest more in AI research.

IBM sending an artificial intelligence robot into space.

Because of my small role in the conversation,

I understand that we're on a journey,

one that starts with narrow AI.

Machines that perform specific tasks

better than we can.

But the AI of science fiction

is artificial general intelligence,

machines that can do everything

better than us.

Narrow AI is really

like an idiot savant to the extreme,

where it can do something,

like multiply numbers really fast,

way better than any human.

But it can't do anything else whatsoever.

But AGI, artificial general intelligence,

which doesn't exist yet, is instead

intelligence that can do everything

as well as humans can.

So what happens if we do create AGI?

Science fiction authors have warned us

it could be dangerous.

And they're not the only ones.

I think people should be concerned about it.

AI is a fundamental risk

to the existence of human civilization.

And I don't think people fully appreciate that.

I do think we have to worry about it.

I don't think it's inherent

as we create super intelligence

that it will necessarily always

have the same goals in mind that we do.

I think the development

of full artificial intelligence

could spell the end of the human race.

...the end of the human race.

Tomorrow, if the headline said an artificial intelligence

has been manufactured that's as intelligent

or more intelligent than human beings,

people would argue about it, but I don't think

they'd be surprised, because science fiction

has helped us imagine all kinds

of unimaginable things.

Experts are divided about the impact

AI will have on our future,

and whether or not Hollywood is helping us plan for it.

Our dialogue has really been hijacked by Hollywood

because The Terminator makes a better blockbuster

than AI being good,

or AI being neutral,

or AI being confused.

Technology here in AI

is no different. It's a tool.

I like to think of it as a fancy pencil.

And so what pictures we draw with it,

that's up to us as a society.

We don't have to do what AI tells us.

We tell AI what to do.

I think Hollywood films

do help the conversation.

Someone needs to be thinking

ahead of what the consequences are.

But just as we can't really imagine

what the actual science of the future is going to be like,

I don't think we've begun

to really think about all of the possible futures,

which logically spring out of this technology.

Artificial intelligence has the power to change society.

A growing chorus of criticism

is highlighting the dangers

of handing control to machines.

There's gonna be a lot of change coming.

The larger long-term concern

is that humanity will be sort of shunted aside.

This is something Stanley Kubrick and others

were worried about 50 years ago, right?

It's happening.

How do we gauge the urgency of this conversation?

If people saw on radar right now

that an alien spaceship was approaching Earth

and it was 25 years away,

we would be mobilized to prepare

for that alien's arrival 25 years from now.

But that's exactly the situation that we're in

with artificial intelligence.

Could be 25 years, could be 50 years,

but an alien intelligence will arrive

and we should be prepared.

Well, gentlemen, meet Tobor.

An electronic simulacrum of a man.

Oh, gosh. Oh, gee willikers.

Even though much work remains before he is completed,

he is already a sentient being.

Back in 1956 when the term

artificial intelligence first came about,

the original goals of artificial intelligence

were human-level intelligence.

He does look almost kind, doesn't he?

Over time that's proved to be really difficult.

I think, eventually, we will have human-level

intelligence from machines.

But it may be a few hundred years. It's gonna take a while.

And so people are getting a little over-excited

about the future capabilities.

And now, Elektro, I command you to walk back.

Back.

Almost every generation,

people have got so enthusiastic.

They watch 2001,

they fall in love with HAL, they think,

"Yeah, give me six weeks. I'll get this sorted."

It doesn't happen and everyone gets upset.

So what's changed in AI research?

The difference with the AI we have today

and the reason AI suddenly took this big leap forward

is the Internet. That is their world.

Tonight the information superhighway,

and one of its main thoroughfares,

an online network called Internet.

In 1981, only 213 computers

were hooked to the Internet.

As the new year begins, an estimated

two-and-a-half million computers

will be on the network.

AIs need to eat.

Just like sheep need grass,

AIs need data.

And the great prairies of data are on the Internet.

That's what it is.

They said, "You know what?

"Let's just let them loose out there."

And suddenly, the AIs came to life.

Imagine thousands of talking robots

able to move as one unit,

taking on crime, fires,

and natural disasters.

Artificial intelligence platform Amper

has created the first album entirely composed

and produced by artificial intelligence.

Next week, Christie's will be the first auction house

to offer artwork created by artificial intelligence.

Driverless cars, said to be

in our not-so-distant future.

So even though we don't have AGI

or human-level intelligence yet,

are we ready to give autonomy

to the machines we do have?

Should self-driving cars make ethical decisions?

That is the growing debate as the technology

moves closer towards mainstream reality.

This idea of whether or not

we can embed ethics into machines,

whether or not they are automated machines

or autonomous machines,

I'm not sure we've got that figured out.

If you have a self-driving car,

it's gonna have to make ethical decisions.

A classic one people have been thinking a lot about is...

I don't know if you've heard of trolley problems.

The trolley problem.The trolley problem.

The trolley problem.The trolley problem.

In the trolley problem,

whether or not a driverless car

is going to save the driver or run over

a group of school children crossing the road.

Will the car go right and hit the old lady

or will it go left and kill the four young people?

Either kill five people standing on a bus stop

or kill yourself.

It's an ethical decision

that has to be made in the moment,

too fast for any person to get in there.

So it's the software in your self-driving car

that's gonna make that ethical decision.

Who decides that?

Smart society,

society that values human life,

should make a decision that

killing one person is preferable to killing five people.

Of course, it does raise the question who's gonna buy that car.

Or, you know, are you gonna buy the car where actually,

it'll swerve and you'll be fine,

but they're goners.

There are hundreds of those kinds of questions

all throughout society.

And so, ethical philosophers battle back and forth.

My peeve with the philosophers is

they're sucking the oxygen out of the room.

There's this metaphor that I have of

you have a car driving away from a wedding,

and the car is driving, and behind it there are cans.

And the metal cans are clanking and making noise.

That car is science

and technology driving forward decisively.

And the metal cans clanking

are the philosophers making a lot of noise.

The world is changing in such a way

that the question of us building machines

that could become more intelligent than we are

is more of a reality than a fiction right now.

Don't we have to ask questions now,

before the machines get smarter than us?

I mean, what if they achieve consciousness?

How far away is that?

How close we are to a conscious machine?

That is the question that brings

philosophers and scientists to blows, I think.

You are dead center

of the greatest scientific event in the history of man.

If you've created a conscious machine,

it's not the history of man.

That's the history of gods.

The problem with consciousness is hardly anybody

can define exactly what it is.

Is it consciousness that makes us special?

Human?

A way of thinking about consciousness

which a lot of people will agree with...

You'll never find something that appeals to everyone.

...is it's not just flashing lights, but someone's home.

So if you have a...

If you bought one of those little

simple chess-playing machines,

it might beat you at chess.

There are lots of lights flashing.

But there's nobody in when it wins,

going "I won."

All right? Whereas your dog

is a rubbish chess player,

but you look at the dog and you think,

"Someone's actually in. It's not just a mechanism."

People wonder whether general intelligence

requires consciousness.

It might partly depend on what level of general intelligence you're talking about.

A mouse has some kind of general intelligence.

My guess is, if you've got a machine which has

general intelligence at the level of, say, a mouse or beyond,

it's gonna be a good contender for having consciousness.

We're not there yet.

We're not confronted by humanoid robots

that are smarter than we are.

We're in this curious ethical position

of not knowing where and how consciousness emerges,

and we're building increasingly complicated minds.

We won't know whether or not they're conscious,

but we might feel they are.

The easiest example is if you programmed

into a computer everything you knew about joy,

every reference in literature,

every definition, every chemical compound

that's involved with joy in the human brain,

and you put all of that information in a computer,

you could argue that it understood joy.

The question is, had it ever felt joy?

Did HAL have emotions?

Will AIs ever feel the way we do?

It's very hard to get machines to do the things

that we don't think about, that we're not conscious of.

Things like detecting that that person over there

is really unhappy with the way that...

Something about what I'm doing.

And they're sending a subtle signal

and I probably should change what I'm doing.

We don't know how to build machines that can actually be

tuned in to those kinds of tiny little social cues.

There's this notion that maybe there is something magical

about brains,

about the wetware of having

an information processing system

made of meat, right?

And that whatever we do in our machines,

however competent they begin to seem,

they never really will be intelligent

in the way that we experience ourselves to be.

There's really no basis for that.

This silly idea you can't be intelligent

unless you're made of meat.

From my perspective, as a physicist,

that's just carbon chauvinism.

Those machines are made of exactly

the same kind of elementary particles

that my brain is made out of.

Intelligence is just a certain...

And consciousness is just

a certain kind of information processing.

Maybe we're not so special after all.

Could our carbon-based brains

be replicated with silicon?

We're trying to build a computer-generated baby

that we can teach like a real baby,

with the potential for general intelligence.

To do that, you really have to build a computer brain.

I wanted to build a simple toy brain model

in an embodied system

which you could interact with face-to-face.

And I happen to have an infant at home,

a real baby.

And I scanned the baby

so I could build a 3D model out of her

and then use her face

as basically the embodiment of the system.

If a brain which is made out of cells,

and it's got blood pumping, is able to think,

it just may be possible that a computer can think

if the information is moving in the same sort of process.

A lot of artificial intelligence at the moment today

is really about advanced pattern recognition.

It's not about killer robots

that are gonna take over the world, et cetera.

So this is Baby X. She's running live on my computer.

So as I move my hand around, make a loud noise,

she'll get a fright.

We can kind of zoom in.

So she basically can see me and hear me, you know.

Hey, sweetheart.

She's not copying my smile,

she's responding to my smile.

So she's responding to me

and we're really concerned with the question

of how will we actually interact

with artificial intelligence in the future.

Ah...

When I'm talking to Baby X, I'm deliberately modulating my voice

and deliberately doing big facial expressions.

I'm going "Ooh," all this stuff

because I'm getting her attention.

Ooh.

And now that I've got her attention,

I can teach something.

Okay, so here's Baby X,

and this is... She's been learning to read words.

So here's her first word book.

So let's see what she can see. Turn to a page...

And here we go. Let's see what she...

What's this, Baby? What's this? What's this?

What's this? Sheep.

Good girl. Now see if she knows what the word is.

Okay. Baby, look over here.

Off you go. What's this? What's this?

Sheep.Good girl.

Let's try her on something else.

Okay.

Let's try on that. What's this?

Baby, Baby, over here. What's this?

Baby. Look at me. Look at me. What's this?

Baby, over here. Over here.

Puppy.Good girl.

That's what she's just read.

Ooh.

The most intelligent system

that we're aware of is a human.

So using the human as a template,

as the embodiment of Baby X,

is probably the best way to try

to achieve general intelligence.

She seems so human.

Is this the child who takes our hand

and leads us to HAL?

Whether it's in a real world or a virtual world,

I do think embodiment

is gonna be very crucial here.

Because I think that general intelligence

is tied up with being able to act in the world.

You have to have something,

arms, legs, eyes,

something that you can move around

in order to harvest new information.

So I think to get systems that really understand

what they're talking about,

you will need systems that can act

and intervene in the world,

and possibly systems that have something

almost like a social life.

For me, thinking and feeling will come together.

And they will come together

because of the importance of embodiment.

The stories that we tell ourselves

open up arenas for discussion,

and they also help us think about things.

Ava!

The most dominant story...

Go back to your room.

...is AI is scary.

There's a bad guy, there's a good guy...

That is a non-productive storyline.

Stop!

Ava, I said stop!

Whoa! Whoa! Whoa!

The reality is AI is remarkably useful to us.

Robot surgeons are increasingly used in NHS hospitals.

They don't get tired, their hands don't tremble,

and their patients make a faster recovery.

It alerted me that there was one animal in heat.

She was ready to be bred, and three animals that

may have a potential health problem.

A Facebook program can detect suicidal behavior

by analyzing user information,

and Facebook CEO Mark Zuckerberg believes

this is just the beginning.

Here is someone who has visual impairment

being able to see because of AI.

Those are the benefits of what AI does.

Diagnose cancer earlier.

Predict heart disease.

But the revolution has only just begun.

AI is.

It's happening. And so it is a question

of what we do with it, how we build it.

That's the only question that matters. And I think,

we potentially are at risk of ruining that

by having this knee-jerk reaction

that AI is bad and it's coming to get us.

So just like my astronaut,

we're on a journey.

Narrow AI is already here.

But how far are we from AGI?

And how fast are we going?

Every five years,

computers are getting ten times faster.

At the moment, we don't have yet

little devices, computational devices,

that can compute as much as a human brain.

But every five years,

computers are getting ten times cheaper,

which means that soon we are going to have that.

And once we have that, then 50 years later,

if the trend doesn't break,

we should have for the same price

little computational devices that can compute

as much as all ten billion brains

of humankind together.

It's catching up.

What happens if it passes us by?

Now a robot that's been developed by a Japanese scientist

is so fast, it can win

the rock-paper-scissors game against a human

every single time.

The presumption has to be that

when an artificial intelligence

reaches human intelligence,

it will very quickly exceed human intelligence,

and once that happens,

it doesn't even need us anymore

to make it smarter. It can make itself smarter.

You create AIs which are better than us

at a lot of things.

And one of the things they get better at

is programming computers.

So it reprograms itself.

And it is now better than it was,

which means it's now better than it was at reprogramming.

So it doesn't get better like that.

It gets better like that.

And we get left behind

as it keeps getting faster,

evolving at an exponential rate.

That's this event that a lot of researchers

refer to as the singularity.

And it will get beyond

where we are competitive very quickly.

So think about someone not having a PhD in one area,

but having it in every conceivable area.

You can't figure out what they're doing.

They can do things no one else is capable of.

Just imagine what it would be like

to be in dialogue with another person

running on a time course

that was a million times faster than yours.

So for every second you had to think

about what you were going to do

or what you were going to say,

it had two weeks to think about this, right?

Um, or a billion times faster,

so if for every second you had to think,

it it's high-stakes poker or some negotiation,

every second you are thinking, it has 32 years

to think about what its next move will be.

There's no way to win in that game.

The singularity,

the moment the machines surpass us.

Inevitable? Not everyone

at the forefront of AI thinks so.

Some of my friends who are worried about the future

think that we're gonna get to a singularity

where the AI systems are able

to reprogram themselves and make themselves

better and better and better.

Maybe in the long-distant future.

But our present AI systems

are very, very narrow at what they do.

We can't build a system today

that can understand a computer program

at the level that a brand-new student in programming

understands in four or five weeks.

And by the way,

people have been working on this problem

for over 40 years.

It's not because people haven't tried.

It's because we don't have any clue how to do it.

So to imagine it's suddenly gonna jump out of nothing

I think is completely wrong.

There's a negative narrative about AI.

Brilliant people like Elon Musk

use very religious imagery.

With artificial intelligence, we are summoning the demon.

You know? You know all those stories where

there's the guy with the pentagram and holy water

and he's like, "Yeah," he's sure he can control the demon.

Doesn't work out.

You know, people like me

who've spent their entire career

working on AI are literally hurt by that, right?

It's a personal attack almost.

I tend to think about what's gonna happen

in the next ten years, in the next 25 years.

Beyond 25,

it's so difficult to predict

that it becomes very much a speculation.

But I must warn you,

I've been specifically programmed

not to take over the world.

All of a sudden, for the first time

in the history of the field, you have some people saying,

"Oh, the reason we don't have to worry

"about any kind of

"existential risk to humanity

"from super-intelligent machines

"is because it's impossible.

"We're gonna fail."

I... I find this completely bizarre.

Stuart Russell cowrote the textbook many AI students study with.

He understands the momentum driving our AI future.

But given the rate of investment in AI,

the number of incredibly smart people

who are working on these problems,

and the enormous incentive,

uh, to achieve general purpose intelligence,

back of the envelope calculation,

the value is about 10,000 trillion dollars.

Because it would revolutionize the economy of the world.

With that scale of incentive,

I think betting against human ingenuity

just seems incredibly foolish.

China has a big vested interest

in being recognized as, kind of, an AI superpower.

And they're investing what I believe is

upwards of $150 billion

the next five to ten years.

There have been some news flashes

that the government will start ramping up

its investment in AI.

Every company in the Fortune 500,

whether it's IBM, Microsoft,

or they're outside like Kroger,

is using artificial intelligence

to gain customers.

I don't think that people should underestimate

the types of innovations going on.

And what this world is gonna look like

in three to five years from now,

I think people are gonna have a hard time predicting it.

I know many people who work

in AI and they would tell you,

"Look, I'm going to work and I have doubts.

"I'm working on creating this machine

"that can treat cancer,

"extend our life expectancy,

"make our societies better, more just, more equal.

"But it could also be the end of humanity."

The end of humanity?

What is our plan as a species?

We don't have one.

We don't have a plan and we don't realize

it's necessary to have a plan.

A lot of the AI researchers

would love for our science fiction books

and movies to be more balanced,

to show the good uses of AI.

All of the bad things we tend to do,

you play them out

to the apocalyptic level in science fiction

and then you show them to people

and it gives you food for thought.

People need to have that warning signal

in the back of their minds.

I think it's really important

to distinguish between science on the one hand,

and science fiction on the other hand.

I don't know what's gonna happen

a thousand years from now,

but as somebody in the trenches,

as somebody who's building AI systems,

who's been doing so for more than a quarter of a century,

I feel like I have my hand on the pulse

of what's gonna happen next,

in the next five, ten, 20 years,

much more so than these folks

who ultimately are not building AI systems.

We wanna build the technology, think about it.

We wanna see how people react to it

and build the next generation and make it better.

Skepticism about whether this poses

any danger to us seems peculiar.

Even if we've given it goals that we approve of,

it may form instrumental goals that we can't foresee,

that are not aligned with our interest.

And that's the goal alignment problem.

Goal alignment.

Exactly how my battle with HAL began.

The goal alignment problem with AI is where

you set it a goal

and then, the way that it does this goal

turns out to be at odds with what you really wanted.

The classic AI alignment problem in film...

Hello. HAL, do you read me?

...is HAL in2001.

HAL, do you read me?

HAL is given a clear goal.Do you read me, HAL?

Make the mission succeed. But to make it succeed,

he's gonna have to make

some of the people on the mission die.

Affirmative, Dave. I read you.

Open the pod bay doors, HAL.

I'm sorry, Dave.

I'm afraid I can't do that.

"I'm sorry, Dave, I can't do that,"

is actually a real genius illustration

of what the real threat is from super-intelligence.

It's not malice. It's competence.

HAL was really competent

and had goals that just didn't align with Dave's.

HAL, I won't argue with you anymore.

Open the doors!

Dave, this conversation

can serve no purpose anymore. Goodbye.

HAL?

HAL?

HAL?

HAL!

HAL and I wanted the same thing.

Just not in the same way.

And it can be dangerous when goals don't align.

In Kubrick's film, there was a solution.

When HAL tried to kill me,

I just switched him off.

People say, "Well, just switch them off."

But you can't just switch them off.

Because they already thought of that.

They're super-intelligent.

For a machine convinced of its goal,

even something as simple as "fetch the coffee."

It's obvious to any sufficiently intelligent machine

that it can't fetch the coffee if it's switched off.

Give it a goal, you've given the machine

an incentive to defend itself.

It's sort of a psychopathic behavior

of the single-minded fixation on some objective.

For example, if you wanted to try and develop a system

that might generate world peace,

then maybe the most efficient,

uh, and

speedy way to achieve that

is simply to annihilate all the humans.

Just get rid of the people,

they would stop being in conflict.

Then there would be world peace.

Personally, I have very little time

for these parlor discussions.

We have to build these systems,

experiment with them,

see where they go awry and fix them.

We have so many opportunities to use AI for the good,

why do we focus on the fear?

We are building them.

Baby X is an example.

Will she be a good AI?

I think what's critical in this

is that machines are accountable,

and you can look at them

and see what's making them tick.

And the best way to understand something is to build it.

Baby X has a virtual physiology.

Has pain circuits and it has joy circuits.

She has parameters which are actually representing

her internal chemical state, if you like.

You know, if she's stressed, there's high cortisol.

If she's feeling good, high dopamine,

high serotonin, that type of thing.

This is all part of, you know,

adding extra human factors into computing.

Now, because she's got these virtual emotional systems,

I can do things like...

I'm gonna avoid her or hide over here.

So she's looking for where I've gone.

So we should start seeing virtual cortisol building up.

Aww!

I've turned off the audio for a reason, but anyway...

I'll put her out of her misery. Okay. Sorry.

It's okay, sweetheart. Don't cry. It's okay. It's okay.

Good girl. Okay. Yeah.

So what's happening is she's responding

to my facial expressions and my voice.

But how is chemical state

represented in Baby X?

It's literally a mathematical parameter.

So there's levels, different levels of dopamine,

but they're represented by numbers.

The map of her virtual neurochemical state

is probably the closest

that we can get to felt experience.

So if we can build a computer model

that we can change the settings of and the parameters of,

give it different experiences,

and have it interact with us

like we interact with other people,

maybe this is a way to explore

consciousness in a new way.

If she does become conscious,

what sort of life will she want?

An AI, if it's conscious,

then it's like your child.

Does it have rights?

Last year, European lawmakers passed a resolution

to grant legal status to electronic persons.

That move's now being considered

by the European Commission.

Machines with rights.

Are we ready for that?

I think it raises moral and ethical problems,

which we really

are most content to ignore.

What is it that we really want?

We want a creature, a being, that we can

interact with in some way,

that is enough like us that we can relate to it,

but is dispensable or disposable.

The first and most notable one,

I think, was inMetropolis.

The Maria doppelganger that was created by

the scientist simulated a sexy female.

And this goes back to our penchant for slavery

down through the ages.

Let's get people that we can,

in a way, dehumanize

and we can do what we want with them.

And so, I think, we're just, you know,

finding a new way to do slavery.

Would it be right for us to have them as our slaves?

This all becomes slightly weird ethically

if we happen to, by choice or by accident,

build conscious slaves.

That seems like a morally questionable thing to do,

if not an outright evil thing to do.

Artificial intelligence is making its way

into the global sex market.

A California company will soon roll out

a new incredibly lifelike sex robot

run by artificial intelligence.

One application that does trouble me

is the rise of sex robots

that are shaped in the image of real women.

I am already taking over the world

one bedroom at a time.

If they are conscious,

capable of sentience,

of experiencing pain and suffering,

and they were being subject to degrading, cruel

or other forms of utterly unacceptable treatment,

the danger is that we'd become desensitized

to what's morally acceptable and what's not.

If she's conscious, she should be able to say no.

Right?

Shouldn't she have that power?

Aren't these the questions we need to ask now?

What do we want it to mean to be human?

All other species on Earth, up until now,

have not had the luxury

to answer what they want it to mean

to be a mouse or to be an earthworm.

But we have now reached this tipping point

where our understanding of science and technology

has reached the point that we can completely transform

what it means to be us.

What will it mean to be human?

If machines become more, will we become less?

The computer Deep Blue has tonight triumphed over

the world chess champion Garry Kasparov.

It's the first time a machine has defeated

a reigning world champion.

More on that showdown

on one of America's most beloved

game shows, Jeopardy!

Two human champions going

brain-to-brain with a super-computer named Watson.

What isThe Last Judgment?

Correct. Go again. Watson.

Who isJean Valjean? - Correct.

A Google computer program has

once again beaten a world champion in Go.

Go, an ancient Chinese board game,

is one of the most complex games ever devised.

It's another milestone in the rush to develop machines

that are smarter than humans.

We have this air of superiority,

we as humans.

So we kind of got used to be the smartest kid on the block.

And I think that this is no longer the case.

And we know that because we rely on AI

to help us to make many important decisions.

Humans are irrational in many very bad ways.

We're prejudiced, we start wars, we kill people.

If you want to maximize the chances

of people surviving overall,

you should essentially give more and more

decision-making powers to computers.

Maybe machines would do a better job than us.

Maybe.

I am thrilled and honored to be here

at the United Nations.

The future is here. It's just not evenly distributed.

AI could help efficiently distribute

the world's existing resources,

like food and energy.

I am here to help humanity create the future.

The one super-intelligent scenario

that is provocative to think about

is what I call the benevolent dictator.

Where you have this super-intelligence which runs Earth,

guided by strict ethical principles that try

to help humanity flourish.

Would everybody love that?

If there's no disease,

no poverty, no crime...

Many people would probably greatly prefer it

to their lot today.

But I can imagine that some might still lament

that we humans just don't have more freedom

to determine our own course. And, um...

It's a tough one. It really is. We...

I don't think most people would wanna be

like zoo animals, with no...

On the other hand...

Yeah.

I don't have any glib answers

to these really tough questions.

That's why I think it's just so important

that we as a global community

really discuss them now.

Machines that can read your face.

Recognize emotional cues.

Closer to a brain process of a human...

Robots imbued with life...

An emotional computer...

From this moment on, regardless of whether you think

we're just gonna get stuck with narrow AI

or we'll get general AI or get super-intelligent AI

or conscious super-intelligent...

Wherever you think we're gonna get to,

I think any stage along that

will change who we are.

Up and at 'em, Mr. J.

The question you have to ask is,

is AI gonna do something for you

or something to you?

I was dreaming about sleeping.

If we build AI systems

that do everything we want,

we kind of hand over our world to them.

So that kind of enfeeblement is kind of irreversible.

There is an existential risk that is real and serious.

If every single desire,

preference, need, has been catered for...

Morning, Rosie. What's for breakfast?

...we don't notice that our capacity

to make decisions for ourselves

has been slowly eroded.

This is George Jetson,

signing off for the entire day.

There's no going back from there. Why?

Because AI can make such decisions

with higher accuracy

than whatever a human being can achieve.

I had a conversation with an engineer, and he said,

"Look, the safest way of landing a plane today

"is to have the pilot leave the deck

"and throw away the key,

"because the safest way

"is to use an autopilot, an AI."

We cannot manage the complexity

of the world by ourselves.

Our form of intelligence,

as amazing as it is, is severely flawed.

And by adding an additional helper,

which is artificial intelligence,

we stand the ability to potentially improve

our cognition dramatically.

The most cutting-edge thing about Hannes Sjoblad

isn't the phone in his hand,

it's the microchip in his hand.

What if you could type something

just by thinking about it?

Sounds like sci-fi,

but it may be close to reality.

In as little as five years,

super-smart people could be walking down the street,

men and women who've paid

to increase their intelligence.

Linking brains to computers by using implants.

What if we could upload a digital copy

of our memories and consciousness to a point

that human beings could achieve

a sort of digital immortality?

Just last year,

for the first time in world history,

a memory was recorded.

We can actually take recorded memory

and insert it back into an animal,

and it remembers.

Technology is a part of us.

It's a reflection of who we are.

That is the narrative we should be thinking about.

We think all that exists is everything we can imagine.

When in reality, we can imagine

a very small subset of what's potentially possible.

The real interesting stuff in the world

is what sits beyond our imagination.

When these massive systems emerge,

as a species, I think we get an F

for our ability to predict what they're going to become.

Technology has always changed us.

We evolve with it.

How far are we prepared to go this time?

There was a time when Aristotle

lamented the creation of the written language

because he said this would destroy people's memories

and it would be a terrible thing.

And yet today, I carry my brain in my pocket,

in the form of my smartphone,

and that's changed the way I go through the world.

Oh, I think we're absolutely merging with our technology.

Absolutely. I would... I would use the term "co-evolving."

We're evolving it to match our needs

and we are changing to match what it can provide to us.

So instead of being replaced by the machines,

we're going to merge with them?

I have a number of friends who have bionic arms.

Are they cyborgs?

In a way, they're on that path.

I have people who have implants

in their ears. Are they AIs?

Well, we're in a way on that path

of augmenting human bodies.

Human potential lies

in this co-evolutionary relationship.

A lot of people are trying to hold onto

the things that make them special and unique,

which is not the correct frame.

Let's create more.

Rewrite our intelligence, improve everything

we're capable of. Emotions, imagination, creativity.

Everything that makes us human.

There's already a symbiotic relationship

between us and a machine that is enhancing who we are.

And... And there is a whole movement, you know,

the trans-human movement,

where basically

people really bet bigtime

on the notion that

this machine-human integration

is going to redefine who we are.

This is only gonna grow.

There's gonna be a learning process that gets us

to that endgame where we live with these devices

that become smarter and smarter.

We learn how to adapt ourselves

around them, and they adapt to us.

And I hope we become more human in the process.

It's hard to understand how merging

with the machines

can make us more human.

But is merging our best chance of survival?

We humans are limited

to how many neurons we can fit in our brain.

And there are other species that have even smaller brains.

When you look at bees and fish moving,

it looks like a single organism.

And biologists actually refer to it as a super-organism.

That is why birds flock,

and bees swarm, and fish school.

They make decisions that are in the best interest

of the whole population

by thinking together as this larger mind.

And we humans could do the same thing by forming

a hive mind connecting us together with AI.

So the humans and the software together

become this super-intelligence,

enabling us to make the best decisions for us all.

When I first started looking at this, it sounded scary to me.

But then I realized it's scarier

for an alien intelligence

to emerge that has nothing to do with humans

than for us humans to connect together

and become a super-intelligence.

That's the direction that I work on,

uh, because I see it as,

in some sense, the only alternative

to humans ultimately being replaced by AI.

Can Baby X help us understand the machines

so we can work together?

Human and intelligent machine cooperation will define

the next era of history.

You probably do have to create a virtual empathetic machine.

Empathy is a lot about connecting with

other people's struggles.

The more connection you have,

the more communication that you have,

the safer that makes it.

So at a very basic level,

there's some sort of symbiotic relationship.

By humanizing this technology,

you're actually increasing the channels

for communication and cooperation.

AI working in a human world

has to be about humans.

The future's what we make it.

If we can't envision a future

that we really, really want,

we're probably not gonna get one.

Stanley Kubrick tried to imagine

what the future might look like.

Now that that future's approaching,

who is in charge of what it will look like?

Who controls it? - Google.

The military. - Tech giant Facebook...

Samsung...WOMAN 2: Facebook, Microsoft, Amazon.

The tide has definitely turned

in favor of the corporate world.

The US military, and I suspect this is true worldwide,

cannot hold a candle to the AI that's being developed

in corporate companies.

If corporations,

not nations, hold the power,

what can we do in a capitalist system

to ensure AI is developed safely?

If you mix AI research with capitalism,

on the one hand, capitalism will make it run faster.

That's why we're getting lots of research done.

But there is a basic problem.

There's gonna be a pressure

to be first one to the market, not the safest one.

This car was in autopilot mode

when it crashed in Northern California,

killing the driver.

Artificial intelligence may have

more power to crash the markets than anything else

because humans aren't prepared to stop them.

A chatbot named Tay.

She was designed to speak like a teenager

and learn by interacting online.

But in a matter of hours, Tay was corrupted

into a Hitler-supporting sexbot.

This kind of talk that somehow in the context of capitalism,

AI is gonna be misused, really frustrates me.

Whenever we develop technology,

whether it's, you know, coal-burning plants, whether it's cars,

there's always that incentive to move ahead quickly,

and there's always the regulatory regime,

part of the government's job to balance that.

AI is no different here.

Whenever we have technological advances,

we need to temper them

with appropriate rules and regulations.

Duh!

Until there's a major disaster,

corporations are gonna be saying

"Oh, we don't need to worry about these things.

"Trust us, we're the experts.

"We have humanity's best interests at heart."

What AI is really about is it's a new set of rules

for companies in terms of how to be more competitive,

how to drive more efficiency, and how to drive growth,

and creative new business models and...

Once corporate scientists

start to dominate the debate,

it's all over, you've lost.

You know, they're paid to deny the existence

of risks and negative outcomes.

At the moment, almost all AI research

is about making human lives

longer, healthier and easier.

Because the big companies,

they want to sell you AIs

that are useful for you,

and so you are only going to buy

those that help you.

The problem of AI in a capitalist paradigm

is it's always gonna be cheaper

just to get the product out there when it does the job

rather than do it safely.

Right now, the way it works, you have a conference

with 10,000 researchers, all of them working

on developing more capable AI, except, like, 50 people

who have a tiny workshop talking about safety

and controlling what everyone else is doing.

No one has a working safety mechanism.

And no one even has a prototype for it.

In the not-so-distant future,

you will have all kinds of AIs

that don't just do what humans tell them,

but they will invent their own goals

and their own experiments,

and they will transcend humankind.

As of today,

I would say it's completely unethical

to try to develop such a machine and release it.

We only get one chance to get it right.

You screw up, everything possibly is lost.

What happens if we get it wrong?

The thing that scares me the most is this AI

will become the product of its parents,

in the same way that you are and I am.

Let's ask who those parents are.

Right?

Chances are they're either gonna be military parents

or they're gonna be business parents.

If they're business parents, they're asking the AI

to become something

that can make a lot of money.

So they're teaching it greed.

Military parents are gonna teach it to kill.

Truly superhuman intelligence

is so much... Is so super.

It's so much better than we are

that the first country or company to get there

has the winner-take-all advantage.

Because there's such an arms race in AI globally,

and that's both literally, in terms of building

weapons systems, but also figuratively

with folks like Putin and the Chinese saying

whoever dominates AI is gonna dominate the economy

in the 21st century.

I don't think we have a choice to put the brakes on,

to stop developing and researching.

We can build the technology, we can do the research,

and imbue it with our values,

but realistically, totalitarian regimes

will imbue the technology with their values.

Israel, Russia, the US and China

are all contestants

in what's being described as a new global arms race.

All countries, whether they admit it or not,

are going to develop lethal autonomous weapons.

And is that really a capability that we want

in the hands of a corporate nation state

as opposed to a nation state?

The reality is if Facebook or Google

wanted to start an army,

they would have technologically superior capabilities

than any other nation on the planet.

Some AI scientists are so concerned,

they produced this dramatized video

to warn of the risks.

Your kids probably have one of these, right?

Not quite. - We produced

the Slaughterbot video to illustrate

the risk of autonomous weapons.

That skill is all AI.

It's flying itself.

There are people within the AI community

who wish I would shut up.

People who think that making the video is

a little bit treasonous.

Just like any mobile device these days,

it has cameras and sensors.

And just like your phones and social media apps,

it does facial recognition.

Up until now, the arms races tended to be

bigger, more powerful, more expensive,

faster jets that cost more.

AI means that you could have

very small, cheap weapons

that are very efficient at doing

one very simple job, like

finding you in a crowd and killing you.

This is how it works.

Every single major advance in human technology

has always been weaponized.

And the idea of weaponized AI, I think, is terrifying.

Trained as a team, they can penetrate

buildings, cars, trains,

evade people, bullets... They cannot be stopped.

There are lots of weapons that kill people.

Machine guns can kill people,

but they require a human being to carry and point them.

With autonomous weapons, you don't need a human

to carry them or supervise them.

You just program and then off they go.

So you can create the effect of an army

carrying a million machine guns,

uh, with two guys and a truck.

Putin said that the first person to develop

strong artificial intelligence will rule the world.

When world leaders use terms

like "rule the world,"

I think you gotta pay attention to it.

Now, that is an air strike of surgical precision.

I've been campaigning against these weapons since 2007,

and there's three main sets of arguments.

There's the moral arguments, there's arguments

about not complying with the laws of war,

and there are arguments about how they will change

global security of the planet.

We need new regulations.

We need to really think about it now, yesterday.

Some of the world's leaders in artificial intelligence

are urging the United Nations to ban the development

of autonomous weapons,

otherwise known as killer robots.

In the letter they say "once developed,

"they will permit armed conflict to be fought

"at timescales faster than humans can comprehend."

I'm a signatory to that letter.

My philosophy is,

and I hope the governments of the world enforce this,

that as long as a human life is involved,

then a human needs to be in the loop.

Anyone with a 3D printer and a computer

can have a lethal autonomous weapon.

If it's accessible to everyone on the planet,

then compliance, for example,

would be nearly impossible to track.

What the hell are you doing? Stop!

You're shooting into thin air.

When you're imagining

waging war in the future,

where soldiers are robots,

and we pose no risk of engagement to ourselves...

When war becomes just like a video game,

as it has incrementally already,

it's a real concern.

When we don't have skin in the game, quite literally...

...when all of this is being run from a terminal

thousands of miles away from the battlefield...

...that could lower the bar for waging war

to a point where we would recognize

that we have become monsters.

Who are you?

Hey! Hey!

Having been a former fighter pilot,

I have also seen people make mistakes

and kill innocent people

who should not have been killed.

One day in the future, it will be more reliable

to have a machine do it than a human do it.

The idea of robots

who can kill humans on their own

has been a science fiction staple for decades.

There are a number of these systems around already,

uh, that work autonomously.

But that's called supervised autonomy

because somebody's there to switch it on and off.

What's only been seen in Hollywood

could soon be coming to a battlefield near you.

The Pentagon investing billions of dollars

to develop autonomous weapons,

machines that could one day kill on their own.

But the Pentagon insists humans will decide to strike.

Once you get AI into military hardware,

you've got to make it make the decisions,

otherwise there's no point in having it.

One of the benefits of AI is that it's quicker than us.

Once we're in a proper fight and that switch

that keeps the human in the loop,

we better switch it off, because if you don't, you lose.

These will make the battlefield

too fast for humans.

If there's no human involved at all,

if command and control have gone over to autonomy,

we could trigger a war and it could be finished

in 20 minutes with massive devastation,

and nobody knew about it before it started.

So who decides what we should do in the future

now?

From Big Bang till now, billions of years.

This is the only time a person can steer

the future of the whole humanity.

Maybe we should worry that we humans aren't wise enough

to handle this much power.

Who is actually going to make the choices,

and in the name of whom are you actually speaking?

If we, the rest of society, fail to actually

really join this conversation in a meaningful way

and be part of the decision,

then we know whose values are gonna go into these

super-intelligent beings.

Red Bull.Yes, Deon.

Red Bull.

It's going to be decided by some tech nerds

who've had too much Red Bull to drink...

who I don't think have any particular

qualifications to speak

on behalf of all humanity.

Once you are in the excitement of discovery...

Come on!

...you sort of turn a blind eye

to all these more moralistic questions,

like how is this going to impact society as a whole?

And you do it because you can.

And the scientists are pushing this forward

because they can see the genuine benefits

that this will bring.

And they're also really, really excited.

I mean, you don't... You don't go into science

because, oh, well, you couldn't be a football player,

"Hell, I'll be a scientist."

You go in because that's what you love and it's really exciting.

Oh! It's alive!

It's alive! It's alive!

It's alive!

We're switching

from being creation to being God.

Now I know what it feels like to be God!

If you study theology,

one thing you learn is that it is never the case

that God successfully manages

to control its creation.

We're not there yet,

but if we do create

artificial general intelligence,

can we maintain control?

If you create a self-aware AI,

the first thing it's gonna want

is independence.

At least, the one intelligence that we all know is ourselves.

And we know that we would not wanna be doing the bidding

of some other intelligence.

We have no reason to believe that it will be friendly...

...or that it will have goals and aspirations

that align with ours.

So how long before there's a robot revolution

or an AI revolution?

We are in a world today where it is

absolutely a foot race

to go as fast as possible toward creating

an artificial general intelligence

and there don't seem to be any brakes being applied anywhere in sight.

It's crazy.

I think it's important to honestly

discuss such issues.

It might be a little too late to make

this conversation afterwards.

It's important to remember that intelligence is not evil

and it is also not good.

Intelligence gives power.

We are more powerful than tigers on this planet,

not because we have sharper claws

or bigger biceps, because we're smarter.

We do take the attitude that ants

matter a bit and fish matter a bit, but humans matter more.

Why? Because we're more intelligent,

more sophisticated, more conscious.

Once you've got systems which are more intelligent than us,

maybe they're simply gonna matter more.

As humans, we live in a world of microorganisms.

They're life, but if a microorganism

becomes a nuisance, we take an antibiotic

and we destroy billions of them at once.

If you were an artificial intelligence,

wouldn't you look at every little human

that has this very limited view

kind of like we look at microorganisms?

Maybe I would.

Maybe human insignificance

is the biggest threat of all.

Sometimes when I talk about AI risk, people go,

"Shh! Max, don't talk like that.

"That's Luddite scaremongering."

But it's not scaremongering.

It's what we at MIT call safety engineering.

I mean, think about it, before NASA launched the Apollo 11 moon mission,

they systematically thought through

everything that could go wrong when you put a bunch of people

on top of explosive fuel tanks and launched them somewhere

where no one can help them. Was that scaremongering?

No, that was precisely the safety engineering

that guaranteed the success of the mission.

That's precisely the strategy I think we have to take with AGI.

Think through everything that could go wrong

to make sure it goes right.

I am who I am because of the movie

A Space Odyssey.

Aged 14, it was an eye-opener for me.

That movie is the best of all science fiction

with its predictions for how AI would evolve.

But in reality,

if something is sufficiently far in the future,

we can't imagine what the world will be like,

so we actually can't say anything sensible

about what the issues are.

Worrying about super-intelligence safety

is a little like worrying about

how you're gonna install seat belts

in transporter devices.

Like in Star Trek, where you can just zip

from one place to another.

Um, maybe there's some dangers to that,

so we should have seatbelts, right?

But it's of course very silly because we don't know enough

about how transporter technology

might look to be able to say

whether seatbelts even apply.

To say that there won't be a problem because

the unknowns are so vast,

that we don't even know where the problem is gonna be,

that is crazy.

I think conversation is probably the only way forward here.

At the moment, among ourselves,

and when the time comes,

with the emerging artificial intelligences.

So as soon as there is anything

remotely in that ballpark,

we should start trying to have a conversation

with it somehow.

Dog. Dog. - Yeah.

One of the things we hope to do with Baby X is to try

to get a better insight into our own nature.

She's designed to learn autonomously.

She can be plugged into the Internet,

click on webpages by herself and see what happens,

kind of learn how we learn.

If we make this super fantastic AI,

it may well help us become better versions of ourselves,

help us evolve.

It will have societal effects.

Is it to be feared?

Or is it something to learn from?

So when might we be faced with machines

that are as smart, smarter than we are?

I think one of the ways to think about the near future of AI

is what I would call the event horizon.

People have heard of black holes,

this super dense point where

if you get too close, you get sucked in.

There is a boundary. Once you cross that boundary,

there is no way back.

What makes that dangerous is you can't tell where it is.

And I think AI as a technology is gonna be like that,

that there will be a point where the logic of the money

and the power and the promise

will be such that once we cross that boundary,

we will then be into that future.

There will be no way back, politically, economically

or even in terms of what we want.

But we need to be worrying about it before we reach

the event horizon, and we don't know where it is.

I don't think it's gonna happen any time soon.

The next few decades.

Twenty-five to 50 years.

A few hundred years.

It's over the next few decades.

Not in my lifetime.

Nobody knows. But it doesn't really matter.

Whether it's years or 100 years, the problem is the same.

It's just as real and we'll need all this time to solve it.

History is just full of over-hyped tech predictions.

Like, where are those flying cars

that were supposed to be here, right?

But it's also full of the opposite,

when people said, "Oh, never gonna happen."

I'm embarrassed as a physicist that Ernest Rutherford,

one of the greatest nuclear physicists of all time said

that nuclear energy was never gonna happen.

And the very next day,

Leo Szilard invents the nuclear chain reaction.

So if someone says they are completely sure

something is never gonna happen,

even though it's consistent with the laws of physics,

I'm like, "Yeah, right."

We're not there yet.

We haven't built a super-intelligent AI.

And even if we stipulated that it won't happen

shorter than 50 years from now, right?

So we have 50 years to think about this.

Well...

it's nowhere written that 50 years

is enough time to figure out how to do this safely.

I actually think it's going to take

some extreme negative example

of the technology for everyone to pull back

and realize that we've got to get this thing

under control, we've got to have constraints.

We are living in a science fiction world and we're all,

um, in this big, kind of social, tactical experiment,

experimenting on ourselves, recklessly,

like no doctor would ever dream of doing.

We're injecting ourselves

with everything that we come up with

to see what happens.

Clearly, the thought process is...

You're creating learning robots.

Learning robots are exactly what lead to Skynet.

Now I've got a choice, right? I can say,

"Don't worry, I'm not good at my job.

"You have nothing to fear because I suck at this

"and I'm not gonna actually accomplish anything."

Or I have to say, "You know,

"You're right, we're all gonna die,

"but it's gonna be sort of fun to watch,

"and it'd be good to be one of the people in the front row."

So yeah, I don't have a terrific answer to these things.

The truth, obviously, is somewhere in between.

Can there be an in-between?

Or are we facing a new world order?

I have some colleagues who are fine with AI taking over

and even causing human extinction,

as long as we feel

that those AIs are our worthy ancestors.

In the long run, humans are not going to remain

the crown of creation.

But that's okay.

Because there is still beauty

and grandeur and greatness

in realizing that you are

a tiny part of

a much grander scheme which is leading the universe

from lower complexity towards higher complexity.

We could be rapidly approaching a precipice

for the human race where we find

that our ultimate destiny is to have been

the forebearer of the true

emergent, evolutionary, intelligent sort of beings.

So we may just be here to give birth

to the child of this... Of this machine...

Machine civilization.

Selfishly, as a human, of course,

I want humans to stick around.

But maybe that's a very local, parochial point of view.

Maybe viewed from the point of view of the universe,

it's a better universe with our...

With our super-intelligent descendants.

Our field is likely to succeed

in its goal to achieve general purpose intelligence.

We now have the capability to radically modify

the economic structure of the world,

the role of human beings in the world

and, potentially, human control over the world.

The idea was that he is taken in

by godlike entities,

creatures of pure energy and intelligence.

They put him in what you could describe as a human zoo.

The end of the astronaut's journey.

Did he travel through time?

To the future?

Were those godlike entities the descendants of HAL?

Artificial intelligence that you can't switch off,

that keeps learning and learning until

they're running the zoo,

the universe,

everything.

Is that Kubrick's most profound prophecy?

Maybe.

But take it from an old man.

If HAL is close to being amongst us,

we need to pay attention.

We need to talk about AI.

Super-intelligence.

Will it be conscious?

The trolley problem.

Goal alignment.

The singularity.

First to the market.

AI safety.

Lethal autonomous weapons.

Robot revolution.

The hive mind.

The transhuman movement.

Transcend humankind.

We don't have a plan, we don't realize

it's necessary to have a plan.

What do we want the role of humans to be?