Original subtitles

Thank you.

Oh my goodness!

What are those?

that will be the end of my terrible Spanish.

for the evening, or possibly we might insert

a little in as we go.

I have been looking at some of the posters

for the festival up around campus,

and also looking at you here and thinking

I might do a little switcheroo on my talk.

it is listed as a talk about

using artificial intelligence to improve collective intelligence.

How can we all be smarter together and I'll talk about that.

but I actually thought

just the tagline

of the conference, Artificial Intelligence

and other intelligence

and trying to understand

the difference between how we think

and how current

bleeding edge

of artificial intelligence thinks would actually be really useful.

Having said all of that, anyone who's here because they want a deep

technical dive into machine learning and artificial intelligence,

please stay around and come talk to me afterwards.

I would be thrilled to answer any questions about anything I talk about,

or if you happen to have the misfortune of knowing my research, or me

for that matter. Anything else.

But instead, I wanted

to start with something different for a minute.

It won't even feel like we're going to talk about artificial intelligence,

because I think the most important question

I can ask you

that's relevant

to how I might affect all of our lives

is cut out.

How are you doing?

Are you happy?

And maybe even more importantly,

if you weren't.

What would you do about it?

And the reason why this is such an important question.

Such a fascinating question, is because research

about intelligence and happiness

has this rather astounding finding.

Intelligence doesn't predict happiness.

We could look at all of us in this room

here at a grand university.

I'm sure some of us are quite clever.

If we looked across all of the different people

and we

went and measured all the different ways

people measure intelligence in the academic world.

IQ, we don't use so much anymore.

But there's a modern version of it called G

or working memory span.

How many things can you keep in mind at any given moment?

How many different things can you attend to?

So all these different ways to measure

our raw cognitive power.

And none of them

correlate in any way with happiness.

What a strange finding.

There are many things to do.

People that are more resilient in their lives

tend to be happier.

People that have better communication skills tend to be happier.

People that have a strong sense of purpose in life.

Meaningfulness are substantially happier than those that don't.

Why not? Intelligence?

Every one of those things I just mentioned purpose,

resilience, communication skills.

They're predictive of a wide range of life outcomes.

How much money you earn in your lifetime, how far you'll go in your education.

But unlike intelligence,

which is a strong predictor, both of those

they also predict happiness.

What's going on there?

Well, I never be happy because it's only intelligence.

Or maybe we're measuring intelligence.

Wrong.

Maybe our own understanding of our own intelligence

has been profoundly misinformed,

and it's affecting our understanding of this new form of intelligence

that mad scientists like

me are bringing into the world.

So let's think about how we measure intelligence,

how people are publishing papers doing this

with new artificial intelligence like large language

models GPT, ChatGPT, Claude Gemini,

all these others, but also with our own children.

What do we do?

We give them a test.

And a test has right and wrong answers.

It could be a fairly simple test.

It could be a test specifically designed to be invariant to language

skills or cultural bias, like Raven's Progressive Matrices.

But still, these are tests

that have explicitly right and wrong answers.

And the more right answers you could give, the smarter you are.

Interestingly, we might

call this kind of a test this kind of a problem.

It's a well-posed problem.

It's a well-posed question is a question

that has an explicitly right answer.

And so if we're measuring people and how well they can answer

questions that have right answers, then we're measuring how

well they can answer well posed questions, well posed

problems.

In 25 years of my career,

I've never worked on a well-posed problem.

Overwhelmingly,

the problems that are of interest to society, to America,

to Mexico, to us in our individual lives are ill posed problems.

Problems that don't have defined

right answers.

If you weren't happy,

what would you do to change that?

That doesn't have a right answer.

It's fact.

It's been well-studied, and it would be different

for almost everyone in this room.

You would need to know yourself.

You'd need to know your context.

You have to change everything you plan to do in response

to a very complicated world.

Why does all of this matter?

Because of artificial intelligence

and the way that we have built it to date.

It's fundamentally designed to solve

well-posed questions

by its very nature.

Let's do a little walkthrough.

I think you've gone through two weeks of festival, but just in case

anyone wants a quick tutorial on artificial intelligence,

as we talked about earlier today,

it's any autonomous system

that can answer questions under uncertainty.

So uncertainty doesn't necessarily

mean that a problem is ill posed.

It could mean there's

this little blob on an x ray cancer

or just an artifact of the machine.

It could mean,

in this contract,

how many flaws are there that an opposing lawyer could exploit?

These, on some level, are well posed questions

in that they have explicit answers to them.

You might need an advanced law degree to read that contract

and see where the flaws are.

on the other hand,

you may not be able to describe as most doctors can.

What about the blob?

Makes it look like cancer to you

versus this other blob?

but we can take lots of images of cancerous,

x rays and lots of images of x rays that don't have cancer in them.

And we can label one as cancer and the other as not,

and give them to an algorithm

and have it learn and learn and learn to distinguish between them.

And in that sense, still fundamentally, this is a well posed question.

We have all the data we need.

We have all the right answers.

We even know the question that we're asking

is this x ray indicative of cancer?

That's a serious skill to have.

But now we live in a

world where machines can do that skill,

increasingly competitive

to us and beyond competitive.

In a recent paper published, a group,

used large language models.

It didn't matter what one.

So these are massive AIS,

some of the biggest that have ever existed.

Trillions and trillions of individual equations

getting computed to produce a next word.

You give it some words and it predicts the next word.

It's the world's largest autocomplete.

And while it may feel like I'm selling GPT short, I'm not.

What's astonishing about it is what that simple mechanism

can do just by producing the next word.

It, for example, can read a contract,

find all of the legal flaws

equitable to a working professional lawyer,

and instead of taking two

hours, it takes 30s.

If you work

out the math on that as published in this paper,

the at similar effectiveness

for identifying problems and contracts.

The AI is 99.7%

no 99.97% less expensive

than a human lawyer.

That's sounds scary.

Scary.

If you're in law school.

Scary if you're in med school.

And by the way, that wasn't just GPT.

That could do it.

Every one of the large language models,

including Lamda, the version that's free and open source out in the world,

could also beat the human lawyers.

so is this the end of the game?

Well, this is why I wanted to talk today

about the differences between human and machine intelligence.

it's not just about

whether you can answer well posed questions.

It's about the messy uncertainty of the real world.

It turns out that's what's

fascinating about us.

So let me take a little detour.

Now that I've framed where I want to take this,

and say

I love working with artificial intelligence.

I built my first model very long ago.

At least in the world of AI 25 years ago.

25 years ago, I was an undergrad.

I thought I was going to stick wires into cat brains

and be a wet neuroscientist and understand how the brain works.

And I took my first and only programing course, and the professor in that course

recommended me to work at this place called the Machine Perception Lab,

where we were building an AI that would analyze people's

faces in real time and tell whether or not they were lying.

If you were curious,

we've been given a bunch of money by the CIA.

so sometimes that's what happens in your life.

a bunch of spies give you money to make,

superintelligent machines that can tell if people are lying.

But we were just doing basic science research.

What could we actually do?

And it was amazing.

I could build a machine that could tell if someone was smiling or not.

And I thought, what can I do with this?

How could you make a difference in the world with something like this?

Even if the original money came from an organization

that not everybody in the world loves, and for very good reason.

Later in my career, I had the chance to build take what

I learned building those original models, and

I built a system for Google Glass.

It could find

someone, read their facial expression,

write the expression in the little heads up display.

Now, there's all sorts of terrible things you could do with this.

many bankers have asked me,

could we get this for all of our tellers?

And then whenever anyone comes in, it would just float everybody's

credit score above their head, so I'd know whether or not

they would qualify for a loan before I've ever even talked

to them.

suffice to say, I'm not interested in that.

We built this for autistic children.

It turned out that when an autistic kid had the chance to talk face

to face and a natural social interaction,

but they were told what the expression was,

something they don't get for free.

Autism is complex.

Some kids are better at these than others, but at its heart, autistic children

often have

incredibly hard times reading facial expressions.

So if you could build this thing

and while they're interacting with the people around them,

they could read their facial expression

or get a signal to teach them how.

And that was the amazing thing.

Not only did these kids learn how to read facial expressions

better than the standard, which at the time and still is today,

cartoon faces on flashcards, you just show them a face.

This is happy.

This is sad.

Now they were interacting with an actual person

that was smiling or frowning.

And they learned even better.

And not only that, they learned

theory of mind, empathy

turned out it's really hard to learn why

other people do what they do if they're all speaking a secret language.

You did not.

So I got to take artificial intelligence

and help these children understand

what makes other people happy and sad.

Again, fundamentally, a human experience.

Later, I took the work I had done doing facial modeling,

and did something even bigger.

I built a game,

a really terrible game,

of which I would be so ashamed.

But you need to wait for the end of the story.

This game was called Sexy Face

and it was online.

And if you went on to the Sexy Face website,

our promise was pick out the faces you think are sexy.

It would just pick random people out of Facebook and Flickr

that dated a little bit.

For those of you who are old enough to remember that,

it would just pick out these random faces and put them in.

Then you pick the faces you think are sexy

and for free will find everyone on Facebook that you think is sexy.

And for $5, we'll find everyone that thinks you're sexy.

Now, if

I am a horrible person, and that's still a reasonable thing

to be wondering about, it's not because of that.

After you played a couple of rounds, it then

confessed, actually, we're Trojan,

which means we're a secret game.

we're an AI, and this is a mind reading game.

So not just sexy faces.

Any face come up with any face category

you can think of and, well, guess it.

And so you think, I don't know, Southeast Asian

with, wispy mustache and a sense of ennui.

The amazing thing is

that our network

actually learned how to distinguish these faces.

It could really find a face

based on someone's hidden idea

of what it is.

and it turns out

even this game wasn't the end of the story.

The End of story is a book.

It's a book with a million photographs in it.

It is a book that is likely filling up with photographs right now as we speak.

It is the U.N.

Refugee Program's book of orphan refugees.

And when we did this project now over ten years ago,

it had a million photographs in it of kids

in refugee camps, all around the world, right there and then.

So if you were fearing the worst

and you hadn't heard from your sister's family,

and you drove all the way outside your country to a refugee camp nearby

and you go to them,

they would give you this book and you'd start leafing through it,

hoping you don't find your niece, hoping you do find your niece.

We, we took

the AI that was trained by people playing

our very stupid game,

and we used it to further train

an AI on the faces of all of these children.

And we built a system that could run on the little tablet,

and it would pull 18

random kids out of the book and would show their faces,

and you'd pick the ones that look more like your niece

and then it would pull up

a next set of faces based on your past responses.

And in about 3 to 5 minutes, if your niece is in the camp

anywhere in the world, you found it.

I love working with AI,

not because I love geeking out

about the mathematics of artificial intelligence,

although there are moments when that is fun,

not because I

have gnarly data problems or I mean fabulous engineer, I am not.

I am a scientist.

I have never written a line of code that didn't involve 32 curse words

for every single function word I put into the code.

I love it because of what it can do.

What we can choose to do with it.

Yeah, we built in the AI that the CIA wanted to use to tell

if people were lying.

But we also built systems to help

autistic children learn how to read facial expressions

and to reunite orphaned refugees with their extended family

through a program called Refugees United.

That's what got me excited about working in this space.

And over the years, I founded companies and I've published, research

in theoretical neuroscience related to our brains

and artificial intelligence, but

still would always really drove me along.

Was the idea that you could change a life

and that that would be valuable?

One of my current companies has the first

ever biological test for postpartum depression,

and it's predictive.

So before a mom ever even gives birth,

before she ever experiences the first symptoms,

we can actually see

in your epigenetics the biological signal

for postpartum depression and begin treatment before it affects you.

This is the most common complication in pregnancy in the United States.

It is the single most common reason women die in childbirth

is related to complications in postpartum

depression, including suicide.

And we can predict.

And it would be impossible to do this without AI,

in this case, a deep neural network that analyzes all the epigenetic data

and makes, the predictions that we need to be able to make.

But we didn't stop there.

The large language models, ChatGPT and others, we built a version of that.

They can also just talk with moms, talk with them before

they're even pregnant, just talk with women.

And in those conversations, we have a completely separate

AI that's analyzing their speech patterns and,

turns out, can also independently predict depression and suicide risk.

Being able to work on projects like these

and bring them out to the world is immensely powerful.

But you'll notice in none of these cases that I build

something that just kind of got turned loose on the world.

And there's a reason for that,

and it's a reason that it's become very salient

in the world of generative AI, which,

if there are any nerds in the room, is a very frustrating term for me,

because generative modeling already means something different mathematically.

And so now for some reason, we're calling this kind of AI

the exact same word that isn't mathematically applicable.

It drives a nerd like me up the wall.

But I think we all know what I'm talking about.

The the chat bots that you can now talk to the

the models that can create an image out of nothing.

And they're amazing.

They are huge breakthrough advances.

I will not share them. Sell them short.

GPT and Gemini.

I truly can surpass

human medical students in answering medical questions

and most average doctors as well.

And wouldn't it be amazing for all of those people

that don't have access to an immediate doctor to be able

to ask questions and get answers?

But here's the flip side.

Now that we've come full circle

and why I started this story

with this question that didn't

seem to have anything to do with artificial intelligence.

Are you happy and how do you become happy?

Because we might do one of two things right now.

We might look at AI being able to do

all of these amazing things and give up.

Just leave it

to jerks like Vivian to invent things,

and they'll do all the work and solve all the problems.

You definitely shouldn't trust me to do anything, but that is an option.

A choice that could be made.

Another.

And let me promise you, there are a hundred thousand startups

trying to replace every job

that universities are currently training students to do.

It's important to understand that.

But the other big problem

is that we start using AI

now, that it's no longer just the domains of the mad scientist like me.

Now that it is something all of us can interact with.

All of us can talk to.

Now, what we're wondering is,

are we going to keep trying as hard as we should be trying?

Are we willing to

put in the effort in our lives when we don't have to,

and something else can put the effort in for us?

So let's go back to this idea that I brought up earlier

that there are ill posed problems in the world,

and there are well posed problems in the world,

and that artificial intelligence

is quickly surpassing human beings

at solving well posed problems

and wonder,

why is that still

how we're educating our children?

Why are we raising generations of kids

to answer routine questions

when all of us in this room,

in our pocket, have the answers to those questions

for free?

Instead,

we should be building kids for the unknown.

We should be building them

to solve ill posed questions.

We need a generation of explorers

because the one thing, at least for now,

that natural intelligence does

and artificial intelligence does not,

is explore the unknown.

Now, to be clear, I use

AI regularly to explore the unknown.

I actually had a question.

this may be a question of zero

interest to the majority of you, but in a way, that's part of the point.

What is the confidence interval of the product of two regression model

parameters?

It's obviously

not a question that pops up in most of our day to day lives.

It's not a question

that actually even pops up in the day to day lives of most scientists.

Even statisticians might have to think hard about,

well, gosh, what is the equation to compute the confidence interval?

Well, fortunately we have GPT and Gemini.

So I ask them and they were wrong.

If you want to know

one simple take away lesson for how you should interact

with large language models like GPT,

they know everything

and they understand nothing.

GPT knows so much more than I do,

despite how much study I have, despite my fancy degrees,

despite my years of research, it just knows more than me.

It's encoded in those trillions of parameters

that are predicting the next word,

but it doesn't understand anything that it knows.

That isn't a knock on it.

What it can do is amazing.

but when I asked it

for the answer to this problem, which, by the way, is in fact a well

posed problem, just a very esoteric one,

a very weird and unusual one.

It gave me the wrong answer.

It gave me the answer to another, more common problem.

It's another thing to understand about large language models

and generative image models.

They are

the best of the average of the internet.

Turns out, the average of the internet doesn't typically have to know

what the confidence interval of the product of two model

parameters is, and so its first answer was wrong.

It answered a much simpler question, but I didn't give up on it.

I stuck with it

and we talked it out.

Now I didn't talk it out like it was a person,

but at the same time I did engage with it the way I might engage with

one of my graduate students, my graduate students, whatever they're studying,

they actually know it better than me.

They're invested in it.

I talked to them an hour a week about this issue.

They're fully invested in it.

This is their dissertation.

This is their ticket to whatever is coming next.

And they know it so much more than I do,

but they don't understand it.

That's what I'm bringing to that relationship

as an understanding of the implications of what they're discovering

and trying to guide them towards a similar understanding.

When I interact with GPT

or Gemini, I treat it the same way.

He knows everything and understands nothing.

And so I said, no, you've understood misunderstood my problem.

You've actually given me

the confidence interval to the product of two random variables.

I know this is super exciting to everyone, but that's what it did.

And so I slowly walked it through the problem.

I didn't know the equation

that I was looking for.

Interestingly, neither did it,

but about

half an hour later we had the actual equation written out and correct

and the code written out in the programing language Python.

To execute that on my data,

something I couldn't have done by myself and nor could it.

That's what excites me about AI isn't

what can I go do by itself?

I'm too busy to write emails to all of my business acquaintances.

I don't have the time to write an essay.

I don't want to have to think about the art I'm putting in my presentation.

Let I do it for you.

No, please don't make that choice.

Think about what you could do better

using AI as a tool.

It's not a magic wand.

It is also not intelligent the way we are.

Intelligent again, it's not a knock is astonishing what it can do,

but it doesn't understand the world.

That's your job.

That's our job.

Instead of building a

generation of students ready to answer tough questions

that this can do so much more easily,

we should build a generation of students

that know how to find good questions.

A generation of students

that when their first try doesn't work, they don't give up.

They keep trying because they have a belief

that their hard work will actually pay off.

None of that applies to artificial intelligence today.

This little interlude is just going to be for the nerds in the room.

There is a famous difference in human learning

that is identified in both neuroscience and psychology, called model free

and model based learning and model free is basically just statistics.

And we in our brains

definitely do a huge amount of model free learning.

And essentially, that's what most of AI is.

It's just a bunch of raw

correlations churned and churned and churned.

Interestingly, on masses of data that no human could ever interact with,

the, the there's a whole other class

of artificial intelligence called reinforcement learning models,

the most famous of which would be AlphaGo,

the AI that beat the world's best human go players.

And since then,

Deep Brain, the Google subsidiary, has gone on to use AlphaGo

to create AlphaFold,

and to create,

the new system that can discover all these proteins.

But in all of these cases,

it's just blindly exploring

in a single night,

the reinforcement learning system from AlphaGo can play something

on the order of 180

years of human experience.

Go in a single night.

It plays so many games against itself.

It is the human being played it with,

you know, only breaking to sleep for 186 years.

It's not so

surprising that it does better than we do,

but does it truly understand? Go.

I mean, it would keep playing go if the game was back or at

it would keep playing go if the house was burning down.

it turns numbers into numbers.

That's a powerful tool.

But it's not the ability to explore the unknown.

Let me ratchet this up another level.

A lot of research has looked at human workers

using large language models to speed up their work.

Published papers, some of them by OpenAI itself, others

by notable economists at Harvard and Stanford and beyond.

So, for example,

let's start at what economists

would see as the low end of the skills ladder.

Call centers.

You're working on call center.

Someone calls up, you take their call, you essentially have a script,

and you just run through the script to try and answer their questions.

And what they found was when call center workers

had a large language model helping them answer the questions.

They were 40% better and 40% more productive.

They answered 40% more calls and had fewer referrals to their manager.

They were better.

Now let's take a step further

and look at programmers.

Maybe I'm not being fair.

Many people go to university to learn how to program.

Many people learn how to program completely on their own.

It's a little unclear.

This was certainly the skill of the future.

Not five years ago,

certainly ten years ago.

Every policy paper in the world

from the UN, from the World Economic Forum, the white House, McKinsey,

every major country had a policy paper

and it said, we need to teach every kid how to program.

Well guess what?

We live in a world in which I can program.

and I could have told you that ten years ago, and I did.

I didn't tell you, but I told everyone writing these policy papers.

What a crazy thing.

Imagine you had asked me,

what is

the world financial markets going to do for the next 30 years?

And I said, don't worry about it.

Simply take all of your money.

Buy nothing

but this one stock and then never change it.

The logic is about the same.

Teach everyone in the world how to program because that will be a protected skill.

Programing is routine knowledge.

It's a solution to well-posed problems.

People can do creative things with programing, but the actual core

of programing is routine and I can do that and do it very well.

So programmers using large language models

to help them program again about 40% faster

with fewer errors.

and now I will step

up, professional writers with university educations.

Again, they're faster and they produce better work.

Boston Consulting Group consultants

with very fancy degrees from elite universities.

They're asked to analyze spreadsheets of data

and produce a report, a written report,

and then a set of slides for a presentation.

And some of them are given GPS

to use and some of them aren't.

And unsurprisingly,

the ones using GPT are faster and produce higher quality work

just the same as all the other workers.

Well, but there's two

incredibly important missing parts to this story.

One in every one of these stories,

it wasn't everyone that received the boost.

Only the least experienced and lowest skilled workers.

Interestingly, at every skill level, the people that already knew

how to do the job didn't actually get benefit from using AI.

That's really interesting,

particularly if I add in this additional twist

from the world of education where we've been studying using AI for decades.

If an

AI tutor ever gives the students the answer.

They never learn anything.

Wait a minute.

Isn't that exactly what's happening with these workers?

With the new career, workers

are starting their jobs and the AI is telling them what to do.

I promise you, they will never learn how to do their job

any better than they're doing it.

The day they show up.

The idea that we should simply offload our effort

and our work to I robs us of the chance to grow.

But there's even a bigger issue,

which is in the Boston Consulting Study.

there was one additional test

because their job, their job

doesn't end with analyze the spreadsheet, which is a well-posed problem,

and write a report which shouldn't be a well-posed problem.

But I don't know how many of you have done a business report.

You might as well let an AI write it.

They read exactly the same every time.

so what was after

that is they then had to say, what should their client

do about the analysis that they just ran?

And so now we've moved from Well-posed to opposed.

We moved into a world in which there isn't a right answer.

And in fact, very cleverly, the researchers set it up

so that GPT would give them the wrong answer.

And the people using the

AI did worse than the people that weren't

when it became a creative task.

That should scare all of us.

We are uniquely good

for now at exploring the unknown.

But these highly educated,

highly motivated workers,

when given an AI tool,

stopped exploring and just did what it told them to.

So how do we reverse this trend?

How do we get the benefits of artificial intelligence?

And again, I can just bragging, Lee, describe my own work

autistic children that can learn how to read facial expressions

or from refugees reunited with their extended family.

I built the first ever system that could,

predict bipolar episodes.

manic episodes and bipolar sufferers.

My company,

has the first test ever for manic depression.

When my son in 2011 was diagnosed with type one diabetes.

I hacked all of his medical devices.

Broke.

As it turns out, all sorts of U.S.

federal laws.

And I developed the first ever AI for diabetes just for my son.

It sent updates to my Google Glass, which, believe it or not,

I wore to a party at the white House.

There is literally a Secret Service policy

at the white House naming me saying,

you can't wear live computers on your head in the white House anymore.

but I did get to meet President Obama because of that.

And after he met me and walked

up, he actually walked up to me, look straight at me,

and just shook his head like I was such a disappointment.

What are you wearing?

A fluorescent blue computer on your face at my fancy party?

But then I said,

okay, Google, take a picture.

so that was my voice activated Google Glass.

It took a picture of the president.

Google has confirmed it is the only presidential portrait

ever taken by Google Glass.

It is a terrible picture.

it did not improve his opinion of me, by the way.

but then he said,

why are you wearing this silly thing?

And I said, because I built an artificial intelligence

that's monitoring my son's diabetes.

He's back in San Francisco.

I'm on the other side of the continent in Washington, D.C..

I just got a text message from my AI

telling me my son's blood glucose levels would go dangerously low.

So I sent a message back to my sister who was watching after him.

She gave him a tracker and his blood glucose levels didn't go up.

I built myself a superpower,

something that every parent should have the chance to build.

I truly can do amazing things.

We should have it in our lives,

but we shouldn't use it to substitute

for the things we can already do for ourselves.

When I use artificial intelligence today.

I use it in a peculiar way.

Now, I'm not talking about the models I built for my research.

All of those I'm building by hand, and I'm trying to solve specific problems

and explore things I don't understand about the world.

But when I use it as a regular person,

I, for example, in

using it to write on a book I'm working on right now.

The worst thing I could do is say GPT,

write the next chapter of my book.

I have actually tried.

It writes garbage.

It is a terrible writer.

Not it gets the punctuation.

Actually, I wish my students spelling

and punctuation was as good as GPT three.

Human beings turn out to be terrible at that,

but it's so banal.

It's so substance less.

Instead, I get to do something different when I engage with it.

I write my own,

my own article.

My own essay.

Of course I do.

In a world where all of the routine knowledge is available

for free in your pocket.

What's truly left for us

is our unique voice,

the way we see the

world differently than anyone else.

Literally anyone else.

And is your resilience

and your,

courage to use that voice

that is your value to the people around you.

So how do I use.

In my case, Gemini, Google's version,

I say, Gemini, you are my worst enemy.

You have been hounding me for my entire career,

finding every floor and everything I've ever written.

You were the reason I don't have a Nobel Prize.

Because you keep pointing out everything I've ever done that's wrong.

Let's just pretend that that's true.

Now read my next essay and find for me

all of the arrows, all of the things

you might criticize.

I'm not speeding up my writing.

I'm slowing it down.

I already went through and wrote the whole thing

in a way that I was happy with,

and now I have turned the AI

against me to tell me all of the mistakes I made,

all the flaws that exist, all of the ways people might disagree with me.

Now, I don't have to believe everything it says.

I work with it like it's my grad student.

It knows everything.

It doesn't understand everything.

And so I take some of what it says.

I ignore some of what it says because I don't think it understood.

And it's what we call productive friction.

Rather than using AI to make my life easier.

I use AI to make myself better.

That's what we should all be doing.

Whether you are interested in building AI yourself,

or you are interested simply

in making use of it for your everyday life.

Ask yourself this very specific question

first.

Am I better when I'm using it?

And then think not just AI, but a lot of technologies.

We might admit we're not really a better person when we use it.

Feel free to interpret better person as you want.

But for me, it means

when I'm not so long from now, 65,

will I still be alive?

Well, I've made lots of money.

Well, I have lots of friends.

Well, I be happy.

Things we might all agree is a better life.

Does this technology, when I'm using it, give me a better life?

But here's the second question.

And to me it's just as important.

Am I a better person than when I started?

When I turn it off again?

And even amazing

useful technologies frequently follow that second test.

And I have a whole story I can share about Google Maps

and automated navigation, and why it's an astonishing technology

that is increasing dementia,

and everyone around the world.

But I've run out of time and so you'll have to ask me about it

later or come chat with me after this event.

It's been a lot of fun.

I guess I never got back around to talking about collective intelligence, but,

So I'm gonna do this one last little plug.

When the pandemic started,

I have a very strange life.

In addition to my companies and my academic work.

My real day job is running a philanthropic lab.

People bring me challenging problems.

And if I think my team and I can make a meaningful difference,

I pay for everything.

And whatever we invent, we give away.

So it's the stupidest startup idea anyone's ever come up with.

Endless demand, zero revenue.

But it's also the best job in the entire world.

And so people write me and they say, my daughter has a life threatening

egg allergy. Please save her life.

I say our company has bias in this promotion process.

Help us figure out where it's coming from.

Our country's education system is failing our students, but we're doing

everything we're supposed to do, and it still isn't helping.

And again, if I think I have my unique voice not because

I'm smarter than everyone else, but because I'm different.

I have become very comfortable with being different.

Does my unique voice and insight have something to say about this problem?

And if so, we just do it.

So at the beginning of the pandemic, I got almost

the identical question from two separate people.

Except they weren't people.

They were giant multinational companies.

Facebook and Amazon called me up and said, we just sent everyone home

and we don't know what to do.

And what we

particularly don't want know what to do is how to be inclusive

and how to innovate through a camera.

If everyone's life is going through a camera

and we never see one another, and we never have time to talk

at the,

over beers or at the watercooler,

where does that collective intelligence

and the brilliance of people come from?

And because I'm weird, I thought that sounds like a reasonable

philanthropic project.

I should help two of the largest, richest companies in the world

answer that question.

But as long as they're fine with me telling everyone what I found.

Then we dove in and I got to see everybody's life

going through those cameras.

All of our interactions around documents.

And we discovered that the smartest teams had three things in common.

They were relatively small, 3 to 5 people.

They had very flat hierarchies.

There might be a Nobel Prize winner and undergraduate intern.

There might be the SVP of engineering

and, someone who's just there for the summer.

It didn't matter when they were collaborating.

It all disappeared and everyone took equal turns.

Again, it is you, your unique voice

that has value to this world.

And the last ingredient

is that those teams were diverse.

So I truly am now finally out of time.

But I will simply say this sounded like a cool thing to build

an artificial intelligence model for,

not one that people would interact with,

but one that we called the matchmaker

and would look through a social network of people

and find all the matches to make and break.

Just as importantly, breaking social connections

to maximize the collective intelligence of teams.

And what we found is you could double

the innovative output of a team of people

by just shifting around those connections

dynamically, using our AI system.

And and again, it was so exciting because it's another look

not at how I can replace us,

but how it can make us better

in this case, not better as individuals,

but by finding the community

with which we shared just enough trust and enough difference

for the sparks to produce something truly amazing.

So I'm going to stop here.

But again, say, for anyone that's foolish enough to want to stay around

and ask me questions, I am happy to spend as much time here afterwards as I can.

Thank you all so very much.

Let's get it.

Thank you very much.

Okay. The.

First video.

Now is.

No more.

Thank you.

Thank you very much.

got all those questions.

So this is easy for you to answer?

the question in English or Spanish and, okay.

Got translated.

Thank you very much.

It has been an amazing talk.

I think, the use that you see here, the research the CIA,

the children we saw, you know,

it's amazing.

So I want to

work with children all week away today. You.

Yeah.

With children, with ADHD

paper.

I can.

Yeah. So,

I think a whole lot of people

like myself have had, diagnoses of, ADHD.

And interestingly, the comorbidity, the genetic underpinnings

of many of these sort of track together.

So ADHD,

we certainly don't think of it as being as challenging as autism

or even things like bipolar or schizophrenia, but

you see similarities genetically and where it's emerging from.

So, yes.

one of the things I'm looking at, more in the neurotechnology side of my life,

one of my companies has developed a,

neuromodulator treatment for Alzheimer's.

so the idea is we stimulate the brain

in a way which,

tricks it into thinking it's working very, very hard.

And this is good for Alzheimer's because it

it causes the brain, particularly support structures

in the brain, glial cells and astrocytes to help clean

up accumulated,

byproducts of, of neural activity.

And that clearly, is helping,

with Alzheimer's.

But it turns out this idea

that you could stimulate the brain and you can stimulate it.

We're using light.

So strobe ascorbic light at 40Hz, 40 times per second.

Light is flicking or at eight hertz.

or you can use electrical stimulation, alternating

current, transcranial stimulation and some other methodologies.

You can cause these,

this synchronized activity to, speed up in the brain.

And I've looked at traumatic brain

injury in kids recovering working memory function.

there's research looking at in autism

at enhancing functioning there and intra cortical connectivity.

So this seems to be one of the challenges with autism.

Is one part of the brain isn't talking to other wide ranging

areas as well as you might in people that were neurotypical. And

certain kinds of stimulation seem to help with that.

This is all very early in ADHD.

It's less clear exactly what the mechanism is,

but it does seem that that 40Hz

stimulation seems to help.

But it's unclear now.

I don't want to totally nerd out in front of everyone else,

but one of the issues with ADHD might simply be that

the resting state of your brain and another state.

So what we call default mode network versus

executive control network, that those

those two states of your brain are very distant and people that have difficulty,

different kinds of difficulties, for example, inhibitory control,

people that commit crimes a lot looks like these states are very different.

It's hard for them to take to shift to a control state

when they suddenly have an impulse to do a thing.

But what it seems like maybe with ADHD

is that the saliency, the sort of sensory states

they're just very active and very close to the default state.

And so as soon as you see something, it kind of takes over and takes

you into a different place.

so all I can really say is

there is some exciting new work using normal jittery, triggers.

And there's generic artificial intelligence work

from my lab and others looking at,

what sort of lived experiences,

what sort of educational interventions are effective across different kids?

So years ago we released a system called muse, which helps new parents

with young children for social emotional development.

and we were super excited and had all sorts of great results,

but it was made for not for average kids, because the whole point of the AI in

our system was figure out who this child is

and what experiences are right for them, not for someone else.

But now we're collaborating with a group in Vancouver

that specifically wants to take this explicitly to non neurotypical.

So ADHD and

other let's

call them non-clinical level, learning challenges.

And I think there's a huge space of just being able to identify

which behavioral therapies will actually be effective,

as well as making space for doing noninvasive

therapies that could also help as well.

And, you know, I don't want to rule out the possibility of primary

treatment, drug treatments helping.

But I think the first step in most of these, the way I think of it

as a computational neuroscientist, is

I want to get these different states.

I want to get the default state closer to the one

we want and farther from the one that's causing the problem.

And that's a very new way of thinking about this sort of a problem.

and if you ever want to talk more, be happy to follow up on it.

But there is work in this space.

I have a my daughter has been diagnosed with,

we work with,

I wish, you know.

People are genuinely working hard on this, but I will say as a having a kid

with diabetes, what I have been told is for the last 50 years,

people have been saying, as soon as, you know, when my kid got,

diagnosed, they told me a cure was right around the corner.

And for 50 years we've been waiting.

So I don't want to over promise.

But we are making progress.

Yeah.

And, My vision for this

as we kind of,

as I think please remember

in the speech you mentioned.

and so how can you use, how we teach,

using the use to make people, a little.

So, hopefully

we can get you to use 3 million.

we have been involved in,

in these issues from,

the war and in the post-World War II.

So we have,

approaches that people can use about people

that take place within the world.

We use in this

been used in school. So,

simplification and this the

how can we go from, in order to world.

There is no doubt

that two of the biggest consumers of artificial intelligence,

though they will never admit it publicly, are the U.S

and the Chinese military.

but even aside from that,

OpenAI and Microsoft, Google,

Baidu and China of two countries

and maybe six companies

effectively control the artificial intelligence infrastructure of the planet.

It's not that there aren't

other people doing interesting things with the AI in the world,

but if you're going to use a large language model,

there are only so many that actually exist, and all of them

are trained on the infrastructure of either Amazon, Google or Baidu.

So, or the internal infrastructure of a company like Facebook.

So that's problematic.

I have two answers to your question.

One is an essay.

I just wrote, an op ed

that was published in the Financial Times of the United Kingdom,

and it had this very provocative opening

as it is being used today, artificial intelligence

is fundamentally incompatible

with any modern notion of civil rights.

The idea that you might be

denied a job or a loan,

or that you might receive

additional scrutiny from the police,

you have no control over that.

And I know this because some of that were my companies.

One of my most successful companies was using AI and hiring my customer

wasn't you.

My customer was Google and Citibank and Nestlé.

So we built an AI

that figured out who they should hire.

We didn't build an AI to tell them to hire you.

So, and my new company's working in health,

working on Alzheimer's, working on postpartum depression and more.

let's say you go to a different doctor because you want another diagnosis,

but they're just running the exact same AI algorithm,

so you get the exact same diagnosis.

And probably if you're in America,

your insurance company is not going to pay for it anyways.

So what are we going to do?

So the end of that essay, my argument was

just as we have a right to a doctor

or a lawyer acting solely in our interest,

access to AI acting only for us

must be a civil right itself.

Otherwise civil rights just doesn't make sense.

If groups that you might be concerned about that

I'm concerned about are making decisions in 50 milliseconds

in a server farm in Wyoming,

and you never even knew that decision was being made,

how would you be able to take action against it?

So in that spirit, I've tried to do something different

with my company, Denise's health, that's

working in on postpartum depression, epigenetics.

I made a promise before we ever founded the company.

At the same time, we would found an independent

nonprofit, what's called the Data Trust.

And it it not my company

would own all of the data and run all of the algorithms.

And it sole reason for existing and fancy language.

It's so fiduciary responsibility is the mothers from whom that data is coming.

So legally, we've created a legal framework

in which our eyes sole purpose

is to help the health outcomes of those moms.

I actually happen to believe it's actually a good business decision.

If we stop arguing over who owns the data

and instead thought does May I save lives?

Is it actually making people's lives better?

Is my business based on having the best product,

or do I have some unique data

set that no one else has that I stole from someone else?

Anyways?

Well, if we could stop arguing about that,

we could build systems that were genuinely meant to help people.

Now I realize I'm being a little idealistic about it here, but

part of what I wanted to

just show is someone could found a company based on artificial intelligence

and make an explicit commitment

that it would only serve the interests of the people that we were trying to help.

We're going to make lots of money.

We have the only test in the world, and it's a test

for a condition that saves lives.

so if we can't make a business out of that,

we should stop pretending to be business people.

I don't need to also sell people's data.

I don't need to.

Also abuse the data and trick them into doing things.

So this is this is hard.

This knowledge of how to build an AI is out there in the world,

and the ability to do it at massive scale is controlled

by a very small number of organizations and countries.

What are we

in Mexico and the United States around the world?

All of the rest of us are going to decide

is how this AI is going to affect our lives.

And I'm not saying this again, as someone that is reflexively against AI,

it's amazing what it can do.

We have a right to its benefits in our lives,

but we also have a right

to decide what those benefits

should be for ourselves and the trust to know

that that is happening in the way we want it to happen.

So the simple truth is,

people are going to abuse this stuff.

What we need to decide

is what are the rules we want to put in place

that bring the most benefits while minimizing the harms as much as possible.

And unfortunately, right now

we live in a world where people like Elon Musk and Sam Altman,

on one day are warning about

the evils of AI as they themselves are building them,

which should make us very suspicious.

when Sam Altman

actually just before my op ed

came out, Sam Altman

in an interview said, you know, will judges

be able to subpoena your AI, your personal assistant,

so that they would be able to use that information against you in court?

And that is a legitimate issue also addressed in my op ed.

What I note he's not pointing out is he already has all that information

and is happily using it against us.

It's a little bit like the snake from The Jungle Book warning

the little kid about the tiger.

I I'm worried about both of them.

we're the ones that need the protections, not Sam Altman.

and certainly not the US government.

So your concerns are well-founded,

but there are people trying to build these things in the world

on principles other than how rich can we get

and how much of the world we can take over?

but in the end,

it's our decisions that matter.

it's our ability to,

to collectively decide

what we want these borders to be.

It shouldn't be me.

Just because I build them first doesn't mean I get to decide.

I want this to not be treated like it's science fiction.

I think we feel that way maybe today about AI.

But I'll also say about neuro technologies.

Elon Musk also has a neurotechnology company.

And trust me, if you didn't like having Facebook

knowing everything about you, just wait until they get inside

your head to literally,

let's take these issues genuinely seriously now.

And ultimately, I think the best way to do it is to empower individual people

to be able to truly make decisions about their own data

and their own interactions with artificial intelligence,

rather than broad guidelines.

Or on the other side, just

trust me, I'm here to save the world.

so I wish I had a more concrete

answer for you, but all I can say is what I personally am doing,

and I'm hoping that we'll be able to prove something with our new trust.

So there were a number of hands, up.

Okay.

And get first.

thank you for an interest in that.

well, I have a question for AI since,

I've been doing these,

tools or

make better some of them.

And really, changed my life.

I guess some need for, to build up my state of the brain.

and, I and some other to do

and make my life easier.

And in number three.

But those two are only useful for me for a moment.

And I think there

might be many

people doing the same as I'm doing at this moment.

But we have the resources that we must have.

And so I need

would like to help other people

and to have this, these to make our lives better.

And but my question for you is.

What or where

should I start if I want to?

You have any and the air.

what should be my first?

Yes. Where can I go?

yeah.

So there are different ways to answer that question.

One is whether you specifically want to have an impact in the AI world.

And even then, we could break it up.

Do you mean the bleeding edge mathematics call researchers

that are looking at convergence proofs and complexity

theorems, around what's possible with given structures?

or, you know, the build

community that's really going out and trying to make something.

And even then I would say that there's a whole other space

which is just going and building things,

that, that they happen to use.

AI is secondary, that they solve a problem of interest is primary.

so that's really what that latter group was,

what it was the experience for me.

I didn't plan on studying, machine learning.

I couldn't have I, I mean, I was a big science

fiction nerd as a kid, so of course, I'd read.

I don't know what the first thing

there is an AI in an episode of Star Trek.

Maybe that was the very first AI I'd ever seen.

There's the AI in 2001,

but as a very little kid, I probably wouldn't have watched that movie.

but that was my and most people's introduction to artificial intelligence.

For me, it's interesting

and noteworthy that I was a secondary thing.

I was studying brains and people, and then I got introduced

to a specific subfield, theoretical neuroscience.

where kind of like theoretical physics,

we would use mathematical principles to study the brain.

And it turns out if that's where you're starting from,

then you very quickly become a machine learning researchers

because it's mathematically principled and it looks like the brain

does something at least roughly similar to that

usually, especially for early vision

and early audition.

so then for me, I was always a tool.

It was always a way to understand something else.

So that was that is and how I have always engaged with it in this way.

It's kind of funny because then I get introduced as being an AI expert.

Well, let's be fair.

There are people that spend every minute of their life

working on the next limb or the next generative model.

Actually, the guy who came up

with the original diffusion model is a former lab mate of mine,

eyes, the original first author on the very first paper, a brilliant guy

who then wrote a whole essay apologizing for it later,

because of

how he didn't like how it's being used.

and, you know,

he put a lot of work, for example, into using AI modeling in education.

That was something he thought was worthwhile, whereas putting artists

out of business didn't feel like a great job to him.

so there are all these different ways you could go about doing it.

And I'm not saying my way is the right way.

The right way for you is kind of where you want to take it.

There are communities out there kind of around the world,

that are in these build spaces,

I'm sure, for example, that there are Reddit communities

working on stuff like this, although you couldn't get me

to spend 30s on Reddit for anything ever.

So, yeah,

you know, there's a lot, at UC Berkeley,

there is a really vibrant community,

hackers that just get together.

So I'm the chair of the Neurotic Collider Lab at UC Berkeley,

the chair of the external board, and,

this what we do is just students from across the university,

the neuroscience Institute, the School of Engineering,

even the business school, they all come together,

they form up a team and they try and build a product.

And then, you know, we take it from there and it's really cool.

Most of what they come up with doesn't go anywhere.

Some of it's genuinely interesting and a whole lot of it uses.

I mean, you can't do modern day neuroscience

without machine learning.

and but they're in the computer science department at Cal.

We have a whole new school of data and society,

where people doing stuff like you.

I'm not pitching you on, Cal, but I bet there's a hacker

culture around here that's doing that same sort of thing of building

and trying out ideas and seeing what connects.

You don't happen to be down the street from,

you know, a $1 billion venture fund that could look at it.

But, I know there's some money in this town and there's probably some VCs

that are playing around here looking for interesting ideas.

I will

simply say, whatever approach you take,

put the problem first.

I super cool, but again, still a tool.

Overwhelmingly,

the hardest thing is finding the right question.

When Amazon tried to hire me to be their chief scientist,

one of the projects I would have run is a deep neural network

that would help speed up the hiring process,

that would filter out who they ought to bring in for interviews.

And I saw what they were doing with it,

and I told them at the time, this isn't going to work.

And then it actually came out publicly

that it just it would not hire women,

which is exactly what I expected.

but they're not necessarily bad people.

So they did everything you're supposed to do.

They took out all the women's colleges and all the names of everyone off the.

They took out everything that could trigger a human being.

So you could see that this app job applicant was a woman.

And I did something amazing, something no human could do.

It figured out who was a woman and wouldn't hire them.

So why did the whole system fail?

Because Amazon hired

a bunch of brilliant young, largely men.

That doesn't make them bad, but it just was the case.

And for the seven years prior to that, working on their PhDs,

their advisor had said, all right, we're going to build an AI,

and it's going to identify all of the dog breeds in photographs,

which is an actual competition that I claim every year.

and here is all the training data.

Here's all of the right answers.

And the question, is there a dog in this picture?

And if there is, what breed is it?

Never.

That's a perfect example of a well posed question.

Amazon was confronting an ill posed question

how could an AI improve our hiring?

And the first thing you have to do there is figure out what the right question is.

What Amazon chose was,

let's build an AI

and train it on who got a promotion in their first year at Amazon.

Well, guess what?

The single biggest correlate of

getting a promotion your first year at Amazon is being male.

So that's what the the I learned no matter what they did,

no matter how they tried to retrain it, that's all it would learn

because it doesn't know what makes a great employee.

It just knows the correlations.

Same as GPT.

It doesn't know language.

It knows the correlations it produces.

The next best word.

Both of them did amazing things, one

traumatically wrong, but still kind of amazing.

And the other,

it produces errors.

Again, if we mis understand a large language model

as actually a reasoning machine, that's

really, truly thinking things out,

then then you're mystified as to why

it just makes things up every now and then and completely confabulation.

So when you're working on a problem,

start question first.

What is the problem that I want to solve so much

that I'm willing to put time into it, even if no one ever knows that I did.

And if this is a problem, why is it a problem?

Why haven't we figured it out?

Because if it's still a problem today, that almost certainly means

no one is thinking about it correctly.

So now you have a place

where you can build some really cool AI to do something interesting.

Only once you've reached that point.

So I wish I had a recommendation for a bunch of really

cool communities for you to connect with here in Mexico.

Around the world I have the social skills of an autistic badger.

I forget incredibly well on a stage, but I enjoy talking

to random people about as much as I enjoy poking out my eyes.

So I love going into my lab

all by myself and building something new.

and then I listen to the world

and see if it actually works and learn from that.

And then people come to me

and talk to me about that thing that I built

and how it worked, and maybe a way to do it better.

So maybe I am suggesting that also is

don't just build a thing,

release it into the world.

There's nothing like the learning experience

of releasing something and seeing it getting used or not,

and then studying why it worked or didn't

and trying again and again.

Because it takes a long time to get good at this stuff.

The solving of the problem stuff, the AI stuff also takes a long time,

but it will inevitably connect you

into community, which I can say as someone

genuinely on the spectrum.

And yet I get to travel around the world.

World leaders ask my opinions about things

that should scare all of us.

and yet it happens.

Despite my total lack of social skills, despite the fact that I don't network

with anyone because I build things and I put them out into the world,

people come to me and

you should give that a try.

I have, I have for sure.

One is did you see the as

can the way of making decisions.

This is about you stand by money on that one thing that weighs

you building up confidence.

the second question scared me.

So I just want to know,

the first time I ever wonder, what would you recommend me?

you know, paper

towels, those things that you love, you so

you actually think. Go.

Okay.

So to the first one,

I think that the problem of automated navigation

systems, paradoxically, sometimes making traffic worse

is exactly some of the issue with AI

that as it gets built in a way,

I'm actually going to return it and say it's a it's a problem with us.

We thought, oh God, if only I knew

that secret fastest route

that only the locals knew then I could get around.

But then if you make that available to everybody,

things go awry.

and often people

building these systems don't really think through

the chain of causation that runs out from there.

So my comments earlier, for example, about inexperienced early career workers

being harmed by the AI, the very AI that's helping them do their job

because it's robbing them of the chance to learn

how to do their job, I think is a very similar phenomenon.

All of these unintended negative consequences.

it's one

of the reasons why, while I am not reflexively

supportive of the idea that we should sandbox all new AI.

So this is idea much like drug discovery,

where we should test it extensively before it gets released into the wild.

One of the reasons I don't believe in it is just, how are you going to stop

everyone at this university, much less everyone in the world?

From building AI?

There's no realistic constraint on it. But,

but it has implications.

And I alluded to one earlier, Google Maps

causing,

cognitive decline.

So one of the things we know is good for long term

cognitive health is navigating through space.

Well, if you're using an AI to tell you

when to make your turns, then you're not doing that anymore.

So here's how I use Google Maps.

So I'm not saying Google Maps is bad.

I get to be a native everywhere

I go, in every city in the world.

I can open up my phone and it can tell me how to get to where I want to go.

Not always the best route, but a good enough route.

And the way I use Google Maps is I open it up.

I look at its route, the way I use it at home when I'm in a new city.

I don't do terrible things to myself.

When I'm at home, I open it up.

I check out to make sure that there isn't a traffic jam or something,

and then I try and beat Google Maps.

It's given me its route and how long it thinks I will take.

And then I think, what do I know about Berkeley, California?

What do I know about San Francisco?

What do I know about LA?

That I can beat Google Maps without cheating, without speeding

and driving like a crazy person?

Then I have to think about it.

I got the benefits of the AI telling me how traffic is

and what the best route would be, but I'm still thinking

deeply about the problem and reaping those benefits.

So that's how I.

I wish the people building these systems

thought through them as much as I'm trying to.

The overwhelming ethic in AI development right now and technology

development writ large is make life easier.

I hope I convinced someone here today

that we should actually build technology to make life

more challenging in a productive way.

and that's what I'd love to see out of Waze in Google Maps.

and then the other one is,

I mean, the dirty truth is, yes,

because most of the AI working in

hiring today is bad.

It's lazy, it's unscientific, it's unproven,

and it's largely just looking for keywords on your resume.

Absolutely lutely.

You could have GPT

all by itself, read to your resume and help you optimize it.

Now, the very things that it would optimize

are the things we shouldn't be using and hiring.

What does the recruiter do when they look at a resume?

They give you five seconds.

Now look at your name, your school, and your last job.

and it turns out once I know almost anything about you,

none of those are meaningfully predictive anymore.

So I can be very realistic

about it and say, absolutely, Gemini and Gpt3 could help you

revise your resume, make it more likely to get hired.

But what it's going to do is implant,

aspects that recruiters respond to

that are not, in fact, predictive

of whether you're actually good at the job.

so feel free,

play the game on the system.

But in reality, here's what I tell all of my students

don't get a job with a resume.

Get them to come to you,

but build something.

in fact, this may seem a little off topic, but when we look at women and wage

gap penalties between men and women, when you turn that around

and you look at employees that are actively recruited,

wage gap quickly disappears.

When I'm not hiring a generic person to fill a job,

I'm hiring you.

Amazon wanted me to be

their chief scientist for people.

that is the only way I tell my students to get a job.

now, I also want to be realistic.

Most people don't have the privilege of doing it that way, but,

I strongly recommend

that you don't march to the front door with a resume.

You come in the back door

because people there wanted you to be there.

It's just it doesn't compare in terms of the career outcomes that emerged from it.

But genuinely, you are right in speculating, GPT

actually does a pretty spectacular job in polishing resumes.

Hey. Dave.

Dave. Hey.

It's the opening night.

Welcome.

For the significant full

inclusion for us, candidates are going to see a more.

At that point.

It's time once again, both keynote.

Okay.

And this includes nuclear waste and security removal

from us. And.

Nuclear security and social networking.

liquid metal enthusiast.

lot of us in the way meaningful

community.

No city.

This site could easily put.

Thousands of years in behind.

Put a.

Uniform on a good way to say,

Same. No.

I think

we missed the moment, but maybe we can.

I got about 15% of that begin with.

What is,

a kind of of.

Okay.

I can hear you now. Yes.

What changes? Yes.

You can go from moving

over to.

The to city and.

Guys,

may in 2013, at some point.

Okay.

Us one those that oppose.

And you know

me said can be the,

That these policy issues with us

take us to you for the final.

Year.

The California

entertainment.

Groups again to for.

You can us for to see some.

Unity in this game so.

Please have

for this is. A.

What that what we have to.

For missing

individuals that you cannot forget.

Who? Moses

said about me.

And I assume.

Yeah.

that's an excellent question.

And it's interesting because you frame it

well in terms of,

how many communities,

as you say, such as? Well, Hakka.

I do a lot of work in South Africa.

or, an India where there are just parts of the country

that are plugged into the global data stream

and parts that aren't,

and so

I think you're right.

It's not just an issue of, of inequality, but also that there's

this privileged position of being able to hold back sets of information.

It's interesting that the flip side also happens with, information

that might otherwise be available being expunged from the record.

I've visited China many times.

I think it's an amazing place, but they've made the decision

that certain kinds of information absolutely will never enter

their data sphere, information that ought to be there.

And so, in a sense, there's this complex

really of the data that's easy to get, the surface

level data, the data that people see as economically valuable,

you know, open AI was supposed to be a nonprofit helping the world

very quickly and demonstrably changed into something wildly different.

As soon as billions of dollars started flooding in from Microsoft.

So I,

as I was saying earlier, when I was talking about data trust,

I'm a firm believer in the idea

that these systems should be working for us.

there's plenty of wonderful

business opportunities for people building things.

They also they don't get to own me.

They don't get to turn my my books into knowledge that is available

not for me, but from, there I necessarily.

And there's this interesting question.

Also, I think one of the concerns is that,

you know, as people access these systems

and different language versions of large models

will be somewhat different, particularly

in, you know, say,

Mandarin Chinese versus English,

but there's also a lot of underlying model structure.

So there's a huge potential homogenization

that I will facilitate throughout the world.

And that's deeply problematic, cause us all to we have this amazing tool

that could help us to all think differently.

And it seems much more likely that in the near term at least,

it will cause us to all think more homogeneously.

even if we're speaking different

languages, I suspect it will induce people

to use similar patterns of thought and similar pieces of information.

Some of which may not even be true.

in that sense,

if we weren't talking about the private property of OpenAI or Google,

then we might feel like there's a real value in being able

to have a broader range of knowledge available in the world.

I would love to know

if someone living,

you know, much as their ancestors

did in the mountains of Borneo,

something that could make my life better, but I just don't have access to it.

It would be wonderful if somehow I could find that.

but I think you're on to a pretty profound

challenge.

in these spaces, most of the AI I built,

has been about solving a specific problem.

A specific model solves a specific problem.

Even in the domains in which I have a lens.

large language models in my work,

they're more like an interface.

You can talk to them so that our suicide,

risk prevention model could act, or you could talk to them

to help us identify and diagnose postpartum depression. But

our actual

diagnoses aren't through the large language model.

They are through a specialized model.

And what we find is we have to develop, in

many cases, a different model for every group of people

that we're working with, even within one notional community.

So even if we're just talking about Mexico, absolutely,

we would have different models in the North in the South,

different models for the cities versus rural areas.

different model,

clearly different models for Spanish speaking and non Spanish speaking.

Mexicans,

all of which have to be there.

when we build models for our education system,

in the States, all originally in English,

we quickly found that the way it spoke to people had to be distinctly different

for different populations, or they just wouldn't engage with it.

so again and again, in

most of my work, we're looking at

trying to, well, describe a specific problem

and building a specific mean machine learning or artificial intelligence tool

that could be used in that space, often use

by a person,

say being used by a doctor or nurse or teacher,

sometimes working completely on its own.

and in that sense, I think you're identifying

some of the challenges with existing

large language models is and again,

it doesn't have to be this way,

but it is the way that it currently is

that they tend to just suck everything in

and make very little distinction within them.

That does not exist within the corpus of language

they've been trained on, or the corpus of images.

A diffusion model has been trained on.

And it would actually be amazing

if these systems actually understood these differences.

So one of the emerging areas of, research,

is developing causal modeling in AI.

And this takes a different forms.

Probably the most common right now

is less exciting to me, but is very exciting in places like OpenAI

and Google is combining reinforcement learning models with lens.

You stick them together and you create an independent agent

that can go out and collect its own data about the world.

but that runs into the some of the very problems

you're talking about in terms of, well, maybe people didn't want to share that

data with an agent, or they didn't know they were interacting with an agent.

And also fair to what you were saying, in some places in the world,

how would such an agent actually get access to that information physically?

How could it be present to get information

in a place that isn't connected, into the net?

And again, AI is not

AI as much as anyone have a say in this, but I know more than anyone

should I have the say of how these things happen.

So I love questions like that because it brings in a very different

maybe there are parts of our selves in our lives

that we want to preserve.

and preserve in a way

that isn't simply just more data points

in some giant companies database.

But I also, in my heart, feel like,

how are the ways that we can share that my sister and brother in law

go to Wallach, every year and hike around

and spend time in villages, and they absolutely love it.

but, not everyone

just does.

Not everyone wants to share their data with the world.

Not everyone can travel to other parts of the world to experience it.

And finding that

not even a dividing line, maybe something messy and fuzzy

between what we keep for ourselves and what we share with the world,

and how we want to make certain that that's respected.

And I think that's one of the big things.

Are those distinctions being respected?

And I don't have to think

the Sam Altman is a bad guy.

I don't happen to think he's a great guy, but I don't happen to think

he's a bad guy, that his opinion alone should decide these questions.

These are things

we collectively should decide, and that just isn't happening right now.

So that is unfortunately the best

flailing answer I can offer to you.

In the back.

Okay, great.

Thank.

Hey. Thank you again for

the insightful, for sharing your thoughts on this.

we. I some,

work for me. for.

And to see you, they start to encourage

us to use this part of our space.

So, I was just,

so that I, you know, what be I sharing that,

we actually may spend.

it's also like, in,

this work.

So right now.

I, I may be today, reading it,

and then we might.

but I think that I still have some.

kind of a favorite, at least from what I do.

These three days.

So if you have any surprises

on the species or the price of the weekend,

if you go to

get out,

for the days we think that the price,

just for us can be developments or,

some, some things as basic as, like

operations or.

Yeah. And, you know,

yeah.

so I don't have

specific courses to recommend, but,

I will say, though admittedly not what you're asking for,

but which are already doing, just experimenting with it.

again, I'm really encouraging people to engage and explore.

having said that, I think some of the best sources

for interesting insights about these sorts of technologies.

I highly recommend MIT Technology Review.

is a good source, and they have published some articles

about specifically about using large language models and,

and thoughts and recommendations around that.

I have a newsletter.

You're welcome to sign up to it.

I don't specialize in talking about LMS, but I certainly have included a couple

of specific, episodes of that newsletter in which I went through people using it in

for education purposes, and for,

even for financial modeling purposes

and recommendations around how to get the most out of it.

and it turns out there are actually a fair number of,

how to write with GPT videos on YouTube right now.

I wouldn't recommend them because I think they're very much

of the vein of it's going to speed it all up and make it so fast.

Let's write a book in a day.

And that's taking the exact wrong lesson out of this.

So steer away from those sorts of things.

Feel free to look at it's all free on my site.

Feel free to look

and see some of the recommendations I've had in my past newsletters,

and check out Technology Review.

I'm not an MIT grad.

I decided to go to a place where people are,

collaborated with each other instead of stabbing each other in the back.

But it's still an amazing place with brilliant people, and

it's a good source of info.

Thank you.

Thank you very much.

so some of those things to me today, so we'll say thank you

to people. So

in the code society,

Something about.

Small business.

Thank you.

Thank you very much for your time.

Thanks.