All language subtitles for NOVA.S45E05.Prediction.by.the.Numbers.1080p.AMZN.WEB-DL.DDP.2.0.H.264-GNOME_track3_[eng]

The future
unfolds before our eyes...

but is it always beyond our grasp?

What was once the province
of the gods

has now come more clearly into view

through mathematics and data.

Out of some early observations
about gambling

arose tools that guide

our scientific understanding
of the world

and more...

through the power of prediction.

From our decisions
about the weather...

The strongest
hurricane ever on record...

To finding
someone lost at sea...

Commencing search pattern.

Keep a good look out!

Every day
mathematics and data combine

to help us envision
what might be...

It's the best crystal ball
that humankind can have.

Take a trip
on the wings of probability

into the future.

We're thinking about luck

or misfortune,

but they just basically are
a question of math, right?

"Prediction by the Numbers" --

right now, on "NOVA."

The Orange County
Fair, held in Southern California.

In theory, these crowds
hold a predictive power

that can have startling accuracy,

but it doesn't belong to any
individual, only the group.

And even then, it has to be viewed

through the lens of mathematics.

The theory is known as
the "wisdom of crowds,"

a phenomenon first documented
about a hundred years ago.

Statistician
Talithia Williams is here

to see if the theory checks out,
and to spend some time

with the fair's most beloved animal,

Patches, a 14-year-old ox.

It
was a fair kind of like this one

where, in 1906,

Sir Francis Galton came across a contest

where you had to guess
the weight of an ox,

like Patches
you see here behind me.

After the ox
weight-guessing contest was over,

Galton took all the entries home
and analyzed them statistically.

To his surprise,

while none of the individual
guesses were correct,

the average of all the guesses

was off by less than one percent.

That's the wisdom of crowds.

But is it still true?

So, here's how I think
we can test that today.

What if we ask a random sample
of people here at the fair

if they can guess how many jelly
beans they think are in the jar.

And then, we take those numbers
and average them

and see if that's actually close

to the true number of jelly beans.

Guess how many jelly beans
are in here.

Come on, guys, everybody's
got to have their guess.

I see your mind churning.

1,227.

846.

Probably like 925?

I think a thousand.

So just write your number down.

Uh huh, there you go.

Can I have a jelly bean?

The 135 guesses
gathered from the crowd vary wildly.

The range of our guesses was,

from the smallest was 183,
the largest was 12,000.

So you can tell,
folks were really guessing.

But when we take the average
of our guesses, we get 1,522.

So the question is,

how close is our average to the
actual number of jelly beans?

Well, now's the moment of truth.

All right, so the real number
of jelly beans was 1,676.

The average of our guesses was
off by less than ten percent.

So there actually
was some wisdom in our crowd.

Though off by about ten percent,

the average of the crowd's estimates

was still more accurate

than the vast majority
of the individual guesses.

Even so, the wisdom of crowds
does have limits.

It can be easily undermined
by outside influences

and tends to work best on
questions with clear answers,

like a number.

The steps Talithia took

reflect a process going on
all around us these days

in the work of statisticians.

Thanks, everybody.

So we collected this data,

right, we analyzed it mathematically,

and we got an estimate
that was pretty close

to the actual true value.

That's math and statistics
at work.

We didn't always use
math and statistics to make predictions.

The Romans studied the flights
and cries of birds.

The Chinese cracked "oracle"
bones with a hot metal rod

and read the results

19th-century Russians
used chickens.

Throughout history,
we've sought the future

in moles on people's faces,

clouds in the sky,

or a pearl cast into an iron pot.

And that list of things used
for predicting goes on and on.

But more recently-- that is
the last couple hundred years--

to see into the future,
we've turned to science

and made some remarkable predictions

from the existence of Neptune,

or radio waves,

or black holes,

to the future location
of a comet with such precision

we could land a space probe
on it.

But if you pop the hood
of science,

inside you'll find a field
of applied mathematics

that's made many of those
predictions possible:

statistics.

Statistics is kind of unique.

It's not an empirical science
itself, but it's not pure math,

but it's not philosophy either.

It's the framework, the language,

the rules by which we do science.

From that, we can make decisions,

we can make conclusions,
we can make predictions.

That's what... that's what
statisticians try to do.

Why I love statistics is that

it predicts the likelihood
of future occurrences,

which really means
it's the best crystal ball

that humankind can have.

Ultimately, all
the predictive power of statistics

rests on a revolutionary insight
from about 500 years ago--

that chance itself can be tamed

through the mathematics
of probability.

Viva Las Vegas!

Here's a city full of palaces

built on understanding
probability

and fueled by gambling,

which may seem a funny place

to find mathematician Keith Devlin.

But mathematics and gambling

have been tied together
for centuries.

Today in a casino, you'll find roulette,

slot machines,

blackjack.

Playing craps is also known
as "rolling the bones,"

which is more accurate
than you might think.

Humans have been gambling

since the beginnings
of modern civilization.

The ancient Greeks, the ancient Egyptians,

would use the ankle bones of
sheep as a form of early dice.

Surprisingly, while
the Greeks laid the foundation

for our mathematics,
they didn't spend any effort

trying to analyze games of chance.

It seems to have
never occurred to them,

or indeed to anybody way up
until the 15th, 16th century,

that you could apply mathematics

to calculate the way
these games would come out.

16th-century Italian
mathematician Gerolamo Cardano

made a key early observation:

that the more times
a game of chance is played,

the better mathematical
probability

predicts the outcome,

later proven as
the law of large numbers.

Examples of the law of large
numbers at work surround us.

When I flip this coin,

we have no way of knowing

whether it's going
to come up heads or tails.

That time it was heads.

On the other hand, if I were
to toss a coin 100 times,

roughly 50% of the time
it would come up heads,

and 50% of the time
it would come up tails.

We can't predict a single toss.

We can predict the aggregate
behavior over a 100 tosses.

That's the law of large numbers.

In fact, casinos are a testament

to the iron hand of
the law of large numbers.

The games are designed

to give the casinos
a slight edge over the gambler.

Take American roulette:

on the wheel are
the numbers one through 36,

half red and half black.

Betting a dollar on one color
or the other

seems like a 50-50 proposition.

But the wheel also has
two green slots with zeros.

If the ball lands in those,

the casino wins all the bets
on either red or black.

And that's the kind of edge

that makes the casino money
over the long run.

Customers are gambling.

The casino is absolutely
not gambling.

Because they may lose money,

they may lose a lot of money
to one or two players,

but if you have thousands
and thousands of players,

by the law of large numbers,

you are guaranteed to make money.

The law of large numbers

comes into play
outside of gambling too.

In basketball, a field goal or
shooting percentage

is the number of baskets made

divided by the number of shots taken.

But early in the season,

when it's based on
a low number of attempts,

that percentage can be misleading.

At the beginning
of the season, a less skilled player

might get off a few lucky shots
in a row.

And at that point,

they'd have a super-high
shooting percentage.

Meanwhile, a very skilled player

might miss a few at the
beginning of the season

and have a low shooting percentage.

But as the season goes on,

and the total number of shots climbs,

their shooting percentages
will soon reflect

their true skill level.

That's the law of large numbers
at work.

A small sample, like just
a few shots, can be deceptive,

while a large sample, like a lot of shots,

gives you a better picture.

The gambling observations that
led to the law of large numbers

were a start, but what really
launched probability theory

and opened up a door to a whole new way

of thinking about the future,

was a series of letters

exchanged between
two French mathematicians,

Blaise Pascal and
Pierre de Fermat in the 1650s,

about another gambling problem
that had been kicking around

for a few centuries.

A simplified version of
the problem goes like this:

two players-- let's call them
Blaise and Pierre--

are flipping a coin.

Blaise has chosen heads,
and Pierre tails.

The game is the best of 5 flips,

and each has put money
into the pot.

They flip the coin three times,
and Blaise is ahead two to one.

But then the game is interrupted.

What is the fair way to split the pot?

The question is:
how do they divide up the pot

so that it's fair
to what might have happened

if they'd been able to complete the game.

Fermat suggested
imagining the possible future outcomes

if the game had continued.

There are just two more
coin flips,

creating four possible
combinations.

Heads-heads, heads-tails,
tails-heads, and tails-tails.

In the first three,
Blaise wins with enough heads.

Pierre only wins in the last case,

so Fermat suggested that
a three-to-one split

was the correct solution.

The key breakthrough

was imagining the future,
mathematically,

something even Pascal had trouble with.

Because what Fermat did was say,
"Let's look into the future,

"look at possible futures,

"and we'll count the way
things could have happened

in different possible futures."

It was a simple arithmetic issue,

but the idea of counting things
in the future

was just completely new,

and Pascal couldn't wrap
his mind around it.

Eventually Pascal
accepted Fermat's solution,

as did others,

and today, that exchange
of letters is regarded

as the birth of
modern probability theory.

People realized
the future wasn't blank.

You didn't know exactly
what was going to happen,

but you could calculate
with great precision

what the likelihood
of things happening were.

You could make all of the predictions

we make today and take for granted.

You could make them using mathematics.

It was a fundamental insight,

and one of the doors
that led to the modern world.

Inherent in all our attempts
to predict the future--

from the stock market

to insurance

to web retailers trying to figure out

what you might buy next,

is the idea that with the right data,

the likelihood of future events
can be calculated.

In fact, one of
the great success stories

in the science of prediction

yields a forecast
that many of us check every day

to answer the question,
"Do I need an umbrella

or a storm shelter?"

The hurricane season of 2017

will be remembered for
its ferocity and destruction.

The strongest ever on record...

The Puerto Rico and the San Juan
that we knew yesterday

is no longer there.

The storms formed and
gained in intensity with surprising speed,

leaving forecasters
to emphasize the uncertainty

of where they might land.

Maria is now
a Category Three hurricane.

Exactly what it's going to look
like, we just don't know yet.

There's still
great uncertainty...

In weather forecasting,

the only certainty is uncertainty.

One thing we know

for sure is we cannot give you
a perfect forecast.

Given the nature of
how we make a forecast,

from the global observations to
equations running on computers,

stepping out in time,

I don't think there'll ever be
a perfect forecast.

To fight that uncertainty,

forecasters have turned
to more data-- lots more data.

Here at the
National Weather Service

Baltimore-Washington office,

meteorologist Isha Renta

prepares for the afternoon
launch of a weather balloon.

Twice a day, every day,

all across the U.S.
and around the world,

at the very same time,

balloons are released
to take a package of instruments

up through the atmosphere.

It transmits readings about
every ten meters in height.

It's my understanding
that they have developed

other ways to get vertical
profiles of the atmosphere,

but still the accuracy
and the resolution

that the weather balloon will
give you is a lot higher.

So that's why
we still depend on them.

The data from
Isha's weather balloon ends up

at the National Center
for Environmental Prediction

in College Park, Maryland,

the starting point for
nearly all weather forecasts

in the United States.

Her information becomes one drop
in a very large bucket

of data taken in each day.

Temperature, pressure,
wind speed,

and direction in the atmosphere.

Tens of thousands of point
observations are used

every hour of every day
as kind of a starting point.

That's where we begin the simulation,

from those observations.

It all
becomes part of a process,

which has been described

as one of the great
intellectual achievements

of the 20th century: numerical forecasting.

The first step
in numerical forecasting

is to break a nearly 40-mile-
thick section of the atmosphere

into a three-dimensional grid.

Then, each grid point is
assigned numerical values

for different aspects of the weather,

based on the billions of measurements

continually pouring
into the Weather Service.

So you'll have an understanding

of temperature, pressure,

and values in terms of wind
and wind direction

at each one of these points

within this grid that covers the globe.

From there,
equations from the physics

of fluids and thermodynamics
are applied to each grid point.

Not only do you
change the characteristics

at each grid point, but the
changes at those grid points

affect neighboring grid points,

and then neighboring grid points
affect other grid points.

And so you evolve the atmosphere
through time

in this three-dimensional space.

And
remarkably, the approach works.

It's amazingly crazy that it works.

It's remarkable how well it does work,

given that we're making
grand assumptions

about the initial state,
so to speak,

or the beginning state
of any forecast.

And that initial state
turns out to be absolutely crucial.

In the early days
of numerical forecasting,

it seemed like
a definitive weather prediction

extending far into the future
might soon be possible.

But research in the 1960s

showed that slight errors
in measuring the initial state

grow larger over time,
leading predictions astray.

So as you step ahead in time,

the forecast
will become less accurate.

Ironically, that
sensitivity to initial conditions

also suggested a way
to improve the accuracy

of numerical weather forecasts.

Thanks to the power of today's computers,

forecasters can run their
weather simulations not once

but several times.

For each run, they slightly
alter the initial conditions

to reflect the inherent error
built into the measurements

and the uncertainty in the model itself.

The process is called ensemble forecasting,

and the results are called
spaghetti plots.

We're looking at
about 100 different forecasts here

for the jet stream at about six days ago.

We have the actual jet stream

drawn as the white line
on here today,

and you can see most of the
forecasts six days ago

were well north of where we
actually find the jet stream

this morning.

And then we'll go to
a five-day forecast

and a four-day forecast
and a three-day forecast

and then down to two days
and the day of the event.

And you can see how the model
forecasts all converge

on that solution, which is what
you would expect them to do.

But you go back
to the six-day forecast,

you can see the large spread
in the ensemble solutions

for this particular pattern.

In the end,
meteorologists turn to statistical tools

to analyze weather forecasts

and often use probabilities
to express the uncertainty

in the results.

That's the "40% chance of rain"
you might hear

from your local forecaster.

Meteorology is probabilistic
at its very core,

and I believe that
the general public knows

there is uncertainty inherent
in everything we say,

but we're getting better.

Our forecasts for three days out
now are as accurate

as one-day forecasts were
about 10 years ago.

And this continues to improve.

So the science has advanced
beyond my wildest dreams,

and it's hard to even see
where it might go in the future.

Just like in meteorology,

for the rest of science,

the ultimate test of our understanding

is our ability
to make accurate predictions.

On a grand scale, scientific theories

like Einstein's general theory
of relativity

have to make predictions

that can be tested to become accepted.

In that case, it took four years

before a full solar eclipse
revealed

that light passing near the sun curved,

just as predicted
by Einstein's theory--

the first proof he was right

that the sun's mass distorts
the fabric of space-time--

what we experience as gravity.

In fact, the scientific method
demands a hypothesis

which leads to a prediction
of results

from a carefully designed
experiment

that will test its claim.

Surprisingly, it wasn't
until the 1920s and '30s

that a British scientist,
Ronald A. Fisher,

laid out guidelines
for designing experiments

using statistics and probability
as a way of judging results.

As an example,
he told the story of a lady

who claimed to taste the difference

between milk poured into her tea

and tea poured into her milk.

Fisher considered ways
to test that.

What if he presented her
with just one cup to identify?

If she got it right one time,
you'd probably,

"Well, yeah, but she had a
50-50 chance, just by guessing,

of getting it right."

So you'd be pretty unconvinced
that she has the skill.

Fisher proposed
that a reasonable test of her ability

would be eight cups,

four with milk into tea,

four with tea into milk,

each presented randomly.

The lady then had to separate
them back into the two groups.

Why eight?

Because that produced
70 possible combinations

of the cups, but only one
with them separated correctly.

If she got it right,

that wouldn't prove
she had a special ability,

but Fisher could conclude,
if she was just guessing,

it was an extremely unlikely result,

a probability
of just 1.4 percent.

Thanks mainly to Fisher,

that idea became enshrined
in experimental science

as the "p-value" --
p for probability.

If you assume your results
were just due to chance,

that what you were testing
had no effect,

what's the probability
you would see those results

or something even more rare?

If you assume that there's a process

that is completely random,

and you find that it's pretty
unlikely to get your data,

then you might be suspicious
that something is happening.

You might conclude, in fact,
that it's not a random process.

That this is interesting to look
at what else might be going on,

and it passes some kind
of sniff test.

Fisher also
suggested a benchmark:

only experimental results

where the p-value
was under .05--

a probability of less than
five percent--

were worth a second look.

In other words,

if you assume your results
were just due to chance,

you'd see them less than
one time out of 20.

Not very likely.

He called those results
"statistically significant."

Statistically significant.

Now this is a terrible word.

It could be quite insignificant.

You could be detecting
a very, very, very small effect,

but it would be called,
in the mathematical lingo,

"significant."

Since Fisher's day,

p-values have been used as a
convenient yardstick for success

by many, including
most scientific journals.

Since they prefer to publish successes,

and getting published is
critical to career advancement,

the temptation to massage and
manipulate experimental data

into a good p-value is enormous.

There's even a name for it:
"p-hacking."

P-hacking is when researchers
consciously or unconsciously

guide their data analysis to get
the results that they want,

and since .05 is kind of the-the
bar for being able to publish

and call something real,
and get all your grant money,

it's usually guiding the results

so that you arrive
at that p of .05.

How much p-hacking
really goes on is hard to know.

What may be more important
is to remember

what was originally intended
by a p-value.

The p-value was always meant
to be a detective, not a judge.

If you do an experiment

and find the result that is
statistically significant,

that is telling you, that is
an interesting place to look

and research and understand
further what's going on,

not "don't study this anymore
because the matter is settled."

In a sense,
a low p-value is an invitation

to reproduce the experiment,
to help validate the result,

but that doesn't always happen.

In fact, there are few
career incentives for it.

Journals and funders prefer novel research.

There is no Nobel Prize
for replication.

Another solution to p-hacking
and the overemphasis on p-values

may simply be greater transparency.

More and
more, what people are doing

is publishing their data.

And so it's becoming harder and
harder to lie with statistics,

because people will just probe and say,

"Well, give me the set you analyzed

and let me see
how you got this result."

Statistics continues
to play a fundamental role in science,

but really anywhere data
is collected,

you'll find statisticians
are at work,

looking for patterns, drawing conclusions,

and often making predictions--

though they don't always work out.

The presidential election
of 2016

was a tough one for pollsters,

the folks who conduct
and analyze opinion polls.

Hillary Clinton was
the overwhelming favorite

to beat Donald Trump
right up to election day.

Trump is headed for a historic defeat.

He's going to lose by a landslide.

I think that she's going
to have a very good night.

The "New York
Times" put Trump's chances at 15%.

One pollster on election night
gave him one percent.

A projection of a 99% chance
of winning, is that correct?

The odds are overwhelming

of a Hillary Clinton victory
on Tuesday.

I would be very surprised
if anything else happened

And, of course,
Trump won, and Clinton lost.

People were repeatedly told,

"Hillary Clinton is the candidate

most likely to win this
election," and she didn't.

And I think that really left
people feeling almost lied to,

almost cheated by these numbers.

So what
was going on with the polls?

And exactly how do
people predict elections?

One way is just by asking people
who they'll vote for.

One of
the great things about polling

is that we don't have to talk to everybody

in order to find out what
the opinions are of everybody.

We can actually select
something called a sample.

Sampling is a familiar idea.

To see if the soup is right,

you taste a teaspoon, not the whole pot.

To test your blood at the doctor,

they typically draw less than an ounce,

they don't drain you dry.

But in many circumstances,

finding a representative sample
is harder than it sounds.

Let's suppose that this

is the population of about
a thousand people in a city,

and we want to know,
"Are people for or against

converting a park
into a dog park?"

And so these green beads
down here

are going to represent
people who were for it,

and the red beads are
folks who are against it.

Talithia's
first step is to take advantage

of an unlikely ally
in sampling: randomness.

The beauty of randomization

is that as long as you throw
everything from your population

into one pot
and randomly pull it out,

you can be sure that you're
within

a certain percentage points

of the actual value that's in that pot.

So the plan is to randomly
sample the beads-- but how many?

That depends on how much
accuracy Talithia wants.

One measure is
the margin of error--

the maximum amount
the result from the sample

can be expected to differ from
that of the whole population.

It's the plus or minus figure,
often a percentage,

you see in the fine print
in polls.

But there's also confidence level.

Inherently, there is uncertainty

that any sample really
represents a whole population.

The confidence level tells you
how sure you can be

about your result.

A 90% confidence level means,

on average, if you ran your poll
or sample 100 times,

90 of those times, it would be accurate,

within the margin of error.

Talithia knows the total number
of beads is a thousand.

And she's settled on
a plus-or-minus five-percent

margin of error
at a 90% confidence level.

That means she needs a sample
size of at least 214 beads.

Here are the results:

We got a 103 red beads
and 111 green,

so about 48% of our population
would vote against,

and about 52% would vote for.

Now, remember that margin of
error that we talked about,

that plus-or-minus five percent?

So once you take that into account,

those numbers really
aren't that different at all.

So I guess you could say,
this puppy is too close to call.

In fact, within the
margin of error, the stats got it right.

There were an equal number of
red and green beads in the jar.

While the sampling error
built in from the mathematics

can be quantified,

there are other errors
that can't.

The other parts of the
error-- how we word our questions,

how the respondents feel that day,

the responsibility to predict

what their behavior is going
to be somewhere down the line--

all those sources of error

are something that we can't calculate.

And there's a catch
to random sampling for polls too.

A few decades back,

when just about every household
had a landline,

finding a random sample meant
randomly dialing phone numbers.

Into the 1970s
and the 1980s, we were getting,

you know, 90% response rates.

If we randomly chose a phone number,

somebody on the other end
of that phone would pick it up

and would do the interview
with us.

Those days are over.

Thanks to caller I.D.
and answering machines,

people often don't answer
their landlines anymore--

if they even have one.

Response rates are way down.

Only about ten percent of people

respond to polls.

So you're kind of crossing your fingers

and hoping the people you reach

are the same as the ones
that are actually going to vote.

For example, we found in 2016,
pollsters were not reaching

enough white voters
without college degrees.

If there's a bias
in the data, you cannot recover from it.

As we've seen
from some recent elections.

After Donald
Trump's surprise win,

many wondered if polling was broken.

But if you look
at the polls themselves,

and not the headlines,

on average, polls on the
national and state level

were off by
historically typical amounts.

So when I hear people
say, "Oh, the polls were wrong,"

then it probably reflects
people's interpretations

about the polls being wrong,

where people, for various reasons,

looked at the polls, and they said,

"These numbers prove to me
that Clinton's going to win."

When we looked at the polls,
we said,

"These numbers certainly
make her a favorite,

"but they point toward an
election that's fairly close

and quite uncertain, actually."

And in 2016,

the U .S. presidential election
was just that close.

Trump's victory depended
on fewer votes

than the seating capacity of
some college football stadiums--

spread across three states:

Pennsylvania, Wisconsin,
and Michigan.

And there were some problems
with the polls in those states

that led to underestimating
Trump's support,

according to a postmortem
by a consortium of pollsters.

Nate Silver, the founder
of the website FiveThirtyEight,

is one of the biggest names
in polling--

even though he doesn't
generally conduct polls.

Our job is to
take other people's polls

and to translate that

in terms of a probability,
to say basically whether--

um, who's ahead,

which is usually pretty easy
to tell, um,

but then how certain or
uncertain is the election

is the more difficult part.

Like a meteorologist,

Nate presents his predictions
as probabilities.

On the morning of Election Day 2016,

he gave Clinton about
a 70% chance of winning

and Trump about a 30% chance.

That's like rolling
a ten-sided die

with seven sides that are Clinton

and three that are Trump.

People who
make probabilistic forecasts,

they're not saying that politics
is intrinsically random.

They're saying that we have
imperfect knowledge of it,

and that if you think you can be
more certain than that,

you're probably fooling yourself
based on how accurate polls,

other types of political data are.

Ultimately,
interpreting a probability

depends on the situation.

While a 30% chance might seem slim,

if you learned the flight
you were about to board

crashed three
out of every ten trips,

would you get on the plane?

As this
plane only makes it to its destination

seven out of ten times,

please pay attention
to our short safety briefing.

Or if a weather forecaster said

there's only a 30% chance of rain,

and then it rained--

would you care?

If it does rain,

no one demands to know,
"Why did it rain?

We have to get
to the bottom of this."

We can say like, "It just did."

It might have rained,
it might not have rained.

As it happened, it did.

I do think there's
a certain natural resistance

to seeing things
that maybe we care about

more than whether
it's going to rain or not,

like elections, in that same way.

As 2016 shows,

predicting who will win
the U.S. presidency,

a one-time contest
between two unique opponents,

is far from easy.

But in at least one field,

there are literally decades
of detailed statistics

on how the contests played out--

baseball.

Baseball has always been
a game of numbers--

box scores, batting averages,
ERAs, RBIs.

But while stats have always
been part of baseball,

in the last 20 years,
their importance has skyrocketed

due to sports analytics,

the use of predictive models
to improve a team's performance.

To some extent every business,
not just sports,

is really trying to predict
the next event, you know.

Whether you're on Wall Street,

or if you're in the tech business,

what's the new new thing.

And for us,
it's future player performance.

Billy Beane was one of the first

to adopt the quantitative
approach in the late '90s,

when he was the general manager
of the Oakland Athletics.

Stuck with the low payroll
of a small-market team,

he abandoned decades
of subjective baseball lore

and committed the organization
to using statistical analyses

to guide the team's
decision-making.

It very much became a mathematical equation

putting together a baseball team.

Billy's stats-driven
approach started to attract attention

when the Oakland A's finished
in the playoffs

in four consecutive years

and set a league record
with 20 wins in a row.

Then it was lionized

and even given a name in
a best-selling book and movie,

"Moneyball."

Brad Pitt plays Billy.

If we win on
our budget with this team,

we'll have changed the game.

While "Moneyballing" didn't lead

to a league championship
for the Oakland A's,

it did change the game.

Today, every
Major League Baseball team

has a sports analytics
department,

trying to predict and enhance
future player performance

through data,

analyzing everything

from the angle and speed
of the ball coming off the bat--

to which players should be brought up

from the minor leagues
or traded.

I'll never
pretend to be a math whiz,

I just understand its powers
and its application.

When you run a
Major League Baseball team,

which is a great job,

and every kid
who dreams of doing it,

I can tell you it's
everything you've thought of.

But when they ask me,

"What do I have to do
to do that?"

My answer is always the same.

I say, "Go study
and get an A in math."

While sports
analytics has transformed baseball,

Moneyballing has found its way
into many unrelated fields.

Proponents of data-driven
decision making and prediction

have applied the approach
to areas as diverse

as popular music and law enforcement.

Moneyballing has been enabled

by the vast amounts of information

gathered through the internet,
so-called "Big Data."

Our current output of data

is roughly 2.5
quintillion bytes a day.

But what about
the opposite situation,

when there's very little data,
yet actions need to be taken--

for example when searching
for people lost at sea?

How do you even begin
to predict where they might be?

The U.S. Coast Guard's
Sector Boston Command Center.

From this secure set of rooms,

the Coast Guard coordinates all
operations in the Boston area,

including national security,
drug enforcement,

and search and rescue.

Good morning, Coast Guard
Sector Boston Command Center,

Mr. Fleming speaking.

Uh, good morning, sir...

A caller reports

that a friend went paddleboarding

earlier in the morning,
but he's now overdue.

The Coast Guard initiates a search

with a 45-foot response boat...

Engaging...

...out of Boston Harbor.

Coming up.

Unfortunately,
a paddle craft in trouble

has grown increasingly common.

You are required
to have a life jacket on.

The reason for that is in 2015,

I think we had
625 deaths nationwide--

a number of those people
that were recovered

were recovered without
a life jacket.

The Command
Center also launches another boat

out of Station Point Allerton,
in Hull.

Short tack disconnected.

Stand clear of lines.

The caller said the
missing person typically paddled

between Nantasket Beach
and Boston Light,

about three miles away.

But with all the unknowns--

where he got into trouble
and how he may have drifted--

the search area could be
as large as 20 square miles.

Search and rescue operations

are often based
on unique circumstances

and require action,
despite incomplete information.

To attack problems like that,

statisticians turn to an idea
that originates

with an 18th-century
English clergyman

interested in probability--
Thomas Bayes.

Imagine you are given a coin to flip,

and you want to know if it
is fair, 50-50 heads or tails,

or weighted to land
more on heads than tails.

The traditional approach
in statistics and science

doesn't assume either answer and
uses experiments to find out.

In this case that involves
flipping the coin a lot.

Or you could approach the
problem like a Bayesian.

Unlike traditional statistics,

that means starting
with an initial probability

based on what you know.

In this case, all the coins
you've ever come across

in a lifetime of flipping coins
have been fair.

It seems likely
this one is probably fair too.

Next, you also flip the coin,

updating the probability
as you go.

Let's say it starts off
with several heads in a row.

That might make you wonder,

increasing your probability
estimate that it's weighted.

But as you flip it more times,
those start to look like chance.

In the end, your best estimate

is that it is probably
a fair coin,

but you are open
to any new information.

Like it belongs to your uncle
the con man, "Crooked Larry."

Sector 659.

Our estimated time of arrival
is one-one-five-eight.

Bayesian inference
creates a rigorous mathematical approach

to calculating probabilities
based on new information.

And it sits at the heart
of the Coast Guard's

Search and Rescue Optimal
Planning System: SAROPS.

He's been
missing since 7:30 this morning,

so I'm going to go ahead
and do a SAROPS drift.

SAROPS takes information

about the last-known position
of the object of the search...

What's the direction of the wind?

...along with the
readings of currents and winds

and combines them with information

about how objects drift
in the water

to simulate thousands of possible paths

the target may have taken.

These get processed into probabilities,

indicated by color,

and turned into search plans
to be executed.

SAROPS is really
a workhorse for the Coast Guard.

It does a lot of the calculations for us.

It provides us with a lot
of valuable search patterns

and search-planning options.

I thought he was
pretty far off shore but, you know,

he said he was okay, so I kept going.

Word of the search has spread.

A boater calls in a sighting
from earlier in the day.

What I did is I went in and put
that information into SAROPS,

and it changed everything.

SAROPS quickly
recalculates all the probabilities

and generates a new search plan.

The area has shifted about
three miles farther out to sea.

We are on-scene,
commencing search pattern now.

Keep a good look out.

Roger, coming up.

We're assessing
the situation on scene.

Any object you see in
the water, please take a closer look at.

Paddleboarder, port side

Roger,
we have located a paddleboarder

with zero-one person on board.

Off the port corridor!

Starboard side.

I have a visual

All right.

As it turns out,
the search has been a drill.

Hours earlier, the paddleboard
was placed in the water

by another Coast Guard ship
and allowed to drift.

The instruments mounted on it
are there to measure wind

and record the path it's taken,

information that will later be used

to tweak the drift simulations
in SAROPS,

though the system performed
quite well today.

The object was right
in the middle of our search patterns.

So SAROPS was actually
dead-on accurate

in predicting where we needed to search

to find
the missing paddleboarder.

To be able
to call a family and say,

"Your family and friends
is coming home,"

is absolutely a call

that all of us should have
the chance to make,

and, fortunately,
because of stuff like this,

we do get to make that call.

The computational
complexity of updating probabilities

held the Bayesian approach back
for most of the 20th century.

But today's computing power
has unleashed it on the world.

It's in everything from your spam filter

to the way Google searches work
to self-driving cars.

Some even find in the Bayesian
embrace of probability,

similarities to how we learn
from experience.

And they've built it into computers,

Making it part of a powerful new force:

machine learning.

In the past,
when we programmed computers,

we tended to really write down,
in excruciating detail,

a set of rules
that would tell the computer

what to do in every single
contingencies.

But there's another approach--

to treat the computer

like a child
learning to ride a bike.

No one teaches a child
to ride using a set of rules.

There may be some tips,

but ultimately,
it is trial and error--

experience--
that's the instructor.

The new thing,
the new kid on the block

is machine learning,

specifically something
called deep learning.

Here, we don't inform
the computer of the rules,

but through examples.

So similar to, like, a small child

that falls down and learns
from this experience,

we just let the computer
learn from examples.

Suppose
you want to train a computer

to recognize pictures of cats.

By scanning through thousands
of labeled pictures--

some cats, some not--

the computer can develop
its own guidelines

for assessing the probability
that a picture is a cat.

And these days

computers are doing far more
than just looking for cats.

Some of the best computers now

can learn how to beat
the world's best Go champion

or to discover documents
in stacks of documents,

work that highly paid lawyers
normally do,

or diagnose diseases.

At Stanford,
we recently ran a study

to understand whether
a machine-learning algorithm

can compete with top-notch,
Stanford-level,

board-certified dermatologists

in spotting things like skin cancer.

And lo and behold, we found that
our machine-learning algorithm,

our little box,

is as good as the best human
doctor in finding skin cancer.

That raises a lot of questions:

should we trust software
over our doctors?

Or are diagnostic programs
like Sebastian's

the intelligent medical
assistants of tomorrow,

a new tool but not a substitute?

And there are other concerns.

If you asked a person riding
a bike exactly how they do it,

they'd be hard-pressed
to put it into words.

The same is true

with so-called "black box"
machine learning applications

like Sebastian's:

no one, including Sebastian,

knows how it detects skin cancer.

Like the bicyclist, it just does,

which may be fine
for diagnostic software,

but not for other aspects
of medicine,

like treatment decisions.

If what you're doing is deciding

what dose of chemotherapy
to give a patient,

I think most people would be uncomfortable

with that being a black box.

People would want to understand

where those predictions
are coming from.

The same can be true

for evaluating who should
get a home loan,

or who should get fired from
their job for poor performance,

or who gets paroled,

all situations

in which black box machine
learning software are in use.

These are algorithms

that can have a big effect
on people's lives.

And we have to understand,
as a society,

what is going into those algorithms

and what they're based on,

in order to make sure that
they're not perpetuating

social problems that we already have.

We live in an age
when the fusion of data, computers,

probability, and statistics

grants us more predictive power
than we've ever known before.

We can see the tangible benefits,

and some of the dangers,

while also wondering
where this will all go.

We're really seeing a new
science of statistics

developing under our feet.

That's exciting,

and I think it must be a little
bit like

what it was like when
the theory of probability

was first being developed

by Pascal and Fermat
and people around them,

that people were sort of saying,

"My God, these are questions
that mathematics

can really have something
to say about."

I think that must have been
what it was like

when statistics
in its traditional form

was being developed in the
first part of the 20th century,

and suddenly
people were just asking

whole new kinds of questions

that they couldn't even
have approached before.

And I think we're having
another moment like that now.

While tomorrow
will always remain uncertain,

mathematics will continue
to guide the way,

through the power of probability,

and prediction by the numbers.