Original subtitles

The future unfolds before our eyes...

but is it always beyond our grasp?

What was once the province of the gods

has now come more clearly into view

through mathematics and data.

Out of some early observations about gambling

arose tools that guide

our scientific understanding of the world

and more...

through the power of prediction.

From our decisions about the weather...

The strongest hurricane ever on record...

To finding someone lost at sea...

Commencing search pattern.

Keep a good look out!

Every day mathematics and data combine

to help us envision what might be...

It's the best crystal ball that humankind can have.

Take a trip on the wings of probability

into the future.

We're thinking about luck

or misfortune,

but they just basically are a question of math, right?

"Prediction by the Numbers" --

right now, on "NOVA."

The Orange County Fair, held in Southern California.

In theory, these crowds hold a predictive power

that can have startling accuracy,

but it doesn't belong to any individual, only the group.

And even then, it has to be viewed

through the lens of mathematics.

The theory is known as the "wisdom of crowds,"

a phenomenon first documented about a hundred years ago.

Statistician Talithia Williams is here

to see if the theory checks out, and to spend some time

with the fair's most beloved animal,

Patches, a 14-year-old ox.

It was a fair kind of like this one

where, in 1906,

Sir Francis Galton came across a contest

where you had to guess the weight of an ox,

like Patches you see here behind me.

After the ox weight-guessing contest was over,

Galton took all the entries home and analyzed them statistically.

To his surprise,

while none of the individual guesses were correct,

the average of all the guesses

was off by less than one percent.

That's the wisdom of crowds.

But is it still true?

So, here's how I think we can test that today.

What if we ask a random sample of people here at the fair

if they can guess how many jelly beans they think are in the jar.

And then, we take those numbers and average them

and see if that's actually close

to the true number of jelly beans.

Guess how many jelly beans are in here.

Come on, guys, everybody's got to have their guess.

I see your mind churning.

1,227.

846.

Probably like 925?

I think a thousand.

So just write your number down.

Uh huh, there you go.

Can I have a jelly bean?

The 135 guesses gathered from the crowd vary wildly.

The range of our guesses was,

from the smallest was 183, the largest was 12,000.

So you can tell, folks were really guessing.

But when we take the average of our guesses, we get 1,522.

So the question is,

how close is our average to the actual number of jelly beans?

Well, now's the moment of truth.

All right, so the real number of jelly beans was 1,676.

The average of our guesses was off by less than ten percent.

So there actually was some wisdom in our crowd.

Though off by about ten percent,

the average of the crowd's estimates

was still more accurate

than the vast majority of the individual guesses.

Even so, the wisdom of crowds does have limits.

It can be easily undermined by outside influences

and tends to work best on questions with clear answers,

like a number.

The steps Talithia took

reflect a process going on all around us these days

in the work of statisticians.

Thanks, everybody.

So we collected this data,

right, we analyzed it mathematically,

and we got an estimate that was pretty close

to the actual true value.

That's math and statistics at work.

We didn't always use math and statistics to make predictions.

The Romans studied the flights and cries of birds.

The Chinese cracked "oracle" bones with a hot metal rod

and read the results

19th-century Russians used chickens.

Throughout history, we've sought the future

in moles on people's faces,

clouds in the sky,

or a pearl cast into an iron pot.

And that list of things used for predicting goes on and on.

But more recently-- that is the last couple hundred years--

to see into the future, we've turned to science

and made some remarkable predictions

from the existence of Neptune,

or radio waves,

or black holes,

to the future location of a comet with such precision

we could land a space probe on it.

But if you pop the hood of science,

inside you'll find a field of applied mathematics

that's made many of those predictions possible:

statistics.

Statistics is kind of unique.

It's not an empirical science itself, but it's not pure math,

but it's not philosophy either.

It's the framework, the language,

the rules by which we do science.

From that, we can make decisions,

we can make conclusions, we can make predictions.

That's what... that's what statisticians try to do.

Why I love statistics is that

it predicts the likelihood of future occurrences,

which really means it's the best crystal ball

that humankind can have.

Ultimately, all the predictive power of statistics

rests on a revolutionary insight from about 500 years ago--

that chance itself can be tamed

through the mathematics of probability.

Viva Las Vegas!

Here's a city full of palaces

built on understanding probability

and fueled by gambling,

which may seem a funny place

to find mathematician Keith Devlin.

But mathematics and gambling

have been tied together for centuries.

Today in a casino, you'll find roulette,

slot machines,

blackjack.

Playing craps is also known as "rolling the bones,"

which is more accurate than you might think.

Humans have been gambling

since the beginnings of modern civilization.

The ancient Greeks, the ancient Egyptians,

would use the ankle bones of sheep as a form of early dice.

Surprisingly, while the Greeks laid the foundation

for our mathematics, they didn't spend any effort

trying to analyze games of chance.

It seems to have never occurred to them,

or indeed to anybody way up until the 15th, 16th century,

that you could apply mathematics

to calculate the way these games would come out.

16th-century Italian mathematician Gerolamo Cardano

made a key early observation:

that the more times a game of chance is played,

the better mathematical probability

predicts the outcome,

later proven as the law of large numbers.

Examples of the law of large numbers at work surround us.

When I flip this coin,

we have no way of knowing

whether it's going to come up heads or tails.

That time it was heads.

On the other hand, if I were to toss a coin 100 times,

roughly 50% of the time it would come up heads,

and 50% of the time it would come up tails.

We can't predict a single toss.

We can predict the aggregate behavior over a 100 tosses.

That's the law of large numbers.

In fact, casinos are a testament

to the iron hand of the law of large numbers.

The games are designed

to give the casinos a slight edge over the gambler.

Take American roulette:

on the wheel are the numbers one through 36,

half red and half black.

Betting a dollar on one color or the other

seems like a 50-50 proposition.

But the wheel also has two green slots with zeros.

If the ball lands in those,

the casino wins all the bets on either red or black.

And that's the kind of edge

that makes the casino money over the long run.

Customers are gambling.

The casino is absolutely not gambling.

Because they may lose money,

they may lose a lot of money to one or two players,

but if you have thousands and thousands of players,

by the law of large numbers,

you are guaranteed to make money.

The law of large numbers

comes into play outside of gambling too.

In basketball, a field goal or shooting percentage

is the number of baskets made

divided by the number of shots taken.

But early in the season,

when it's based on a low number of attempts,

that percentage can be misleading.

At the beginning of the season, a less skilled player

might get off a few lucky shots in a row.

And at that point,

they'd have a super-high shooting percentage.

Meanwhile, a very skilled player

might miss a few at the beginning of the season

and have a low shooting percentage.

But as the season goes on,

and the total number of shots climbs,

their shooting percentages will soon reflect

their true skill level.

That's the law of large numbers at work.

A small sample, like just a few shots, can be deceptive,

while a large sample, like a lot of shots,

gives you a better picture.

The gambling observations that led to the law of large numbers

were a start, but what really launched probability theory

and opened up a door to a whole new way

of thinking about the future,

was a series of letters

exchanged between two French mathematicians,

Blaise Pascal and Pierre de Fermat in the 1650s,

about another gambling problem that had been kicking around

for a few centuries.

A simplified version of the problem goes like this:

two players-- let's call them Blaise and Pierre--

are flipping a coin.

Blaise has chosen heads, and Pierre tails.

The game is the best of 5 flips,

and each has put money into the pot.

They flip the coin three times, and Blaise is ahead two to one.

But then the game is interrupted.

What is the fair way to split the pot?

The question is: how do they divide up the pot

so that it's fair to what might have happened

if they'd been able to complete the game.

Fermat suggested imagining the possible future outcomes

if the game had continued.

There are just two more coin flips,

creating four possible combinations.

Heads-heads, heads-tails, tails-heads, and tails-tails.

In the first three, Blaise wins with enough heads.

Pierre only wins in the last case,

so Fermat suggested that a three-to-one split

was the correct solution.

The key breakthrough

was imagining the future, mathematically,

something even Pascal had trouble with.

Because what Fermat did was say, "Let's look into the future,

"look at possible futures,

"and we'll count the way things could have happened

in different possible futures."

It was a simple arithmetic issue,

but the idea of counting things in the future

was just completely new,

and Pascal couldn't wrap his mind around it.

Eventually Pascal accepted Fermat's solution,

as did others,

and today, that exchange of letters is regarded

as the birth of modern probability theory.

People realized the future wasn't blank.

You didn't know exactly what was going to happen,

but you could calculate with great precision

what the likelihood of things happening were.

You could make all of the predictions

we make today and take for granted.

You could make them using mathematics.

It was a fundamental insight,

and one of the doors that led to the modern world.

Inherent in all our attempts to predict the future--

from the stock market

to insurance

to web retailers trying to figure out

what you might buy next,

is the idea that with the right data,

the likelihood of future events can be calculated.

In fact, one of the great success stories

in the science of prediction

yields a forecast that many of us check every day

to answer the question, "Do I need an umbrella

or a storm shelter?"

The hurricane season of 2017

will be remembered for its ferocity and destruction.

The strongest ever on record...

The Puerto Rico and the San Juan that we knew yesterday

is no longer there.

The storms formed and gained in intensity with surprising speed,

leaving forecasters to emphasize the uncertainty

of where they might land.

Maria is now a Category Three hurricane.

Exactly what it's going to look like, we just don't know yet.

There's still great uncertainty...

In weather forecasting,

the only certainty is uncertainty.

One thing we know

for sure is we cannot give you a perfect forecast.

Given the nature of how we make a forecast,

from the global observations to equations running on computers,

stepping out in time,

I don't think there'll ever be a perfect forecast.

To fight that uncertainty,

forecasters have turned to more data-- lots more data.

Here at the National Weather Service

Baltimore-Washington office,

meteorologist Isha Renta

prepares for the afternoon launch of a weather balloon.

Twice a day, every day,

all across the U.S. and around the world,

at the very same time,

balloons are released to take a package of instruments

up through the atmosphere.

It transmits readings about every ten meters in height.

It's my understanding that they have developed

other ways to get vertical profiles of the atmosphere,

but still the accuracy and the resolution

that the weather balloon will give you is a lot higher.

So that's why we still depend on them.

The data from Isha's weather balloon ends up

at the National Center for Environmental Prediction

in College Park, Maryland,

the starting point for nearly all weather forecasts

in the United States.

Her information becomes one drop in a very large bucket

of data taken in each day.

Temperature, pressure, wind speed,

and direction in the atmosphere.

Tens of thousands of point observations are used

every hour of every day as kind of a starting point.

That's where we begin the simulation,

from those observations.

It all becomes part of a process,

which has been described

as one of the great intellectual achievements

of the 20th century: numerical forecasting.

The first step in numerical forecasting

is to break a nearly 40-mile- thick section of the atmosphere

into a three-dimensional grid.

Then, each grid point is assigned numerical values

for different aspects of the weather,

based on the billions of measurements

continually pouring into the Weather Service.

So you'll have an understanding

of temperature, pressure,

and values in terms of wind and wind direction

at each one of these points

within this grid that covers the globe.

From there, equations from the physics

of fluids and thermodynamics are applied to each grid point.

Not only do you change the characteristics

at each grid point, but the changes at those grid points

affect neighboring grid points,

and then neighboring grid points affect other grid points.

And so you evolve the atmosphere through time

in this three-dimensional space.

And remarkably, the approach works.

It's amazingly crazy that it works.

It's remarkable how well it does work,

given that we're making grand assumptions

about the initial state, so to speak,

or the beginning state of any forecast.

And that initial state turns out to be absolutely crucial.

In the early days of numerical forecasting,

it seemed like a definitive weather prediction

extending far into the future might soon be possible.

But research in the 1960s

showed that slight errors in measuring the initial state

grow larger over time, leading predictions astray.

So as you step ahead in time,

the forecast will become less accurate.

Ironically, that sensitivity to initial conditions

also suggested a way to improve the accuracy

of numerical weather forecasts.

Thanks to the power of today's computers,

forecasters can run their weather simulations not once

but several times.

For each run, they slightly alter the initial conditions

to reflect the inherent error built into the measurements

and the uncertainty in the model itself.

The process is called ensemble forecasting,

and the results are called spaghetti plots.

We're looking at about 100 different forecasts here

for the jet stream at about six days ago.

We have the actual jet stream

drawn as the white line on here today,

and you can see most of the forecasts six days ago

were well north of where we actually find the jet stream

this morning.

And then we'll go to a five-day forecast

and a four-day forecast and a three-day forecast

and then down to two days and the day of the event.

And you can see how the model forecasts all converge

on that solution, which is what you would expect them to do.

But you go back to the six-day forecast,

you can see the large spread in the ensemble solutions

for this particular pattern.

In the end, meteorologists turn to statistical tools

to analyze weather forecasts

and often use probabilities to express the uncertainty

in the results.

That's the "40% chance of rain" you might hear

from your local forecaster.

Meteorology is probabilistic at its very core,

and I believe that the general public knows

there is uncertainty inherent in everything we say,

but we're getting better.

Our forecasts for three days out now are as accurate

as one-day forecasts were about 10 years ago.

And this continues to improve.

So the science has advanced beyond my wildest dreams,

and it's hard to even see where it might go in the future.

Just like in meteorology,

for the rest of science,

the ultimate test of our understanding

is our ability to make accurate predictions.

On a grand scale, scientific theories

like Einstein's general theory of relativity

have to make predictions

that can be tested to become accepted.

In that case, it took four years

before a full solar eclipse revealed

that light passing near the sun curved,

just as predicted by Einstein's theory--

the first proof he was right

that the sun's mass distorts the fabric of space-time--

what we experience as gravity.

In fact, the scientific method demands a hypothesis

which leads to a prediction of results

from a carefully designed experiment

that will test its claim.

Surprisingly, it wasn't until the 1920s and '30s

that a British scientist, Ronald A. Fisher,

laid out guidelines for designing experiments

using statistics and probability as a way of judging results.

As an example, he told the story of a lady

who claimed to taste the difference

between milk poured into her tea

and tea poured into her milk.

Fisher considered ways to test that.

What if he presented her with just one cup to identify?

If she got it right one time, you'd probably,

"Well, yeah, but she had a 50-50 chance, just by guessing,

of getting it right."

So you'd be pretty unconvinced that she has the skill.

Fisher proposed that a reasonable test of her ability

would be eight cups,

four with milk into tea,

four with tea into milk,

each presented randomly.

The lady then had to separate them back into the two groups.

Why eight?

Because that produced 70 possible combinations

of the cups, but only one with them separated correctly.

If she got it right,

that wouldn't prove she had a special ability,

but Fisher could conclude, if she was just guessing,

it was an extremely unlikely result,

a probability of just 1.4 percent.

Thanks mainly to Fisher,

that idea became enshrined in experimental science

as the "p-value" -- p for probability.

If you assume your results were just due to chance,

that what you were testing had no effect,

what's the probability you would see those results

or something even more rare?

If you assume that there's a process

that is completely random,

and you find that it's pretty unlikely to get your data,

then you might be suspicious that something is happening.

You might conclude, in fact, that it's not a random process.

That this is interesting to look at what else might be going on,

and it passes some kind of sniff test.

Fisher also suggested a benchmark:

only experimental results

where the p-value was under .05--

a probability of less than five percent--

were worth a second look.

In other words,

if you assume your results were just due to chance,

you'd see them less than one time out of 20.

Not very likely.

He called those results "statistically significant."

Statistically significant.

Now this is a terrible word.

It could be quite insignificant.

You could be detecting a very, very, very small effect,

but it would be called, in the mathematical lingo,

"significant."

Since Fisher's day,

p-values have been used as a convenient yardstick for success

by many, including most scientific journals.

Since they prefer to publish successes,

and getting published is critical to career advancement,

the temptation to massage and manipulate experimental data

into a good p-value is enormous.

There's even a name for it: "p-hacking."

P-hacking is when researchers consciously or unconsciously

guide their data analysis to get the results that they want,

and since .05 is kind of the-the bar for being able to publish

and call something real, and get all your grant money,

it's usually guiding the results

so that you arrive at that p of .05.

How much p-hacking really goes on is hard to know.

What may be more important is to remember

what was originally intended by a p-value.

The p-value was always meant to be a detective, not a judge.

If you do an experiment

and find the result that is statistically significant,

that is telling you, that is an interesting place to look

and research and understand further what's going on,

not "don't study this anymore because the matter is settled."

In a sense, a low p-value is an invitation

to reproduce the experiment, to help validate the result,

but that doesn't always happen.

In fact, there are few career incentives for it.

Journals and funders prefer novel research.

There is no Nobel Prize for replication.

Another solution to p-hacking and the overemphasis on p-values

may simply be greater transparency.

More and more, what people are doing

is publishing their data.

And so it's becoming harder and harder to lie with statistics,

because people will just probe and say,

"Well, give me the set you analyzed

and let me see how you got this result."

Statistics continues to play a fundamental role in science,

but really anywhere data is collected,

you'll find statisticians are at work,

looking for patterns, drawing conclusions,

and often making predictions--

though they don't always work out.

The presidential election of 2016

was a tough one for pollsters,

the folks who conduct and analyze opinion polls.

Hillary Clinton was the overwhelming favorite

to beat Donald Trump right up to election day.

Trump is headed for a historic defeat.

He's going to lose by a landslide.

I think that she's going to have a very good night.

The "New York Times" put Trump's chances at 15%.

One pollster on election night gave him one percent.

A projection of a 99% chance of winning, is that correct?

The odds are overwhelming

of a Hillary Clinton victory on Tuesday.

I would be very surprised if anything else happened

And, of course, Trump won, and Clinton lost.

People were repeatedly told,

"Hillary Clinton is the candidate

most likely to win this election," and she didn't.

And I think that really left people feeling almost lied to,

almost cheated by these numbers.

So what was going on with the polls?

And exactly how do people predict elections?

One way is just by asking people who they'll vote for.

One of the great things about polling

is that we don't have to talk to everybody

in order to find out what the opinions are of everybody.

We can actually select something called a sample.

Sampling is a familiar idea.

To see if the soup is right,

you taste a teaspoon, not the whole pot.

To test your blood at the doctor,

they typically draw less than an ounce,

they don't drain you dry.

But in many circumstances,

finding a representative sample is harder than it sounds.

Let's suppose that this

is the population of about a thousand people in a city,

and we want to know, "Are people for or against

converting a park into a dog park?"

And so these green beads down here

are going to represent people who were for it,

and the red beads are folks who are against it.

Talithia's first step is to take advantage

of an unlikely ally in sampling: randomness.

The beauty of randomization

is that as long as you throw everything from your population

into one pot and randomly pull it out,

you can be sure that you're within

a certain percentage points

of the actual value that's in that pot.

So the plan is to randomly sample the beads-- but how many?

That depends on how much accuracy Talithia wants.

One measure is the margin of error--

the maximum amount the result from the sample

can be expected to differ from that of the whole population.

It's the plus or minus figure, often a percentage,

you see in the fine print in polls.

But there's also confidence level.

Inherently, there is uncertainty

that any sample really represents a whole population.

The confidence level tells you how sure you can be

about your result.

A 90% confidence level means,

on average, if you ran your poll or sample 100 times,

90 of those times, it would be accurate,

within the margin of error.

Talithia knows the total number of beads is a thousand.

And she's settled on a plus-or-minus five-percent

margin of error at a 90% confidence level.

That means she needs a sample size of at least 214 beads.

Here are the results:

We got a 103 red beads and 111 green,

so about 48% of our population would vote against,

and about 52% would vote for.

Now, remember that margin of error that we talked about,

that plus-or-minus five percent?

So once you take that into account,

those numbers really aren't that different at all.

So I guess you could say, this puppy is too close to call.

In fact, within the margin of error, the stats got it right.

There were an equal number of red and green beads in the jar.

While the sampling error built in from the mathematics

can be quantified,

there are other errors that can't.

The other parts of the error-- how we word our questions,

how the respondents feel that day,

the responsibility to predict

what their behavior is going to be somewhere down the line--

all those sources of error

are something that we can't calculate.

And there's a catch to random sampling for polls too.

A few decades back,

when just about every household had a landline,

finding a random sample meant randomly dialing phone numbers.

Into the 1970s and the 1980s, we were getting,

you know, 90% response rates.

If we randomly chose a phone number,

somebody on the other end of that phone would pick it up

and would do the interview with us.

Those days are over.

Thanks to caller I.D. and answering machines,

people often don't answer their landlines anymore--

if they even have one.

Response rates are way down.

Only about ten percent of people

respond to polls.

So you're kind of crossing your fingers

and hoping the people you reach

are the same as the ones that are actually going to vote.

For example, we found in 2016, pollsters were not reaching

enough white voters without college degrees.

If there's a bias in the data, you cannot recover from it.

As we've seen from some recent elections.

After Donald Trump's surprise win,

many wondered if polling was broken.

But if you look at the polls themselves,

and not the headlines,

on average, polls on the national and state level

were off by historically typical amounts.

So when I hear people say, "Oh, the polls were wrong,"

then it probably reflects people's interpretations

about the polls being wrong,

where people, for various reasons,

looked at the polls, and they said,

"These numbers prove to me that Clinton's going to win."

When we looked at the polls, we said,

"These numbers certainly make her a favorite,

"but they point toward an election that's fairly close

and quite uncertain, actually."

And in 2016,

the U .S. presidential election was just that close.

Trump's victory depended on fewer votes

than the seating capacity of some college football stadiums--

spread across three states:

Pennsylvania, Wisconsin, and Michigan.

And there were some problems with the polls in those states

that led to underestimating Trump's support,

according to a postmortem by a consortium of pollsters.

Nate Silver, the founder of the website FiveThirtyEight,

is one of the biggest names in polling--

even though he doesn't generally conduct polls.

Our job is to take other people's polls

and to translate that

in terms of a probability, to say basically whether--

um, who's ahead,

which is usually pretty easy to tell, um,

but then how certain or uncertain is the election

is the more difficult part.

Like a meteorologist,

Nate presents his predictions as probabilities.

On the morning of Election Day 2016,

he gave Clinton about a 70% chance of winning

and Trump about a 30% chance.

That's like rolling a ten-sided die

with seven sides that are Clinton

and three that are Trump.

People who make probabilistic forecasts,

they're not saying that politics is intrinsically random.

They're saying that we have imperfect knowledge of it,

and that if you think you can be more certain than that,

you're probably fooling yourself based on how accurate polls,

other types of political data are.

Ultimately, interpreting a probability

depends on the situation.

While a 30% chance might seem slim,

if you learned the flight you were about to board

crashed three out of every ten trips,

would you get on the plane?

As this plane only makes it to its destination

seven out of ten times,

please pay attention to our short safety briefing.

Or if a weather forecaster said

there's only a 30% chance of rain,

and then it rained--

would you care?

If it does rain,

no one demands to know, "Why did it rain?

We have to get to the bottom of this."

We can say like, "It just did."

It might have rained, it might not have rained.

As it happened, it did.

I do think there's a certain natural resistance

to seeing things that maybe we care about

more than whether it's going to rain or not,

like elections, in that same way.

As 2016 shows,

predicting who will win the U.S. presidency,

a one-time contest between two unique opponents,

is far from easy.

But in at least one field,

there are literally decades of detailed statistics

on how the contests played out--

baseball.

Baseball has always been a game of numbers--

box scores, batting averages, ERAs, RBIs.

But while stats have always been part of baseball,

in the last 20 years, their importance has skyrocketed

due to sports analytics,

the use of predictive models to improve a team's performance.

To some extent every business, not just sports,

is really trying to predict the next event, you know.

Whether you're on Wall Street,

or if you're in the tech business,

what's the new new thing.

And for us, it's future player performance.

Billy Beane was one of the first

to adopt the quantitative approach in the late '90s,

when he was the general manager of the Oakland Athletics.

Stuck with the low payroll of a small-market team,

he abandoned decades of subjective baseball lore

and committed the organization to using statistical analyses

to guide the team's decision-making.

It very much became a mathematical equation

putting together a baseball team.

Billy's stats-driven approach started to attract attention

when the Oakland A's finished in the playoffs

in four consecutive years

and set a league record with 20 wins in a row.

Then it was lionized

and even given a name in a best-selling book and movie,

"Moneyball."

Brad Pitt plays Billy.

If we win on our budget with this team,

we'll have changed the game.

While "Moneyballing" didn't lead

to a league championship for the Oakland A's,

it did change the game.

Today, every Major League Baseball team

has a sports analytics department,

trying to predict and enhance future player performance

through data,

analyzing everything

from the angle and speed of the ball coming off the bat--

to which players should be brought up

from the minor leagues or traded.

I'll never pretend to be a math whiz,

I just understand its powers and its application.

When you run a Major League Baseball team,

which is a great job,

and every kid who dreams of doing it,

I can tell you it's everything you've thought of.

But when they ask me,

"What do I have to do to do that?"

My answer is always the same.

I say, "Go study and get an A in math."

While sports analytics has transformed baseball,

Moneyballing has found its way into many unrelated fields.

Proponents of data-driven decision making and prediction

have applied the approach to areas as diverse

as popular music and law enforcement.

Moneyballing has been enabled

by the vast amounts of information

gathered through the internet, so-called "Big Data."

Our current output of data

is roughly 2.5 quintillion bytes a day.

But what about the opposite situation,

when there's very little data, yet actions need to be taken--

for example when searching for people lost at sea?

How do you even begin to predict where they might be?

The U.S. Coast Guard's Sector Boston Command Center.

From this secure set of rooms,

the Coast Guard coordinates all operations in the Boston area,

including national security, drug enforcement,

and search and rescue.

Good morning, Coast Guard Sector Boston Command Center,

Mr. Fleming speaking.

Uh, good morning, sir...

A caller reports

that a friend went paddleboarding

earlier in the morning, but he's now overdue.

The Coast Guard initiates a search

with a 45-foot response boat...

Engaging...

...out of Boston Harbor.

Coming up.

Unfortunately, a paddle craft in trouble

has grown increasingly common.

You are required to have a life jacket on.

The reason for that is in 2015,

I think we had 625 deaths nationwide--

a number of those people that were recovered

were recovered without a life jacket.

The Command Center also launches another boat

out of Station Point Allerton, in Hull.

Short tack disconnected.

Stand clear of lines.

The caller said the missing person typically paddled

between Nantasket Beach and Boston Light,

about three miles away.

But with all the unknowns--

where he got into trouble and how he may have drifted--

the search area could be as large as 20 square miles.

Search and rescue operations

are often based on unique circumstances

and require action, despite incomplete information.

To attack problems like that,

statisticians turn to an idea that originates

with an 18th-century English clergyman

interested in probability-- Thomas Bayes.

Imagine you are given a coin to flip,

and you want to know if it is fair, 50-50 heads or tails,

or weighted to land more on heads than tails.

The traditional approach in statistics and science

doesn't assume either answer and uses experiments to find out.

In this case that involves flipping the coin a lot.

Or you could approach the problem like a Bayesian.

Unlike traditional statistics,

that means starting with an initial probability

based on what you know.

In this case, all the coins you've ever come across

in a lifetime of flipping coins have been fair.

It seems likely this one is probably fair too.

Next, you also flip the coin,

updating the probability as you go.

Let's say it starts off with several heads in a row.

That might make you wonder,

increasing your probability estimate that it's weighted.

But as you flip it more times, those start to look like chance.

In the end, your best estimate

is that it is probably a fair coin,

but you are open to any new information.

Like it belongs to your uncle the con man, "Crooked Larry."

Sector 659.

Our estimated time of arrival is one-one-five-eight.

Bayesian inference creates a rigorous mathematical approach

to calculating probabilities based on new information.

And it sits at the heart of the Coast Guard's

Search and Rescue Optimal Planning System: SAROPS.

He's been missing since 7:30 this morning,

so I'm going to go ahead and do a SAROPS drift.

SAROPS takes information

about the last-known position of the object of the search...

What's the direction of the wind?

...along with the readings of currents and winds

and combines them with information

about how objects drift in the water

to simulate thousands of possible paths

the target may have taken.

These get processed into probabilities,

indicated by color,

and turned into search plans to be executed.

SAROPS is really a workhorse for the Coast Guard.

It does a lot of the calculations for us.

It provides us with a lot of valuable search patterns

and search-planning options.

I thought he was pretty far off shore but, you know,

he said he was okay, so I kept going.

Word of the search has spread.

A boater calls in a sighting from earlier in the day.

What I did is I went in and put that information into SAROPS,

and it changed everything.

SAROPS quickly recalculates all the probabilities

and generates a new search plan.

The area has shifted about three miles farther out to sea.

We are on-scene, commencing search pattern now.

Keep a good look out.

Roger, coming up.

We're assessing the situation on scene.

Any object you see in the water, please take a closer look at.

Paddleboarder, port side

Roger, we have located a paddleboarder

with zero-one person on board.

Off the port corridor!

Starboard side.

I have a visual

All right.

As it turns out, the search has been a drill.

Hours earlier, the paddleboard was placed in the water

by another Coast Guard ship and allowed to drift.

The instruments mounted on it are there to measure wind

and record the path it's taken,

information that will later be used

to tweak the drift simulations in SAROPS,

though the system performed quite well today.

The object was right in the middle of our search patterns.

So SAROPS was actually dead-on accurate

in predicting where we needed to search

to find the missing paddleboarder.

To be able to call a family and say,

"Your family and friends is coming home,"

is absolutely a call

that all of us should have the chance to make,

and, fortunately, because of stuff like this,

we do get to make that call.

The computational complexity of updating probabilities

held the Bayesian approach back for most of the 20th century.

But today's computing power has unleashed it on the world.

It's in everything from your spam filter

to the way Google searches work to self-driving cars.

Some even find in the Bayesian embrace of probability,

similarities to how we learn from experience.

And they've built it into computers,

Making it part of a powerful new force:

machine learning.

In the past, when we programmed computers,

we tended to really write down, in excruciating detail,

a set of rules that would tell the computer

what to do in every single contingencies.

But there's another approach--

to treat the computer

like a child learning to ride a bike.

No one teaches a child to ride using a set of rules.

There may be some tips,

but ultimately, it is trial and error--

experience-- that's the instructor.

The new thing, the new kid on the block

is machine learning,

specifically something called deep learning.

Here, we don't inform the computer of the rules,

but through examples.

So similar to, like, a small child

that falls down and learns from this experience,

we just let the computer learn from examples.

Suppose you want to train a computer

to recognize pictures of cats.

By scanning through thousands of labeled pictures--

some cats, some not--

the computer can develop its own guidelines

for assessing the probability that a picture is a cat.

And these days

computers are doing far more than just looking for cats.

Some of the best computers now

can learn how to beat the world's best Go champion

or to discover documents in stacks of documents,

work that highly paid lawyers normally do,

or diagnose diseases.

At Stanford, we recently ran a study

to understand whether a machine-learning algorithm

can compete with top-notch, Stanford-level,

board-certified dermatologists

in spotting things like skin cancer.

And lo and behold, we found that our machine-learning algorithm,

our little box,

is as good as the best human doctor in finding skin cancer.

That raises a lot of questions:

should we trust software over our doctors?

Or are diagnostic programs like Sebastian's

the intelligent medical assistants of tomorrow,

a new tool but not a substitute?

And there are other concerns.

If you asked a person riding a bike exactly how they do it,

they'd be hard-pressed to put it into words.

The same is true

with so-called "black box" machine learning applications

like Sebastian's:

no one, including Sebastian,

knows how it detects skin cancer.

Like the bicyclist, it just does,

which may be fine for diagnostic software,

but not for other aspects of medicine,

like treatment decisions.

If what you're doing is deciding

what dose of chemotherapy to give a patient,

I think most people would be uncomfortable

with that being a black box.

People would want to understand

where those predictions are coming from.

The same can be true

for evaluating who should get a home loan,

or who should get fired from their job for poor performance,

or who gets paroled,

all situations

in which black box machine learning software are in use.

These are algorithms

that can have a big effect on people's lives.

And we have to understand, as a society,

what is going into those algorithms

and what they're based on,

in order to make sure that they're not perpetuating

social problems that we already have.

We live in an age when the fusion of data, computers,

probability, and statistics

grants us more predictive power than we've ever known before.

We can see the tangible benefits,

and some of the dangers,

while also wondering where this will all go.

We're really seeing a new science of statistics

developing under our feet.

That's exciting,

and I think it must be a little bit like

what it was like when the theory of probability

was first being developed

by Pascal and Fermat and people around them,

that people were sort of saying,

"My God, these are questions that mathematics

can really have something to say about."

I think that must have been what it was like

when statistics in its traditional form

was being developed in the first part of the 20th century,

and suddenly people were just asking

whole new kinds of questions

that they couldn't even have approached before.

And I think we're having another moment like that now.

While tomorrow will always remain uncertain,

mathematics will continue to guide the way,

through the power of probability,

and prediction by the numbers.