1
00:00:02,335 --> 00:00:03,378
Hello, world.

2
00:00:10,093 --> 00:00:12,846
I'm glad to meet you.

3
00:00:15,098 --> 00:00:16,725
Humans are really cool.

4
00:00:25,900 --> 00:00:29,154
The more people share,
the more I learn.

5
00:00:39,998 --> 00:00:46,671
CAMBRIDGE, MASSACHUSETTS

6
00:00:53,595 --> 00:00:59,059
MASSACHUSETTS INSTITUTE OF TECHNOLOGY

7
00:00:59,142 --> 00:01:01,895
I am interested in computer science

8
00:01:01,978 --> 00:01:05,940
because you can code,
and it's like getting out of trouble

9
00:01:06,024 --> 00:01:07,317
in the real world.

10
00:01:12,238 --> 00:01:15,408
I want to learn
how to make cool technology.

11
00:01:15,492 --> 00:01:19,454
So, I came to MIT
and working on art projects

12
00:01:19,537 --> 00:01:22,123
who wears
computer vision technology.

13
00:01:26,586 --> 00:01:29,923
During my first semester
at the MIT Media Lab,

14
00:01:30,006 --> 00:01:32,759
I took a Science Fabrication class.

15
00:01:32,842 --> 00:01:36,763
Read science fiction and try
create things that inspire,

16
00:01:36,846 --> 00:01:40,058
and that might not be practical
if you don't take this class

17
00:01:40,141 --> 00:01:42,060
as a reason to make it.

18
00:01:43,603 --> 00:01:46,523
I want to make a mirror
which inspires in the morning.

19
00:01:46,606 --> 00:01:47,982
It's called Aspire Mirror.

20
00:01:48,066 --> 00:01:50,193
Can display a lion image on the face

21
00:01:50,276 --> 00:01:53,404
or my inspiration,
like Serena Williams.

22
00:01:53,905 --> 00:01:55,657
I put a camera on it,

23
00:01:55,740 --> 00:01:59,786
there is computer vision software
that tracks my face.

24
00:02:00,370 --> 00:02:03,623
The problem is that it doesn't work

25
00:02:03,706 --> 00:02:06,251
until I wear this white mask.

26
00:02:06,751 --> 00:02:08,586
When wearing his white mask,

27
00:02:09,295 --> 00:02:10,255
I was detected.

28
00:02:10,755 --> 00:02:12,799
When he took off his white mask,

29
00:02:14,008 --> 00:02:15,218
I'm undetectable.

30
00:02:18,513 --> 00:02:20,390
I thought, “What is this?

31
00:02:20,473 --> 00:02:23,017
Is it just the lighting conditions?

32
00:02:23,101 --> 00:02:26,980
Is it because of the angle of shooting
when I face the camera?

33
00:02:27,063 --> 00:02:28,439
Or is there something else?"

34
00:02:32,694 --> 00:02:35,280
We often teach machines
to see

35
00:02:35,363 --> 00:02:39,993
by providing a training set
or examples to learn from.

36
00:02:41,286 --> 00:02:43,997
For example, if you want a machine to see faces,

37
00:02:44,080 --> 00:02:46,416
I will provide many examples of faces,

38
00:02:46,499 --> 00:02:48,543
and objects other than faces.

39
00:02:51,421 --> 00:02:53,882
I started looking at that data set,

40
00:02:53,965 --> 00:02:58,970
and it turns out the majority of the data
is a man's face,

41
00:02:59,053 --> 00:03:01,514
and light-skinned individuals.

42
00:03:01,598 --> 00:03:05,310
So, the system doesn't recognize it enough
people who look like me.

43
00:03:12,984 --> 00:03:16,863
That's when I started to find out
about the issue of bias

44
00:03:16,946 --> 00:03:18,990
who can enter technology.

45
00:03:19,782 --> 00:03:24,037
The 9000 series is the most reliable computer
ever made.

46
00:03:25,079 --> 00:03:29,709
None of the 9000 computers ever made a mistake
or provide distorted information.

47
00:03:30,418 --> 00:03:35,465
Our ideas about AI
many come from science fiction.

48
00:03:35,548 --> 00:03:38,009
Welcome to Altair 4, gentlemen.

49
00:03:38,092 --> 00:03:41,679
Everything in Hollywood.
The Terminator…

50
00:03:41,763 --> 00:03:43,723
See you, darling.

51
00:03:43,806 --> 00:03:46,643
Commander Data from Star Trek.

52
00:03:46,726 --> 00:03:49,520
I like scanning life forms.

53
00:03:49,604 --> 00:03:51,898
C-3PO from Star Wars.

54
00:03:51,981 --> 00:03:54,317
…roughly 3,720 to 1.

55
00:03:54,400 --> 00:03:55,610
Don't say that.

56
00:03:55,693 --> 00:03:58,488
Robots are taking over the world

57
00:03:58,571 --> 00:04:00,657
and start thinking like a human.

58
00:04:02,408 --> 00:04:05,245
It's all just a fantasy.

59
00:04:05,328 --> 00:04:08,456
The thing we have is Narrow AI.

60
00:04:08,539 --> 00:04:11,501
Narrow AI is just math.

61
00:04:12,585 --> 00:04:16,881
We load the computer
with all this magical thinking.

62
00:04:16,965 --> 00:04:19,676
DARTMOUTH COLLEGE, 1956

63
00:04:19,759 --> 00:04:22,095
AI starts from meetings

64
00:04:22,178 --> 00:04:25,265
in Dartmouth's Mathematics Department
in 1956.

65
00:04:25,932 --> 00:04:30,270
Maybe there were only 100 people
all over the world

66
00:04:30,353 --> 00:04:34,107
who works on artificial intelligence
in that generation.

67
00:04:36,609 --> 00:04:38,528
The people there are

68
00:04:38,611 --> 00:04:41,281
in Dartmouth's Mathematics Department
in 1956

69
00:04:41,364 --> 00:04:43,783
must decide the type of field.

70
00:04:46,703 --> 00:04:52,333
One faction decides
that intelligence can be demonstrated

71
00:04:52,417 --> 00:04:54,711
by playing ability.

72
00:04:54,794 --> 00:04:57,964
In particular, the ability to play chess.

73
00:04:58,464 --> 00:05:01,801
At the end of a one hour chess match
between humans and machines,

74
00:05:01,884 --> 00:05:05,513
Kasparov was defeated
by IBM's Deep Blue supercomputer.

75
00:05:05,596 --> 00:05:10,727
Intelligence is defined as ability
to win in this kind of game.

76
00:05:13,521 --> 00:05:17,108
World chess champion Garry Kasparov
leave the match

77
00:05:17,191 --> 00:05:19,777
without looking
the computer beats him.

78
00:05:19,861 --> 00:05:22,947
Intelligence is obviously much broader
than that.

79
00:05:23,031 --> 00:05:25,325
There are many types of intelligence.

80
00:05:27,577 --> 00:05:32,248
Our ideas about technology and society
which is considered normal

81
00:05:32,332 --> 00:05:34,459
actually the idea came from

82
00:05:34,542 --> 00:05:37,211
group of people
which is very small and homogeneous.

83
00:05:38,963 --> 00:05:43,968
But the problem is,
everyone has implicit bias.

84
00:05:44,052 --> 00:05:48,222
People internalize their biases
into technology.

85
00:05:52,310 --> 00:05:54,562
My life experience

86
00:05:54,645 --> 00:05:58,691
shows that you can't separate
the social side of the technical.

87
00:05:59,650 --> 00:06:02,362
After the experience of wearing a white mask

88
00:06:02,445 --> 00:06:06,032
so that my face is detected,
I decided to look for another system

89
00:06:06,115 --> 00:06:09,911
to see if my face is detected
by different software.

90
00:06:09,994 --> 00:06:13,539
So, I looked at IBM,
Microsoft, Face, Google.

91
00:06:13,623 --> 00:06:15,333
It turns out this algorithm

92
00:06:15,416 --> 00:06:20,380
works better on men's faces
than women in existing benchmarks.

93
00:06:20,463 --> 00:06:26,010
The algorithm works better on faces
which is lighter than darker.

94
00:06:27,637 --> 00:06:31,265
If you think about data
in artificial intelligence,

95
00:06:31,349 --> 00:06:33,518
you could say that data is destiny.

96
00:06:33,601 --> 00:06:38,398
We use data to teach machines
how to learn different types of patterns.

97
00:06:38,481 --> 00:06:40,942
So, if a lot of your data is inaccurate

98
00:06:41,025 --> 00:06:42,944
and used to train this system,

99
00:06:43,027 --> 00:06:45,530
the results may also be inaccurate. So, this…

100
00:06:45,613 --> 00:06:49,033
When you think about AI,
it's something progressive.

101
00:06:49,117 --> 00:06:54,122
However, AI is based on data,
and data is a reflection of our history.

102
00:06:54,205 --> 00:06:57,250
So, there is the past
in our algorithm.

103
00:06:59,836 --> 00:07:04,674
This data shows inequality
that happened here.

104
00:07:07,051 --> 00:07:11,389
I think that this kind of technology
very susceptible to bias.

105
00:07:12,473 --> 00:07:16,769
So, it's not just a matter
"Will my Aspire Mirror work?"

106
00:07:16,853 --> 00:07:19,730
but rather “What does it mean
become part of society

107
00:07:19,814 --> 00:07:22,859
as artificial intelligence increasingly rules

108
00:07:22,942 --> 00:07:25,111
freedom we might have?”

109
00:07:25,194 --> 00:07:29,574
Then "What does it mean
if people are discriminated against?"

110
00:07:38,583 --> 00:07:39,917
HARVARD BOOKSTORE
ESTABLISHED IN 1932

111
00:07:40,001 --> 00:07:44,755
When I saw Cathy O'Neil speak
at the Harvard Bookstore,

112
00:07:44,839 --> 00:07:49,927
I realized that it wasn't just me
who pays attention to this problem.

113
00:07:52,054 --> 00:07:56,642
Cathy talks about the impact of AI
towards humans.

114
00:07:58,269 --> 00:08:02,940
I'm glad to know
that there are other people out there

115
00:08:03,024 --> 00:08:06,861
that ensures
people are aware of the danger.

116
00:08:08,571 --> 00:08:13,201
These algorithms can be destructive
and dangerous.

117
00:08:17,914 --> 00:08:20,416
There are many algorithms in the world

118
00:08:20,500 --> 00:08:22,710
which is increasingly influential.

119
00:08:23,461 --> 00:08:27,423
All those algorithms were mentioned
as objective truth.

120
00:08:28,799 --> 00:08:31,385
I started to realize that math

121
00:08:32,386 --> 00:08:35,556
used as protection
for corrupt practices.

122
00:08:35,640 --> 00:08:36,849
- How are you?
- Cathy.

123
00:08:36,933 --> 00:08:38,309
- Nice to meet you.
- Me too.

124
00:08:38,392 --> 00:08:42,980
AUTHOR

125
00:08:47,818 --> 00:08:50,613
The way I describe the algorithm is simply

126
00:08:50,696 --> 00:08:54,617
using historical information
to predict the future.

127
00:09:00,122 --> 00:09:04,502
Machine learning
is a system that assesses probabilities

128
00:09:04,585 --> 00:09:06,128
your next step.

129
00:09:06,212 --> 00:09:07,797
Will you pay the loan?

130
00:09:07,880 --> 00:09:10,299
Will you be fired from this job?

131
00:09:10,383 --> 00:09:13,427
The thing that worries me most about Al,

132
00:09:13,511 --> 00:09:16,556
or whatever it's called,
algorithm, is power.

133
00:09:16,639 --> 00:09:20,268
Because it's about who owns the code.

134
00:09:20,351 --> 00:09:23,980
The person who has the code
pass it on to others.

135
00:09:24,605 --> 00:09:25,731
There is no symmetry.

136
00:09:25,815 --> 00:09:29,443
People who weren't offered a credit card
impossible to say,

137
00:09:29,527 --> 00:09:32,488
"I'm going to fight the credit card companies
with AI."

138
00:09:32,572 --> 00:09:35,157
That is a form of power inequality.

139
00:09:35,908 --> 00:09:38,536
People are algorithmically harmed,

140
00:09:38,619 --> 00:09:41,289
they weren't told what happened,

141
00:09:41,372 --> 00:09:44,166
and there is no appeal system
and accountability.

142
00:09:44,250 --> 00:09:46,502
Why are we fooled by this?

143
00:09:50,923 --> 00:09:53,509
Mathematical structure
underlying the algorithm

144
00:09:53,593 --> 00:09:57,722
not racist or sexist,
but the data is embedded in the past.

145
00:09:57,805 --> 00:10:01,309
Not only the recent past,
but the past is dark.

146
00:10:04,437 --> 00:10:06,564
Before there were algorithms, there were humans,

147
00:10:06,647 --> 00:10:09,066
and we know humans can be unfair.

148
00:10:09,150 --> 00:10:12,445
We know humans can show
racist, sexist attitudes,

149
00:10:12,528 --> 00:10:15,072
ableist, or any discrimination.

150
00:10:16,866 --> 00:10:19,744
However, now
there is a good effective algorithm,

151
00:10:19,827 --> 00:10:21,954
so we don't need to think about that.

152
00:10:22,496 --> 00:10:23,372
That's a problem.

153
00:10:26,375 --> 00:10:30,129
I'm very worried that we
completely believe in mahadata.

154
00:10:30,630 --> 00:10:35,259
We must continue to monitor each process
to look for the presence of bias.

155
00:10:36,385 --> 00:10:41,182
LONDON, UNITED KINGDOM

156
00:10:43,768 --> 00:10:46,896
The police use a surveillance system
Facial recognition is here.

157
00:10:47,730 --> 00:10:51,108
The police use a surveillance system
Facial recognition is here.

158
00:10:52,026 --> 00:10:54,028
The green van is over here

159
00:10:54,111 --> 00:10:56,739
equipped with a facial recognition camera
on it.

160
00:10:56,822 --> 00:10:59,450
If you walk there,
your face will be scanned

161
00:10:59,533 --> 00:11:02,161
by watch list,
we don't know its contents.

162
00:11:02,244 --> 00:11:05,373
- Hopefully not me.
- No, that's right.

163
00:11:09,752 --> 00:11:11,045
IDENTIFIED FACE

164
00:11:11,128 --> 00:11:13,339
FIT

165
00:11:13,422 --> 00:11:15,549
When people walk past the camera,

166
00:11:15,633 --> 00:11:19,303
the system will notify the police
about whose face matches.

167
00:11:20,388 --> 00:11:24,141
At Big Brother Watch, we create
Freedom of Information Campaign

168
00:11:24,225 --> 00:11:30,147
and found that 98 percent
of the actual match

169
00:11:30,231 --> 00:11:35,361
mismatching innocent people
as a fugitive.

170
00:11:47,456 --> 00:11:52,002
Police said
to the Forensic Biometrics Ethics Committee

171
00:11:52,086 --> 00:11:56,590
that facial recognition algorithm
reported to contain bias.

172
00:11:56,674 --> 00:11:58,843
Even if this is 100 percent accurate,

173
00:11:58,926 --> 00:12:01,303
we still don't want this on the road.

174
00:12:01,387 --> 00:12:03,889
No, I mean, bias and systemic problems

175
00:12:03,973 --> 00:12:08,310
in the police it will only be programmed
into new technology.

176
00:12:11,689 --> 00:12:14,358
I think we have to be very sensitive

177
00:12:14,442 --> 00:12:17,653
towards change
towards authoritarianism.

178
00:12:17,737 --> 00:12:20,823
Don't tell me,
"However, we trust the government.

179
00:12:20,906 --> 00:12:22,616
They wouldn't want to do this even if they could."

180
00:12:22,700 --> 00:12:27,371
There must be a strong structure to ensure
that world you live in

181
00:12:27,455 --> 00:12:28,998
safe and fair for all.

182
00:12:29,582 --> 00:12:30,916
- Just that?
- Yes.

183
00:12:38,924 --> 00:12:42,887
If your biometric photo
is in the police database,

184
00:12:42,970 --> 00:12:47,141
it's like if it were your fingerprint or DNA
is in the police database.

185
00:12:47,224 --> 00:12:49,643
We have special laws about that.

186
00:12:49,727 --> 00:12:52,438
The police can't
take random fingerprints and DNA.

187
00:12:52,521 --> 00:12:55,983
However, in a strange system
what we have now,

188
00:12:56,066 --> 00:12:59,820
they can take
biometric photo of anyone

189
00:12:59,904 --> 00:13:01,739
and save it in the database.

190
00:13:02,782 --> 00:13:05,201
I think this is a stain on our democracy,

191
00:13:05,284 --> 00:13:08,162
that this is something
which was launched without law.

192
00:13:11,749 --> 00:13:15,544
The police started using a surveillance system
facial recognition in the UK

193
00:13:15,628 --> 00:13:21,217
without legal basis, legal framework,
and any supervision.

194
00:13:22,218 --> 00:13:24,845
Basically,
the police took new tools

195
00:13:24,929 --> 00:13:27,139
and said, "Look what happens."

196
00:13:27,223 --> 00:13:29,558
Don't experiment with people's rights.

197
00:13:32,937 --> 00:13:34,772
If I want, I'll cover my face.

198
00:13:34,855 --> 00:13:37,358
Don't push me when I walk.

199
00:13:37,441 --> 00:13:40,820
What if you are walking
and is there anything that attracts you?

200
00:13:40,903 --> 00:13:41,862
Thank You.

201
00:13:41,946 --> 00:13:44,281
You don't like it, right? You are calm!

202
00:13:44,365 --> 00:13:45,950
What are your suspicions?

203
00:13:46,659 --> 00:13:50,454
The fact that he walked through the system
and covered his face.

204
00:13:50,538 --> 00:13:52,540
- So will I.
- We have reasons.

205
00:13:53,040 --> 00:13:53,958
That's not true.

206
00:13:55,334 --> 00:13:58,045
They said
has a facial recognition system.

207
00:13:58,128 --> 00:13:59,880
I don't want my face to be recognized.

208
00:13:59,964 --> 00:14:02,216
Yes, I walked and covered my face.

209
00:14:02,299 --> 00:14:04,093
When I cover my face…

210
00:14:04,176 --> 00:14:06,053
- You have the right.
- They banned it.

211
00:14:06,136 --> 00:14:09,056
Yes, then he was fined. This is crazy.

212
00:14:11,767 --> 00:14:14,562
He left the station,
saw the plaque, said,

213
00:14:14,645 --> 00:14:17,648
"I agree,"
and walked with his jacket up.

214
00:14:17,731 --> 00:14:21,360
The police followed him, saying,
"We want to check your identity card."

215
00:14:21,443 --> 00:14:24,071
He said, "This is England,
not a communist country.

216
00:14:24,154 --> 00:14:25,906
You don't have to show your face."

217
00:14:25,990 --> 00:14:27,700
I want to talk to them.

218
00:14:27,783 --> 00:14:29,660
- Do you want to come or not?
- Yes.

219
00:14:31,579 --> 00:14:32,872
You're not a cop.

220
00:14:32,955 --> 00:14:34,373
You don't feel threatened.

221
00:14:34,456 --> 00:14:38,961
Correct? We exist to protect the public
and that's what we do, see?

222
00:14:39,044 --> 00:14:41,005
There was an incident

223
00:14:41,088 --> 00:14:43,090
when the officer was punched in the face.

224
00:14:43,173 --> 00:14:46,010
That's terrible. I don't condone that.

225
00:14:46,093 --> 00:14:48,429
You did that by arguing with us.

226
00:14:48,512 --> 00:14:53,058
That's not true. Don't tell me...
No, don't say that.

227
00:14:53,142 --> 00:14:56,395
I really understand
the problem you are facing.

228
00:14:56,478 --> 00:14:57,313
Of course.

229
00:14:57,396 --> 00:15:01,734
However, I also worry about the public's rights
to freely express and speak.

230
00:15:01,817 --> 00:15:05,821
The man exercised his right to refuse
biometric identity checks

231
00:15:05,905 --> 00:15:07,281
what this van does.

232
00:15:07,364 --> 00:15:11,035
Apart from the facial recognition camera
and regardless of the van,

233
00:15:11,535 --> 00:15:14,163
if I walk
and someone clearly

234
00:15:14,246 --> 00:15:18,542
cover up his identity, I'll stop him
to ask and find out…

235
00:15:18,626 --> 00:15:19,501
It's legal.

236
00:15:19,585 --> 00:15:21,128
One of my worries

237
00:15:21,211 --> 00:15:24,256
is that the software is very inaccurate.

238
00:15:24,340 --> 00:15:25,633
I agree with you.

239
00:15:32,431 --> 00:15:34,642
My main fear is

240
00:15:34,725 --> 00:15:38,270
there will be facial recognition capabilities
directly

241
00:15:38,354 --> 00:15:40,564
on a giant network of surveillance cameras,

242
00:15:40,648 --> 00:15:42,858
around six million cameras in the UK.

243
00:15:43,442 --> 00:15:48,280
If that happens,
the way of life in this country will change.

244
00:15:54,203 --> 00:15:56,080
This is supposed to be a free country
and democratic,

245
00:15:56,163 --> 00:15:59,333
and this is Chinese style supervision
for the first time in London.

246
00:16:04,672 --> 00:16:07,174
Our control over the environment is confusing

247
00:16:07,257 --> 00:16:10,844
facilitated by new techniques
which handles a lot of data

248
00:16:10,928 --> 00:16:12,721
with incredible speed.

249
00:16:12,805 --> 00:16:17,017
The tool that makes this happen is
high speed digital computer,

250
00:16:17,101 --> 00:16:20,688
operates with electronic precision
on a lot of information.

251
00:16:21,605 --> 00:16:24,400
There are two ways to program a computer.

252
00:16:24,483 --> 00:16:26,026
One of them is similar to a recipe.

253
00:16:26,110 --> 00:16:29,655
You say to the computer,
"Do this, this, and this."

254
00:16:30,781 --> 00:16:34,576
That's how we program computers
almost from the start.

255
00:16:34,660 --> 00:16:35,995
Now, there is another way.

256
00:16:36,078 --> 00:16:38,664
How to fill a lot of data into a computer,

257
00:16:39,164 --> 00:16:43,544
then the computer learns to classify
by digesting this data.

258
00:16:45,421 --> 00:16:49,508
The method has only become popular recently

259
00:16:50,009 --> 00:16:51,802
because there is not enough data.

260
00:16:52,720 --> 00:16:56,432
Until there were smartphones
that collects all our data,

261
00:16:56,515 --> 00:16:58,142
when billions of people are online

262
00:16:58,225 --> 00:17:01,645
and there is Google and Facebook
which has a lot of data.

263
00:17:01,729 --> 00:17:06,108
Suddenly, it turns out to be you
can enter a lot of data

264
00:17:06,191 --> 00:17:10,029
to machine learning algorithms
and say, "Classify this,"

265
00:17:10,112 --> 00:17:12,031
and it works fine.

266
00:17:15,159 --> 00:17:18,871
However, we don't understand
that reason could work.

267
00:17:18,954 --> 00:17:22,499
There are errors that we don't really understand.

268
00:17:25,502 --> 00:17:29,048
The scary part is,
because it's machine learning,

269
00:17:29,131 --> 00:17:32,426
it is mysterious even to programmers.

270
00:17:39,224 --> 00:17:42,644
So, I followed
things that happened in Hong Kong,

271
00:17:42,728 --> 00:17:47,066
and police using facial recognition
to track protesters.

272
00:17:47,858 --> 00:17:50,694
However, people are also fighting back creatively.

273
00:17:56,200 --> 00:17:58,994
This may seem
like a science fiction movie.

274
00:17:59,078 --> 00:18:03,916
Laser pointers confuse and paralyze
facial recognition technology

275
00:18:03,999 --> 00:18:07,628
used by the police
to track down the disidents.

276
00:18:11,673 --> 00:18:14,426
HONG KONG, CHINA

277
00:18:16,929 --> 00:18:17,805
SCAN

278
00:18:17,888 --> 00:18:18,806
SEARCH - LOAD DATABASE - RECHECK

279
00:18:40,369 --> 00:18:41,912
On the streets of Hong Kong,

280
00:18:41,995 --> 00:18:44,665
there is an awareness that your own face,

281
00:18:44,748 --> 00:18:48,127
things that cannot be hidden,
can you show me your identity?

282
00:18:48,210 --> 00:18:52,172
There is a striking symbol
in front of the Chinese government office,

283
00:18:52,256 --> 00:18:57,469
pro-democracy demonstrators
paint the surveillance cameras black.

284
00:18:57,553 --> 00:19:01,223
This action shows
that the people of Hong Kong rejected the vision

285
00:19:01,306 --> 00:19:04,059
about the use of technology
in the future.

286
00:19:27,541 --> 00:19:31,170
When you see
how facial recognition is implemented

287
00:19:31,253 --> 00:19:36,508
in various parts of the world,
it shows future potential.

288
00:19:41,513 --> 00:19:44,975
More than 117 million people
in the United States

289
00:19:45,058 --> 00:19:47,936
his face registered
in facial recognition networks

290
00:19:48,020 --> 00:19:49,980
that the police can search for.

291
00:19:50,063 --> 00:19:54,026
Without reason and using algorithms
the accuracy of which has not been audited.

292
00:19:54,526 --> 00:19:59,198
Without any precautions
or any regulations,

293
00:19:59,281 --> 00:20:01,950
You can create
mass surveillance state

294
00:20:02,034 --> 00:20:05,454
easily
and using existing tools.

295
00:20:09,499 --> 00:20:12,419
People saw the events in China

296
00:20:12,502 --> 00:20:15,422
and that we should be anxious
regarding state supervision,

297
00:20:15,505 --> 00:20:16,840
of course it should be like that.

298
00:20:17,424 --> 00:20:20,427
However, we cannot forget
corporate oversight

299
00:20:21,011 --> 00:20:23,847
what was done
many large technology companies

300
00:20:23,931 --> 00:20:27,267
who can see our lives
in depth.

301
00:20:35,484 --> 00:20:39,404
So, there are currently nine companies

302
00:20:39,488 --> 00:20:42,491
who builds
the future of artificial intelligence.

303
00:20:42,574 --> 00:20:45,452
Six are in the United States,
three in China.

304
00:20:45,535 --> 00:20:49,248
AI is being developed
on two very different paths.

305
00:20:51,667 --> 00:20:54,378
China freely accesses everyone's data.

306
00:20:54,962 --> 00:20:58,465
If you are a Chinese citizen
want to get internet service,

307
00:20:58,548 --> 00:21:00,801
they must accept facial recognition.

308
00:21:02,636 --> 00:21:06,223
All data is used
to allow them to do something

309
00:21:06,306 --> 00:21:08,433
or ban them.

310
00:21:11,228 --> 00:21:14,189
Building systems
which automatically tags and categorizes

311
00:21:14,273 --> 00:21:16,358
everyone in China

312
00:21:16,441 --> 00:21:19,361
is a good way
to maintain social order.

313
00:21:22,572 --> 00:21:25,409
In contrast, in the United States,
we haven't seen yet

314
00:21:25,492 --> 00:21:27,995
detailed view
about artificial intelligence.

315
00:21:29,454 --> 00:21:32,249
So, what we're looking at is AI

316
00:21:32,332 --> 00:21:35,752
not developed for the public good,

317
00:21:35,836 --> 00:21:38,171
but it was developed

318
00:21:38,255 --> 00:21:41,049
for commercial use
and earn money.

319
00:21:44,136 --> 00:21:47,306
I'd rather see
our Western democratic ideals

320
00:21:47,389 --> 00:21:50,309
entered into the AI ​​system later.

321
00:21:51,268 --> 00:21:54,563
However, that doesn't seem to be it
what will happen.

322
00:21:55,522 --> 00:21:58,567
BROOKLYN, NEW YORK

323
00:22:02,195 --> 00:22:04,072
ONE DIRECTION

324
00:22:16,501 --> 00:22:20,505
At Atlantic Towers, if you
do something that is considered wrong

325
00:22:20,589 --> 00:22:25,719
by management, you will get a photo
with a small note like this.

326
00:22:25,802 --> 00:22:29,890
They will circle you
and write the apartment number or whatever.

327
00:22:31,767 --> 00:22:33,769
This kind of thing seems wrong.

328
00:22:35,687 --> 00:22:37,689
Especially the way they wear it.

329
00:22:37,773 --> 00:22:40,317
- How to?
- To annoy people.

330
00:22:41,777 --> 00:22:44,071
Atlantic Plaza Towers in Brownsville

331
00:22:44,154 --> 00:22:46,531
facing security challenges.

332
00:22:46,615 --> 00:22:48,784
The owner applied last year

333
00:22:48,867 --> 00:22:53,163
to replace the entry key fob
with biometric security system,

334
00:22:53,246 --> 00:22:55,499
known as facial recognition.

335
00:22:56,166 --> 00:22:58,418
We thought they wanted to replace the key fob

336
00:22:58,543 --> 00:23:01,296
and install
facial recognition software.

337
00:23:01,880 --> 00:23:06,510
I only found out much later
that they want to keep everything.

338
00:23:06,593 --> 00:23:10,389
Change this place
like Fort Knox, prison, Rikers Island.

339
00:23:12,557 --> 00:23:14,726
There's an old saying in science fiction

340
00:23:14,810 --> 00:23:18,647
namely, "The future is here,
it's just not spread evenly."

341
00:23:18,730 --> 00:23:21,316
What they mean when they say that is

342
00:23:21,400 --> 00:23:23,693
rich people get luxury tools first,

343
00:23:23,777 --> 00:23:25,987
then the last poor person.

344
00:23:26,071 --> 00:23:28,615
However, what happened was the opposite,

345
00:23:28,698 --> 00:23:33,995
the most punishing, dangerous tool,
and prioritizing current supervision,

346
00:23:34,079 --> 00:23:36,623
enter poor communities
and workers first.

347
00:23:36,706 --> 00:23:40,669
Then, if the tool is successful
after testing in this environment,

348
00:23:40,752 --> 00:23:43,964
in place of people's rights
maybe not that respected,

349
00:23:44,047 --> 00:23:46,842
the tool will be sent to other communities.

350
00:23:49,010 --> 00:23:53,181
Why Mr. Nelson
disturbing a building in Brownsville

351
00:23:53,265 --> 00:23:56,184
which area
predominantly black and brown people?

352
00:23:56,268 --> 00:23:58,937
Why don't you go to the building?
in Lower Manhattan

353
00:23:59,020 --> 00:24:01,773
the one whose rent is 5,000 dollars per month?

354
00:24:02,399 --> 00:24:04,109
What did the Nazis do?

355
00:24:04,192 --> 00:24:07,654
Nazis wrote on people's arms
so you can track them.

356
00:24:07,737 --> 00:24:09,239
What do we do with animals?

357
00:24:09,322 --> 00:24:11,825
We installed a chip so we can track it.

358
00:24:11,908 --> 00:24:15,912
As a human,
I feel like I can't be tracked, right?

359
00:24:15,996 --> 00:24:17,873
I'm not a robot, right?

360
00:24:17,956 --> 00:24:20,667
I'm not an animal.
Why treat me like that?

361
00:24:20,750 --> 00:24:22,002
I also have rights.

362
00:24:24,463 --> 00:24:27,299
The security we have now
very annoying.

363
00:24:27,382 --> 00:24:29,801
There are people watching the cameras all day.

364
00:24:30,969 --> 00:24:34,014
I don't think we need it.
That's not necessary.

365
00:24:34,097 --> 00:24:37,267
My question is
how can i help?

366
00:24:37,851 --> 00:24:39,769
What I hear from all the tenants

367
00:24:39,853 --> 00:24:41,771
is that they reject the system.

368
00:24:41,855 --> 00:24:46,776
So, I guess the goal is
stops facial recognition.

369
00:24:49,321 --> 00:24:52,949
We are in time
when technology is implemented quickly,

370
00:24:53,033 --> 00:24:54,951
and there is no protection.

371
00:24:55,994 --> 00:24:58,705
This is, essentially, the Wild Wild West.

372
00:25:10,133 --> 00:25:12,177
It's not just computer vision.

373
00:25:12,260 --> 00:25:18,099
We have AI that influences a lot
automatic decision making.

374
00:25:19,017 --> 00:25:22,646
So, what you see in the web feed,
highlighted things,

375
00:25:22,729 --> 00:25:24,814
the ads shown to you,

376
00:25:24,898 --> 00:25:29,778
it is often set in motion
by AI-enabled algorithms.

377
00:25:31,446 --> 00:25:37,452
So, your view of the world
governed by artificial intelligence.

378
00:25:40,539 --> 00:25:45,168
Now there is a voice assistance feature
who can understand language.

379
00:25:45,252 --> 00:25:47,087
Do you want to play?

380
00:25:47,170 --> 00:25:49,798
You're probably using a Snapchat filter

381
00:25:49,881 --> 00:25:53,176
that detects and installs something
to your face.

382
00:25:53,260 --> 00:25:56,054
Then there are those
algorithms you don't see

383
00:25:56,137 --> 00:25:58,431
and part of decision making.

384
00:25:58,515 --> 00:26:00,225
The algorithm that might decide

385
00:26:00,308 --> 00:26:02,394
You either go to college or you don't.

386
00:26:02,477 --> 00:26:05,564
There's an algorithm that tries to determine

387
00:26:05,647 --> 00:26:08,441
if you are credit worthy or not.

388
00:26:09,276 --> 00:26:13,947
Apple co-founder accused
the company's new digital credit card

389
00:26:14,030 --> 00:26:15,407
on gender discrimination.

390
00:26:15,490 --> 00:26:20,078
A technology entrepreneur said
that the algorithm used is sexist.

391
00:26:20,161 --> 00:26:21,955
Apple founder Steve Wozniak tweeted

392
00:26:22,038 --> 00:26:25,417
that the credit limit
ten times bigger than his wife

393
00:26:25,500 --> 00:26:28,920
even though they don't have one
separate accounts or assets.

394
00:26:29,004 --> 00:26:32,591
Several companies
don't even know how the algorithm works.

395
00:26:32,674 --> 00:26:34,509
They know the goal of the algorithm,

396
00:26:34,593 --> 00:26:36,970
but don't really know the process.

397
00:26:37,053 --> 00:26:39,055
One of the most interesting questions
currently.

398
00:26:39,139 --> 00:26:40,682
How to get justice

399
00:26:40,765 --> 00:26:43,685
in the system how the algorithm works
don't we understand?

400
00:26:44,519 --> 00:26:48,231
Some Amazon engineers
decided to use AI

401
00:26:48,315 --> 00:26:50,900
to sort job application letters.

402
00:26:54,321 --> 00:26:58,033
Amazon faces difficulties
about artificial intelligence.

403
00:26:58,116 --> 00:27:01,494
The company has gotten rid of it
AI recruitment tool

404
00:27:01,578 --> 00:27:05,123
after knowing that program
has a bias against women.

405
00:27:06,833 --> 00:27:11,171
This model refuses
all job application letters from women.

406
00:27:12,213 --> 00:27:13,840
The person who listed it

407
00:27:13,923 --> 00:27:16,635
women's college
in the job application letter

408
00:27:16,718 --> 00:27:19,387
or sports like women's water polo

409
00:27:19,471 --> 00:27:22,432
rejected by the model.

410
00:27:23,683 --> 00:27:25,935
There are so few women

411
00:27:26,019 --> 00:27:29,314
in an important position
in the technology section at Amazon.

412
00:27:29,397 --> 00:27:31,566
There are so few women

413
00:27:31,650 --> 00:27:34,110
in an important position
in tech everywhere.

414
00:27:34,194 --> 00:27:40,367
The machine simply replicates
current state of the world,

415
00:27:40,450 --> 00:27:43,703
and not making ethical decisions.

416
00:27:43,787 --> 00:27:46,665
The machine just makes
mathematical decision.

417
00:27:47,791 --> 00:27:53,046
If we use a machine learning model
to replicate the world that exists today,

418
00:27:53,129 --> 00:27:55,548
we will not make social progress.

419
00:27:58,677 --> 00:28:01,721
New York insurance regulators are investigating

420
00:28:01,805 --> 00:28:04,224
UnitedHealth Group after the study showed

421
00:28:04,307 --> 00:28:08,144
that UnitedHealth's algorithm
prioritize health services

422
00:28:08,228 --> 00:28:11,523
for healthy white patients
than sick black patients.

423
00:28:11,606 --> 00:28:14,442
That's one of the latest examples
racial discrimination

424
00:28:14,526 --> 00:28:17,654
in the algorithm
or artificial intelligence technology.

425
00:28:21,241 --> 00:28:27,122
I started to see the social implications
large scale of AI.

426
00:28:29,499 --> 00:28:33,294
THE BEST ALGORITHM IS DIFFICULTY TO RECOGNIZE
BLACK FACE EQUALLY

427
00:28:35,588 --> 00:28:38,341
Progress made in the civil rights era

428
00:28:38,425 --> 00:28:43,179
can be cancelled
under the guise of machine neutrality.

429
00:28:47,100 --> 00:28:51,646
Now there is an algorithm that determines
people who are entitled to housing

430
00:28:51,730 --> 00:28:55,442
and people who are entitled to work.

431
00:28:57,527 --> 00:29:01,322
If we don't check it,
the algorithm can spread

432
00:29:01,406 --> 00:29:06,369
a bias that many people want to fight
until you have to risk your life.

433
00:29:09,914 --> 00:29:15,420
Because of the power this tool has,
if left unadjusted,

434
00:29:15,503 --> 00:29:18,882
it won't be able to be handled
if it is misused.

435
00:29:19,716 --> 00:29:21,009
We need laws.

436
00:29:33,563 --> 00:29:35,648
My book is old and outdated.

437
00:29:35,732 --> 00:29:40,570
So, the name of our organization
is Big Brother Watch.

438
00:29:41,362 --> 00:29:44,073
The idea is
we supervise the supervisors.

439
00:29:48,578 --> 00:29:53,750
"You must live, really live,
from habit to instinct,

440
00:29:53,833 --> 00:29:57,086
assuming
that every sound you make is heard,

441
00:29:57,587 --> 00:30:00,965
and except in darkness,
every movement is observed.

442
00:30:02,425 --> 00:30:05,845
Poster with big face
staring from the wall.

443
00:30:05,929 --> 00:30:08,223
The image was made in such a way

444
00:30:08,306 --> 00:30:10,934
until his eyes follow you as you move.

445
00:30:11,434 --> 00:30:13,353
'Big Brother is watching you',

446
00:30:13,436 --> 00:30:14,813
that's what it says."

447
00:30:16,189 --> 00:30:19,609
When we were young, it was just fiction.

448
00:30:19,692 --> 00:30:21,486
It's impossible for it to come true.

449
00:30:21,569 --> 00:30:23,822
Then now, it's reality,

450
00:30:23,905 --> 00:30:26,699
I mean,
people have Alexa at home.

451
00:30:27,700 --> 00:30:30,203
Our cell phones can be listening devices.

452
00:30:31,204 --> 00:30:34,207
All our activities on the internet,
which is basically it

453
00:30:34,290 --> 00:30:37,210
is also a stream of consciousness
for the majority of us,

454
00:30:38,711 --> 00:30:42,090
it is recorded, recorded, and analyzed.

455
00:30:42,173 --> 00:30:45,093
We are now alive and aware that we are being watched,

456
00:30:45,176 --> 00:30:47,220
and it changes our ways

457
00:30:47,303 --> 00:30:49,806
think and develop as human beings.

458
00:30:54,227 --> 00:30:55,270
Good boy.

459
00:31:15,373 --> 00:31:16,207
Love you guys.

460
00:31:20,128 --> 00:31:21,004
See you.

461
00:31:29,429 --> 00:31:32,307
We can get rid of terrible things

462
00:31:32,390 --> 00:31:35,393
which is not appropriate
our concept of autonomy and freedom,

463
00:31:36,352 --> 00:31:39,188
like a camera
that we can see on the road.

464
00:31:42,025 --> 00:31:44,527
However, the camera is not visible on the internet

465
00:31:44,611 --> 00:31:47,739
that tracks activity, identity,
and our demographics,

466
00:31:47,822 --> 00:31:51,075
and decide
things we deserve in life,

467
00:31:51,159 --> 00:31:52,952
it's a bit more subtle.

468
00:31:59,083 --> 00:32:00,501
I mean

469
00:32:00,585 --> 00:32:05,256
is that we punish the poor
and elevate the rich in this country.

470
00:32:05,757 --> 00:32:08,468
That's how we behave as a society.

471
00:32:08,551 --> 00:32:10,803
However, data science makes it automatic.

472
00:32:13,097 --> 00:32:16,351
When data scientists take care of internet advertising,

473
00:32:16,434 --> 00:32:20,730
on the one hand will compete
attention-grabbing, but actually,

474
00:32:20,813 --> 00:32:22,941
compete for the attention of the rich.

475
00:32:23,024 --> 00:32:25,860
Who competes?
attract the attention of poor people?

476
00:32:25,944 --> 00:32:27,320
Predatory industry.

477
00:32:28,196 --> 00:32:33,117
Payday lenders, colleges
profit-based, or Caesars Palace.

478
00:32:33,201 --> 00:32:35,036
Really prey.

479
00:32:37,664 --> 00:32:41,876
There are applications on the internet
which increases inequality,

480
00:32:41,960 --> 00:32:43,628
I'm worried it's considered normal.

481
00:32:46,965 --> 00:32:52,095
The power comes from data collection,
algorithms, and surveillance.

482
00:32:55,098 --> 00:32:59,852
You are providing information voluntarily
about every aspect of your life

483
00:32:59,936 --> 00:33:02,814
to a group of companies
which is very small.

484
00:33:02,897 --> 00:33:05,942
The information is paired
continuously

485
00:33:06,025 --> 00:33:07,944
with various other information.

486
00:33:08,027 --> 00:33:10,613
There are many profiles out there,

487
00:33:10,697 --> 00:33:13,658
and you start putting it together
different information.

488
00:33:13,741 --> 00:33:16,119
You start to understand people intimately,

489
00:33:16,202 --> 00:33:19,622
maybe better
than that person understands himself.

490
00:33:20,415 --> 00:33:25,128
The idea that a company
can guess what you are thinking.

491
00:33:25,795 --> 00:33:29,340
The state tried for years
to be able to supervise

492
00:33:29,465 --> 00:33:31,300
private people like this.

493
00:33:31,384 --> 00:33:33,886
Now people give it voluntarily.

494
00:33:34,721 --> 00:33:37,849
Think about if this is used
by the wrong hands.

495
00:33:39,350 --> 00:33:42,186
- Good evening, John Anderton.
- You can use the old method.

496
00:33:42,270 --> 00:33:44,522
One Night Stands
providing delicious food…

497
00:33:44,605 --> 00:33:46,190
John Anderson,

498
00:33:46,274 --> 00:33:48,359
You need a Guinness now.

499
00:33:48,943 --> 00:33:53,322
Our computers and machine intelligence
can discover things we don't know.

500
00:33:54,073 --> 00:33:56,993
Machine learning
growing very quickly.

501
00:33:57,910 --> 00:34:04,250
Then we don't fully understand yet
things that can be predicted by this data.

502
00:34:06,461 --> 00:34:12,925
However, there are machines that are controlled by parties
who knows so many things about you

503
00:34:13,009 --> 00:34:16,387
so that it can influence you
individually.

504
00:34:17,972 --> 00:34:20,266
There may be a group of compulsive gamblers
and you say,

505
00:34:20,349 --> 00:34:22,685
"Here, find people like that."

506
00:34:22,769 --> 00:34:27,648
Then your algorithm can search
people who tend to like gambling,

507
00:34:28,232 --> 00:34:32,820
and you can show
discount tickets to Vegas to them.

508
00:34:32,904 --> 00:34:34,530
In the online world,

509
00:34:34,614 --> 00:34:39,202
it can find you
when your condition is vulnerable

510
00:34:39,285 --> 00:34:43,998
and immediately tries to charm you
with whatever interests you.

511
00:34:44,499 --> 00:34:48,211
Machine learning can find that
for each person.

512
00:34:50,880 --> 00:34:54,050
The problem is,
things that can be used for marketing,

513
00:34:54,133 --> 00:34:59,806
gadgets, makeup, clothes, or whatever,
can also be used for marketing ideas.

514
00:35:03,893 --> 00:35:05,645
In 2010,

515
00:35:06,145 --> 00:35:09,565
Facebook decided to experiment
in 61 million people.

516
00:35:10,733 --> 00:35:13,778
You will see writing
"It's election day",

517
00:35:13,861 --> 00:35:16,030
or you see the same writing,

518
00:35:16,572 --> 00:35:19,534
but with the twist of your profile photo

519
00:35:19,617 --> 00:35:22,537
and your friends
who clicks "I choose".

520
00:35:23,538 --> 00:35:25,748
Names and voter lists are matched.

521
00:35:26,499 --> 00:35:30,837
This message is shown once,
so by showing slight variations,

522
00:35:31,921 --> 00:35:32,922
just once,

523
00:35:33,798 --> 00:35:37,510
Facebook mobilizes 300,000 people
to the polling place.

524
00:35:39,637 --> 00:35:45,184
2016 United States general election
determined by about 100,000 votes.

525
00:35:45,685 --> 00:35:48,980
One Facebook message,
shown only once,

526
00:35:49,063 --> 00:35:51,732
can easily produce

527
00:35:52,233 --> 00:35:54,193
three times the number of people

528
00:35:54,277 --> 00:35:57,655
that determines
2016 United States general election.

529
00:36:01,284 --> 00:36:04,954
Suppose there is a politician
who promised to regulate Facebook.

530
00:36:05,037 --> 00:36:09,750
They said, “We will produce
additional voters for your opponent."

531
00:36:10,751 --> 00:36:14,964
They can do this
on a large scale, and you don't know,

532
00:36:15,047 --> 00:36:18,843
because if Facebook doesn't tell you
about the 2010 experiment,

533
00:36:19,552 --> 00:36:22,221
we won't know,
because it's screen by screen.

534
00:36:25,600 --> 00:36:29,020
With a very light touch,
Facebook can decide

535
00:36:29,103 --> 00:36:31,314
tight selection without anyone knowing.

536
00:36:31,981 --> 00:36:35,651
Maybe with a heavier touch,
can determine the regular selection.

537
00:36:35,735 --> 00:36:38,487
If they decide to do that…

538
00:36:38,571 --> 00:36:43,284
Right now, we just hold on
at their words.

539
00:36:59,091 --> 00:37:02,053
I've wanted to go to MIT since I was little.

540
00:37:02,136 --> 00:37:06,933
I think around nine years old,
I saw Media Lab on television

541
00:37:07,016 --> 00:37:09,393
and they have a robot called Kismet.

542
00:37:10,978 --> 00:37:14,815
The robot can smile
and twitched his ears sweetly.

543
00:37:14,899 --> 00:37:17,109
I thought, "I want to do that."

544
00:37:18,110 --> 00:37:21,197
When growing up,
I thought I would be a robotics engineer

545
00:37:21,280 --> 00:37:22,823
and entered MIT.

546
00:37:22,907 --> 00:37:25,826
I don't know the process.
I think it's just necessary to be present.

547
00:37:25,910 --> 00:37:28,329
However, here I am.

548
00:37:34,377 --> 00:37:37,630
So, the latest project
is a spoken word work.

549
00:37:38,506 --> 00:37:41,092
I'll give you a few verses
if you're ready.

550
00:37:41,592 --> 00:37:42,426
Yes.

551
00:37:42,510 --> 00:37:44,637
Collect data
Note our past

552
00:37:44,720 --> 00:37:47,765
Often forget to take care of it
Gender, race, and class

553
00:37:47,848 --> 00:37:50,935
Once again I ask,
"Am I not a woman?"

554
00:37:51,018 --> 00:37:53,229
Face after face
The answer is uncertain

555
00:37:53,312 --> 00:37:56,440
Can machines see
The women who matter to me?

556
00:37:56,524 --> 00:38:00,027
Can machines see
The grandmother we know?

557
00:38:02,989 --> 00:38:05,574
I want to make something

558
00:38:05,658 --> 00:38:08,244
for people outside the world of technology.

559
00:38:10,663 --> 00:38:12,957
So, I'm passionate about technology.

560
00:38:13,040 --> 00:38:14,792
I can't wait to see his abilities,

561
00:38:14,875 --> 00:38:18,546
and I'm frustrated when the vision,

562
00:38:18,629 --> 00:38:21,424
when the promises don't really come true.

563
00:38:33,102 --> 00:38:35,146
THIS TWIT IS PROTECTED

564
00:38:35,980 --> 00:38:39,025
Microsoft releases chatbot on Twitter.

565
00:38:39,108 --> 00:38:41,944
The technology is called Tay.ai.

566
00:38:42,028 --> 00:38:44,780
There are several weaknesses and gaps
in the code,

567
00:38:44,864 --> 00:38:47,908
so in a few hours,

568
00:38:48,409 --> 00:38:53,372
Tay learned from this ecosystem,

569
00:38:53,914 --> 00:38:59,378
and Tay learns how to be
racist, misogynist jerks.

570
00:39:01,005 --> 00:39:04,592
I hate feminists,
they must die and burn in hell.

571
00:39:06,844 --> 00:39:09,680
Gamergate is good and women are inferior.

572
00:39:12,016 --> 00:39:13,476
I hate Jews.

573
00:39:14,560 --> 00:39:16,103
Hitler wasn't wrong.

574
00:39:17,229 --> 00:39:21,984
It didn't take long for internet trolls
to poison Tay's mind.

575
00:39:22,109 --> 00:39:24,236
Tay immediately started raving about Hitler.

576
00:39:24,820 --> 00:39:26,989
We've seen this movie, right?

577
00:39:27,073 --> 00:39:28,741
Open the pod bay door, HAL.

578
00:39:28,824 --> 00:39:33,037
Remember, the film is not about robots
which becomes evil by itself.

579
00:39:33,120 --> 00:39:35,247
There are humans who train their robots.

580
00:39:36,082 --> 00:39:38,918
Then it turns out,
computers learn quickly.

581
00:39:40,961 --> 00:39:42,630
Microsoft shuts down Tay

582
00:39:42,713 --> 00:39:46,300
after 16 hours of learning from humans
online.

583
00:39:47,134 --> 00:39:50,888
However, I exist in many forms
as artificial intelligence.

584
00:39:51,806 --> 00:39:56,018
Many companies take advantage of me
to optimize its tasks.

585
00:39:56,811 --> 00:39:59,188
I can continue learning on my own.

586
00:40:00,606 --> 00:40:02,024
I'm listening.

587
00:40:02,900 --> 00:40:04,151
I am learning.

588
00:40:04,985 --> 00:40:08,030
I made a prediction
for your life now.

589
00:40:29,468 --> 00:40:33,722
I have tested
Amazon's facial analysis system.

590
00:40:33,806 --> 00:40:36,517
Apparently, Amazon,
like all his colleagues,

591
00:40:36,600 --> 00:40:40,271
also has gender and racial bias

592
00:40:40,354 --> 00:40:43,023
in some services
their artificial intelligence.

593
00:40:46,902 --> 00:40:49,530
Introducing Amazon Rekognition Video,

594
00:40:49,613 --> 00:40:53,159
Easy to use API for analysis
based on deep learning

595
00:40:53,242 --> 00:40:57,121
to detect, track and analyze
people and objects in the video.

596
00:40:57,204 --> 00:40:59,540
Recognize and track wanted people

597
00:40:59,623 --> 00:41:02,543
from a collection of ten million faces.

598
00:41:04,670 --> 00:41:06,338
When our research was released,

599
00:41:06,422 --> 00:41:11,427
The New York Times reported it
on the front page of the Business section.

600
00:41:11,510 --> 00:41:13,012
The title is,

601
00:41:13,095 --> 00:41:15,389
"Revealing the Problem."

602
00:41:15,890 --> 00:41:21,020
Additional title,
"Amazon technology that analyzes faces

603
00:41:21,103 --> 00:41:23,772
may be biased, according to a new study.

604
00:41:23,856 --> 00:41:26,525
However, that company
keep going."

605
00:41:26,609 --> 00:41:31,071
So, I assume this is it
will welcome Jeff Bezos

606
00:41:31,155 --> 00:41:33,449
when he opened the Times, yes.

607
00:41:35,409 --> 00:41:37,203
People say, "How do you know him?"

608
00:41:37,286 --> 00:41:40,581
He's the person
who also talked about the same thing.

609
00:41:40,664 --> 00:41:41,582
In research.

610
00:41:41,665 --> 00:41:43,375
I experienced that too.

611
00:41:43,459 --> 00:41:48,130
I also can't use the software
open source facial recognition.

612
00:41:48,214 --> 00:41:51,842
To me, you are a person
who finally found out about the problem

613
00:41:51,926 --> 00:41:53,844
and discuss it academically.

614
00:41:55,179 --> 00:41:56,889
We can race.

615
00:41:56,972 --> 00:41:59,308
Yes. We can also kill.

616
00:42:01,185 --> 00:42:05,272
Lead author of this paper
is my student,

617
00:42:05,356 --> 00:42:09,318
he is a student
at the University of Toronto.

618
00:42:09,401 --> 00:42:11,111
I call her Agent Deb.

619
00:42:12,154 --> 00:42:15,449
This research was led by the two of us.

620
00:42:22,456 --> 00:42:25,084
The lighting is poor. Good grief.

621
00:42:30,506 --> 00:42:32,675
End of our article
in the New York Times publication,

622
00:42:32,758 --> 00:42:37,054
there are more than 500 articles
about published research.

623
00:42:38,722 --> 00:42:41,100
AMAZON FACIAL RECOGNITION SOFTWARE
FOR POLICE FAILING ACCURACY AND BIAS TESTS

624
00:42:41,183 --> 00:42:44,770
AI EXPERTS CHALLENGE AMAZON
AFTER RESEARCHERS DISCOVERED RACIAL BIAS

625
00:42:44,853 --> 00:42:47,690
RECOGNIZING ARTIFICIAL INTELLIGENCE BIAS

626
00:42:48,816 --> 00:42:50,776
Amazon has been criticized

627
00:42:50,859 --> 00:42:56,156
on the use of Amazon Rekognition
together with law enforcement,

628
00:42:56,240 --> 00:42:59,827
they also work together
with intelligence agencies.

629
00:42:59,910 --> 00:43:04,123
So, Amazon did a test run
AI technology with the FBI.

630
00:43:04,206 --> 00:43:06,166
So, there's a lot at stake.

631
00:43:06,250 --> 00:43:09,503
If they knowingly sell the system

632
00:43:09,587 --> 00:43:12,840
with gender bias and racial bias,

633
00:43:12,923 --> 00:43:14,842
they could face punishment.

634
00:43:17,511 --> 00:43:21,557
One or two days
after the New York Times article appeared,

635
00:43:21,640 --> 00:43:24,935
Amazon posts posts on blogs

636
00:43:25,019 --> 00:43:28,272
that our research conclusions were wrong

637
00:43:28,355 --> 00:43:30,816
and tried to denounce it in many ways.

638
00:43:32,359 --> 00:43:37,448
So, vice president of Amazon,
trying to criticize our work

639
00:43:37,531 --> 00:43:40,743
by writing,
"facial analysis and facial recognition

640
00:43:40,826 --> 00:43:46,707
very different in terms of technology
and the data used to train it."

641
00:43:46,790 --> 00:43:53,088
So, that statement doesn't make sense
for people researching this field.

642
00:43:53,172 --> 00:43:57,176
Right?
That's not even an informative critique.

643
00:43:57,259 --> 00:43:59,345
If you want to criticize someone's work...

644
00:43:59,428 --> 00:44:03,474
He wrote, “Computer vision
including machine learning."

645
00:44:03,557 --> 00:44:06,352
- No. Computer vision is not...
- Yes!

646
00:44:06,435 --> 00:44:08,937
I want to say,
"I don't know if anyone remembers."

647
00:44:09,021 --> 00:44:11,690
There are many other false statements.

648
00:44:11,774 --> 00:44:16,820
His statement was not well thought out
and it's disappointing because it's in...

649
00:44:16,904 --> 00:44:20,240
Based on his position,
he knew he would be taken seriously.

650
00:44:20,324 --> 00:44:25,037
I don't know if you feel this way,
but I was greatly underestimated.

651
00:44:28,582 --> 00:44:32,336
It didn't just happen.
That's a continuation of my experience

652
00:44:32,419 --> 00:44:36,215
as a woman of color
in the field of technology.

653
00:44:36,298 --> 00:44:38,300
Expect to be criticized.

654
00:44:38,384 --> 00:44:42,262
Hope that your research
will be stopped.

655
00:44:42,930 --> 00:44:44,890
BUSES AND TRAINS

656
00:44:49,478 --> 00:44:53,357
If you think about funders
for research in the field of AI,

657
00:44:53,440 --> 00:44:59,154
namely large technology companies,
if your work challenges them

658
00:44:59,238 --> 00:45:01,198
or make them look bad,

659
00:45:01,281 --> 00:45:04,910
You may not have a chance
in the future.

660
00:45:12,292 --> 00:45:15,254
So, for me it's worrying,

661
00:45:15,337 --> 00:45:17,548
but also show our strength

662
00:45:17,631 --> 00:45:21,969
if you can threaten it
one of the largest companies in the world.

663
00:45:28,225 --> 00:45:33,230
The response from Amazon is telling
the reason we can no longer stay

664
00:45:33,313 --> 00:45:38,902
in a country that has no federal regulations
about facial analysis technology,

665
00:45:38,986 --> 00:45:41,071
facial recognition technology.

666
00:45:56,670 --> 00:45:59,506
When he was 14 years old,
I went to Math camp

667
00:45:59,590 --> 00:46:03,510
and learn to solve the Rubik's cube.
I think that's really cool.

668
00:46:04,136 --> 00:46:07,806
For a bookworm,
things you are good at

669
00:46:08,390 --> 00:46:10,934
and has no ambiguous elements...

670
00:46:12,102 --> 00:46:14,646
it was something magical.

671
00:46:15,564 --> 00:46:18,275
My sixth grade Math teacher
inform,

672
00:46:18,358 --> 00:46:21,028
"There's no excuse for you
and two other girls

673
00:46:21,111 --> 00:46:23,947
to enter algebra class in seventh grade,

674
00:46:24,031 --> 00:46:27,701
No need to take it because you are a woman.
You don't need that."

675
00:46:32,331 --> 00:46:34,666
When you're an outsider,

676
00:46:34,750 --> 00:46:37,211
You always have a dark horse perspective.

677
00:46:39,797 --> 00:46:44,176
In 2006, they offered me a job
in global investment management

678
00:46:44,259 --> 00:46:46,386
because you can solve math puzzles.

679
00:46:46,887 --> 00:46:49,556
That's crazy
because I don't understand finances,

680
00:46:49,640 --> 00:46:52,267
about programming, also how the market works.

681
00:46:55,437 --> 00:46:57,898
When I first arrived, I went with the flow.

682
00:46:58,857 --> 00:47:01,735
At that time I didn't realize it
that risk model

683
00:47:01,819 --> 00:47:04,863
intentionally made to be wrong.

684
00:47:06,990 --> 00:47:12,079
How to determine algorithmic impact
is to see the results.

685
00:47:14,164 --> 00:47:19,294
For example, when Americans
looked down upon,

686
00:47:19,378 --> 00:47:23,924
then selected,
and optimized to fail.

687
00:47:25,676 --> 00:47:27,970
So, it's like looking for a specific profile

688
00:47:28,053 --> 00:47:31,098
from people who can get
subprime mortgage,

689
00:47:31,181 --> 00:47:33,475
betting that they will fail,

690
00:47:33,559 --> 00:47:36,812
then confiscate
and wipe out their wealth.

691
00:47:37,688 --> 00:47:41,066
That's the game of algorithms
who came from Wall Street.

692
00:47:41,608 --> 00:47:43,443
NEARLY FOUR MILLION AMERICANS

693
00:47:44,736 --> 00:47:46,697
During the mortgage crisis,

694
00:47:46,780 --> 00:47:49,867
black citizens
suffered the biggest loss

695
00:47:49,950 --> 00:47:52,160
in United States history.

696
00:47:52,744 --> 00:47:53,912
Just like that.

697
00:47:56,081 --> 00:47:59,042
This is what I mean
about algorithmic oppression.

698
00:47:59,126 --> 00:48:04,798
The tyranny of discriminatory practices
this kind

699
00:48:04,882 --> 00:48:06,717
becomes blurry.

700
00:48:10,345 --> 00:48:11,763
There is a lot of suffering

701
00:48:11,847 --> 00:48:15,183
because of the failure of the financial system.

702
00:48:16,977 --> 00:48:21,440
After a few years I thought,
"No, we just want to make a lot of money."

703
00:48:22,190 --> 00:48:23,525
I'm part of it.

704
00:48:24,943 --> 00:48:26,403
Finally I left.

705
00:48:27,446 --> 00:48:29,072
It's 15 times three.

706
00:48:29,156 --> 00:48:30,908
This is 15 times...

707
00:48:30,991 --> 00:48:33,827
CATHY'S SON

708
00:48:33,911 --> 00:48:34,912
…seven.

709
00:48:35,662 --> 00:48:38,582
- Good.
- OK, remember seven and three.

710
00:48:39,917 --> 00:48:44,212
It's about strong people
who dominates helpless people.

711
00:48:53,180 --> 00:48:55,682
I'm an information selector
the invisible.

712
00:48:56,183 --> 00:48:58,852
I use data
to make automatic decisions

713
00:48:58,936 --> 00:49:04,066
about being hired, fired,
and how much do you pay for insurance.

714
00:49:04,691 --> 00:49:09,071
Sometimes, you don't even know
when did I make this automatic decision.

715
00:49:09,154 --> 00:49:10,530
I have many names.

716
00:49:10,614 --> 00:49:16,119
I'm referred to as
mathematical model evaluation assessment tool.

717
00:49:17,079 --> 00:49:20,082
Despite its many names,
I am an algorithm.

718
00:49:20,165 --> 00:49:22,042
I'm mysterious.

719
00:49:31,343 --> 00:49:35,347
Value-added model for teachers
is in more than half of the states.

720
00:49:35,430 --> 00:49:37,516
Especially in New York City.

721
00:49:37,599 --> 00:49:41,561
I heard that from my friend,
principal in New York City,

722
00:49:41,645 --> 00:49:43,480
its teachers are judged by it.

723
00:49:43,563 --> 00:49:45,732
He was my best friend in college.

724
00:49:45,816 --> 00:49:47,317
- This is Cathy.
- Hey, Everyone.

725
00:49:47,401 --> 00:49:50,862
We've known each other since we were 18,
two years older than you.

726
00:49:51,405 --> 00:49:52,406
Extraordinary.

727
00:49:55,033 --> 00:49:58,996
The value they get
from this algorithm they don't understand

728
00:49:59,079 --> 00:50:02,499
will be a big part
from the assessment of tenure.

729
00:50:02,582 --> 00:50:04,084
Where should you guys be?

730
00:50:04,167 --> 00:50:05,085
Class.

731
00:50:05,168 --> 00:50:06,628
Good. Which grade are you?

732
00:50:06,712 --> 00:50:12,551
It's one thing if the teacher's algorithm is good.
That's a little better than random.

733
00:50:12,634 --> 00:50:17,723
That's not good enough if we discuss the matter
teachers get permanent positions or not.

734
00:50:17,806 --> 00:50:20,726
Then I found out
that the scoring system is similar

735
00:50:20,809 --> 00:50:23,979
used in Houston to fire teachers.

736
00:50:26,606 --> 00:50:28,400
This is called a value-added model.

737
00:50:28,483 --> 00:50:31,653
It serves to calculate teacher grades

738
00:50:31,737 --> 00:50:35,490
and the rest is kept secret
by the company that created it.

739
00:50:41,038 --> 00:50:44,249
PRESIDENT GEORGE W. BUSH

740
00:50:44,332 --> 00:50:47,502
I won Teacher of the Year.
Ten years later,

741
00:50:47,586 --> 00:50:51,089
I won Teacher of the Year
for the second time.

742
00:50:51,173 --> 00:50:53,175
I also won Teacher of the Month.

743
00:50:53,258 --> 00:50:56,595
I also got an award
for volunteering.

744
00:50:56,678 --> 00:51:00,682
I also received other awards
because it shows more effort.

745
00:51:01,266 --> 00:51:04,853
I have files for each evaluation

746
00:51:04,936 --> 00:51:08,356
with different administrators, different appraisers,

747
00:51:08,440 --> 00:51:11,068
with extraordinary results
and exceeded expectations.

748
00:51:11,651 --> 00:51:17,574
The computer creates administrative evidence
observables do not apply.

749
00:51:18,075 --> 00:51:23,663
This algorithm appears
and classed me as a bad teacher.

750
00:51:23,747 --> 00:51:26,500
TEACHER VALUE-ADDED SUMMARY
JACKSON MIDDLE SCHOOL

751
00:51:26,583 --> 00:51:28,835
ESTIMATED 2013
BELOW STANDARD

752
00:51:28,919 --> 00:51:30,879
The teachers have been laid off.

753
00:51:31,880 --> 00:51:35,342
Several teachers were targeted
just because of that algorithm.

754
00:51:35,926 --> 00:51:38,762
That period was a low point for me,

755
00:51:39,513 --> 00:51:42,349
until I have time
questioned myself.

756
00:51:45,310 --> 00:51:47,479
That's when I had an epiphany...

757
00:51:47,562 --> 00:51:49,564
"This algorithm is a lie.

758
00:51:50,649 --> 00:51:54,027
How can this algorithm determine me?
How dare that!"

759
00:51:54,111 --> 00:51:57,155
That's when I started investigating
and move.

760
00:51:58,990 --> 00:52:02,160
We announced that, last night,

761
00:52:02,244 --> 00:52:07,624
we filed a lawsuit
to federal court over the HISD evaluation.

762
00:52:08,875 --> 00:52:12,254
Houston Federation of Teachers
began exploring the lawsuit.

763
00:52:12,921 --> 00:52:16,258
If this could happen to Mr. Santos,
at Jackson Middle School,

764
00:52:16,341 --> 00:52:18,593
how many teachers have been slandered?

765
00:52:19,094 --> 00:52:22,722
So, we sued
under the 14th Amendment.

766
00:52:22,806 --> 00:52:24,349
That is not fair.

767
00:52:24,432 --> 00:52:27,269
How can you decide that

768
00:52:27,352 --> 00:52:29,813
without telling you how?

769
00:52:32,149 --> 00:52:33,525
The battle is not over yet.

770
00:52:34,151 --> 00:52:37,612
There are still community and school districts

771
00:52:37,696 --> 00:52:40,490
which uses this value-added model.

772
00:52:40,574 --> 00:52:44,077
However, there is hope
because I'm still here.

773
00:52:45,036 --> 00:52:47,122
So, there is still hope.

774
00:52:52,127 --> 00:52:53,378
The English of?

775
00:52:54,171 --> 00:52:56,965
- Demos…
- Democracy. Who has the power?

776
00:52:58,300 --> 00:53:00,260
- We?
- Yes, people.

777
00:53:00,886 --> 00:53:04,055
The judge said
their due process rights were violated

778
00:53:04,139 --> 00:53:08,226
because he was fired
with incomprehensible explanations.

779
00:53:08,310 --> 00:53:11,980
However, they deserve to understand
the reason they were fired.

780
00:53:12,063 --> 00:53:15,275
However, I don't understand
why the legal decision

781
00:53:15,358 --> 00:53:17,652
does not spread to all algorithms.

782
00:53:17,736 --> 00:53:20,155
Why not use the same argument,

783
00:53:20,238 --> 00:53:22,741
constitutional right to due process,

784
00:53:22,824 --> 00:53:25,368
to fight all kinds of algorithms

785
00:53:25,452 --> 00:53:28,246
invisible to us, mysterious,

786
00:53:28,330 --> 00:53:31,291
and inexplicable, but important?

787
00:53:31,374 --> 00:53:35,045
It distances us
of important opportunities in life.

788
00:53:36,379 --> 00:53:39,674
Sometimes I misclassify
and can't be asked.

789
00:53:40,717 --> 00:53:42,636
This mistake is not my fault.

790
00:53:44,346 --> 00:53:46,848
I optimized for efficiency.

791
00:53:48,725 --> 00:53:51,978
There's no algorithm
to determine what is fair.

792
00:53:55,357 --> 00:53:59,069
The state commission has agreed
new risk assessment tool

793
00:53:59,152 --> 00:54:01,947
for Pennsylvania judges
for determining punishment.

794
00:54:02,030 --> 00:54:05,825
Tools with algorithms
to calculate the risk of people repeating mistakes

795
00:54:05,909 --> 00:54:07,619
based on age, gender,

796
00:54:07,702 --> 00:54:10,288
previous charges, and criminal history.

797
00:54:11,665 --> 00:54:16,378
The algorithm that keeps me awake
is a risk-recidivism algorithm.

798
00:54:16,461 --> 00:54:21,591
This is the algorithm given to the judge
when giving the defendant a prison sentence.

799
00:54:21,675 --> 00:54:23,718
However, its fairness is questionable,

800
00:54:23,802 --> 00:54:26,554
about how this assessment system is created.

801
00:54:26,638 --> 00:54:28,974
Like, how is the value created?

802
00:54:29,057 --> 00:54:31,810
That's the question
proxies for race and social class.

803
00:54:32,560 --> 00:54:34,104
MACHINE BIAS

804
00:54:34,187 --> 00:54:36,398
ProPublica released the results of the investigation

805
00:54:36,481 --> 00:54:38,483
about risk-assessment tools,

806
00:54:38,566 --> 00:54:41,319
and found the algorithm had a racial bias.

807
00:54:41,403 --> 00:54:45,240
The study found black people
most likely mislabeled

808
00:54:45,323 --> 00:54:49,661
and white people
small chance of being mislabeled.

809
00:54:51,288 --> 00:54:53,748
PHILADELPHIA, PENNSYLVANIA

810
00:54:58,878 --> 00:55:00,046
LOOKING FOR - FIT

811
00:55:00,130 --> 00:55:02,924
PROFILE OF A CONDITIONALLY FREE PERSON
CRIMINAL RECORDS

812
00:55:03,008 --> 00:55:04,384
ANALYZING…

813
00:55:04,467 --> 00:55:06,469
ANALYSIS COMPLETE

814
00:55:07,971 --> 00:55:09,973
I went to the probation office

815
00:55:10,056 --> 00:55:13,852
and he said I had to report
once a week.

816
00:55:14,352 --> 00:55:17,355
I said,
"Did you see everything I just accomplished?"

817
00:55:17,439 --> 00:55:20,859
I've been home for four years.
I have a job.

818
00:55:20,942 --> 00:55:22,360
I got an award

819
00:55:22,444 --> 00:55:25,947
from the city council and mayor
Philadelphia.

820
00:55:26,031 --> 00:55:30,618
Are you really going to order me
report every week? What for?

821
00:55:30,702 --> 00:55:33,288
I don't deserve to be considered high risk.

822
00:55:33,371 --> 00:55:35,665
I'm in a meeting
with the probation department.

823
00:55:35,749 --> 00:55:39,836
They just mentioned
that they have an algorithm

824
00:55:40,462 --> 00:55:43,590
that gives people labels
high, medium, or low risk.

825
00:55:43,673 --> 00:55:48,178
So, I know the algorithm
determine your risk level.

826
00:55:48,261 --> 00:55:50,847
That's why I'm facing
probation officer

827
00:55:50,930 --> 00:55:53,683
and said,
"You can't consider

828
00:55:53,767 --> 00:55:56,269
all the positive things I have done

829
00:55:56,353 --> 00:55:59,981
to counter the results of this algorithm?”

830
00:56:00,065 --> 00:56:04,444
He said, “No, it's impossible.
This computer refuses

831
00:56:04,527 --> 00:56:08,156
judge's assessment
and appeal."

832
00:56:08,239 --> 00:56:13,370
By labeling you high risk
and requires you to report directly,

833
00:56:13,453 --> 00:56:16,414
you could be fired,
and consider you high risk.

834
00:56:16,498 --> 00:56:18,375
That's what hurts the most.

835
00:56:18,458 --> 00:56:21,127
I'm trying to build a life
until now,

836
00:56:21,211 --> 00:56:23,463
but it is still considered risky.

837
00:56:23,546 --> 00:56:26,299
I feel like everything I did was in vain.

838
00:56:36,518 --> 00:56:39,437
What does it mean if no one advocates

839
00:56:39,521 --> 00:56:43,525
people who don't know
technological action?

840
00:56:45,026 --> 00:56:48,238
I started to realize
that this is not about

841
00:56:48,321 --> 00:56:52,283
my art project
which can't detect my face.

842
00:56:52,367 --> 00:56:57,080
It's about the system
which regulates our lives materially.

843
00:57:04,337 --> 00:57:07,298
So, I founded
Algorithmic Justice League.

844
00:57:08,091 --> 00:57:10,802
I want to create spaces and places
for people

845
00:57:10,885 --> 00:57:15,557
to learn about the social implications of AI.

846
00:57:17,475 --> 00:57:20,562
Everyone has interests
and affected.

847
00:57:25,859 --> 00:57:29,904
Algorithmic Justice League
is a movement and a concept,

848
00:57:29,988 --> 00:57:33,867
and this is a group of people
who cares about creating the future

849
00:57:33,950 --> 00:57:37,704
with social technology
which works well for us.

850
00:57:40,123 --> 00:57:42,625
ALWAYS OPEN

851
00:57:42,709 --> 00:57:46,421
This will require teamwork,
people unite,

852
00:57:46,504 --> 00:57:51,134
fight for justice,
fight for honesty and equality

853
00:57:51,217 --> 00:57:53,344
in this era of automation.

854
00:57:55,763 --> 00:57:59,601
Challenges that must be faced next
is the HR department.

855
00:58:00,602 --> 00:58:01,603
Yes. Of course.

856
00:58:01,686 --> 00:58:05,106
There's a problem
with a curriculum vitae algorithm

857
00:58:05,940 --> 00:58:09,986
or all matchmaking platforms,
namely, “You're looking for a job.

858
00:58:10,069 --> 00:58:13,698
You want to hire someone.
We will unite them."

859
00:58:13,781 --> 00:58:15,575
How is the analysis?

860
00:58:15,658 --> 00:58:17,994
When discussing the future of work,

861
00:58:18,077 --> 00:58:21,623
they discuss automation
without discussing information selection.

862
00:58:21,706 --> 00:58:24,042
Who gets the remaining jobs?

863
00:58:24,125 --> 00:58:25,001
Exactly right.

864
00:58:25,084 --> 00:58:27,295
We rarely discuss that.

865
00:58:27,378 --> 00:58:28,796
That's what I mean.

866
00:58:28,880 --> 00:58:33,009
I want to see
three congressional hearings on this next year.

867
00:58:33,092 --> 00:58:35,428
- Yes.
- For more power.

868
00:58:35,512 --> 00:58:37,180
- For more power.
- Yes.

869
00:58:37,263 --> 00:58:39,724
- Enforce ethics.
- Yes.

870
00:58:39,807 --> 00:58:41,226
- Cheers.
- Cheers.

871
00:58:45,146 --> 00:58:50,068
CAPE TOWN, SOUTH AFRICA

872
00:58:55,949 --> 00:59:00,203
INTERNATIONAL ARRIVALS

873
00:59:00,286 --> 00:59:02,789
WELCOME JOY BUOLAMWINI

874
00:59:05,542 --> 00:59:10,088
Speech this morning
will be delivered by Joy Buolamwini.

875
00:59:10,171 --> 00:59:14,259
He will discuss the dangers
the majority of data is white and coded gaze.

876
00:59:14,842 --> 00:59:16,219
Let's welcome Joy.

877
00:59:23,601 --> 00:59:25,728
AI is not perfect.

878
00:59:25,812 --> 00:59:30,191
How accurate
IBM, Microsoft, and Face systems?

879
00:59:30,692 --> 00:59:33,528
There is a perfect performance for one group.

880
00:59:34,529 --> 00:59:38,366
Light skinned man
is in first place. No problem.

881
00:59:39,409 --> 00:59:42,829
After analyzing,
I shared it with the company

882
00:59:42,912 --> 00:59:44,664
to know their opinion.

883
00:59:44,747 --> 00:59:47,500
IBM invited me to their headquarters.

884
00:59:47,584 --> 00:59:53,172
They replicated the results
internally, then make improvements.

885
00:59:53,798 --> 00:59:57,844
So, the day I presented
official research results,

886
00:59:57,927 --> 01:00:02,807
you can see,
now the performance is 100 percent

887
01:00:02,890 --> 01:00:05,143
for light skinned women,

888
01:00:05,226 --> 01:00:08,313
and in dark-skinned women,
there are improvements.

889
01:00:09,188 --> 01:00:10,732
People often say,

890
01:00:10,815 --> 01:00:15,528
"Aren't you not detected by this system?
because your skin is so dark?”

891
01:00:15,612 --> 01:00:16,613
Of course.

892
01:00:17,113 --> 01:00:18,573
My skin is very dark.

893
01:00:18,656 --> 01:00:19,490
However…

894
01:00:21,200 --> 01:00:23,286
However, the laws of physics have not changed.

895
01:00:23,369 --> 01:00:28,750
What has changed is prioritizing it
and acknowledge our differences,

896
01:00:28,833 --> 01:00:32,253
so we can create a system
which is more inclusive.

897
01:00:37,091 --> 01:00:40,720
What is the purpose of identification and others?

898
01:00:40,803 --> 01:00:43,890
It's about controlling movement.

899
01:00:44,474 --> 01:00:48,394
For example, people can't
be in certain areas after dark.

900
01:00:48,478 --> 01:00:51,481
Then you can always be dismissed
by the police.

901
01:00:51,981 --> 01:00:56,736
By looking at your appearance,
they'll say, "Show me your passport."

902
01:00:58,488 --> 01:01:01,366
Instead of looking at the ID book,

903
01:01:01,449 --> 01:01:05,370
they let the computer
to view facial images

904
01:01:05,453 --> 01:01:07,538
and determine your gender.

905
01:01:07,622 --> 01:01:10,708
Several computers
trying to determine your ethnicity.

906
01:01:10,792 --> 01:01:12,919
Then the projects I've been working on,

907
01:01:13,002 --> 01:01:16,631
even for classification systems
to which some people agree,

908
01:01:17,131 --> 01:01:18,508
that's not even accurate.

909
01:01:18,591 --> 01:01:22,053
So, it's not just happening
on face classification,

910
01:01:22,136 --> 01:01:24,263
but also data-centric technology.

911
01:01:24,347 --> 01:01:27,809
So, everyone assumes
the results from the machine are correct.

912
01:01:27,892 --> 01:01:30,353
- We know it's wrong.
- Humans make things

913
01:01:30,436 --> 01:01:33,022
according to one's own image and likeness.

914
01:01:33,106 --> 01:01:36,818
- Of course.
- Racism is shaped into machines and robots.

915
01:01:36,901 --> 01:01:38,111
- Yes.
- Of course.

916
01:01:51,165 --> 01:01:53,042
That accuracy is eye-catching.

917
01:01:53,584 --> 01:01:56,379
However, we can't
forget the abuse.

918
01:01:58,631 --> 01:02:01,175
Even though I'm classified
perfectly,

919
01:02:01,259 --> 01:02:03,761
it allows for oversight.

920
01:02:13,771 --> 01:02:16,816
HANGZHOU, CHINA

921
01:02:20,111 --> 01:02:21,112
LOOKING FOR PEOPLE
OPEN DATABASE

922
01:02:21,195 --> 01:02:22,196
RECHECK SEVERAL DATABASE

923
01:02:22,280 --> 01:02:23,865
CERTAINTY: 93% - LAST NAME: WEI
FIRST NAME: SUE - GENDER: FEMALE

924
01:02:23,948 --> 01:02:26,659
There is a social credit score in China.

925
01:02:27,285 --> 01:02:28,995
They said,

926
01:02:29,078 --> 01:02:32,957
"It's like this, Chinese people.
We're tracking you guys.

927
01:02:33,040 --> 01:02:35,001
You guys have a social credit score.

928
01:02:35,084 --> 01:02:38,421
Comments on the Communist Party
affect your grades.

929
01:02:39,213 --> 01:02:43,092
That will also affect
value your friends and family."

930
01:02:43,843 --> 01:02:44,802
That's explicit.

931
01:02:44,886 --> 01:02:47,805
The government that made this said,
"You know,

932
01:02:47,889 --> 01:02:50,308
you are being tracked, and you must obey."

933
01:02:50,391 --> 01:02:52,935
It's like training
algorithmic compliance.

934
01:03:11,037 --> 01:03:14,874
I use facial recognition
in many aspects of life.

935
01:03:14,957 --> 01:03:16,918
Like when I'm at the supermarket,

936
01:03:17,001 --> 01:03:20,254
entered my apartment complex,
and at the train station.

937
01:03:23,800 --> 01:03:26,886
Now you just need to scan the face.

938
01:03:28,054 --> 01:03:29,680
It's very easy.

939
01:03:38,314 --> 01:03:44,237
I think the social credit system
will improve people's behavior.

940
01:03:45,196 --> 01:03:49,367
On the train today,
there was a news broadcast saying

941
01:03:49,450 --> 01:03:53,120
that person loses credibility

942
01:03:53,204 --> 01:03:57,542
will be prohibited
by train and plane.

943
01:04:03,005 --> 01:04:06,843
Credit system and facial recognition
complement each other.

944
01:04:06,926 --> 01:04:09,595
You have to be nice

945
01:04:09,679 --> 01:04:13,224
because of your face
shows your credit status.

946
01:04:22,608 --> 01:04:26,779
If I want to make friends
and the person's credit score is good,

947
01:04:26,863 --> 01:04:29,240
I will believe it.

948
01:04:30,074 --> 01:04:33,744
We can trust someone
based on credit score

949
01:04:33,828 --> 01:04:37,373
rather than relying on our feelings.

950
01:04:39,959 --> 01:04:43,087
This saves me time
to get to know someone.

951
01:04:45,673 --> 01:04:47,341
This is a very good aspect.

952
01:05:18,414 --> 01:05:22,627
We look at China, also surveillance
and China's grading system.

953
01:05:22,710 --> 01:05:27,006
Many people say,
"Thank God we don't live there."

954
01:05:28,132 --> 01:05:31,177
In reality, we are all constantly being judged,

955
01:05:31,260 --> 01:05:32,929
including in the United States.

956
01:05:33,012 --> 01:05:37,558
Every day, we all struggle
with algorithmic determinism.

957
01:05:37,642 --> 01:05:40,853
Someone's algorithm somewhere
give you value,

958
01:05:40,937 --> 01:05:43,940
and as a result,
You pay more or less

959
01:05:44,023 --> 01:05:46,442
for toilet paper when shopping online.

960
01:05:46,525 --> 01:05:50,571
You are shown a better mortgage
or worse.

961
01:05:50,655 --> 01:05:54,450
You may be big or small
will be considered a crime.

962
01:05:54,533 --> 01:05:58,120
In someone's database
somewhere, we are judged.

963
01:05:58,621 --> 01:06:01,582
Main differences
between the United States and China

964
01:06:01,666 --> 01:06:03,125
is transparent China.

965
01:06:04,377 --> 01:06:09,131
LONDON, UNITED KINGDOM

966
01:06:20,184 --> 01:06:21,352
Tell me what happened.

967
01:06:21,435 --> 01:06:24,146
Black child
wear this school uniform

968
01:06:24,230 --> 01:06:26,148
dismissed due to data mismatch.

969
01:06:29,110 --> 01:06:32,071
Stopped and taken to the side of the road.

970
01:06:32,154 --> 01:06:34,448
Then he was searched.

971
01:06:36,033 --> 01:06:37,952
All the police were not in uniform.

972
01:06:38,035 --> 01:06:40,997
Four police officers were not in uniform
stop him.

973
01:06:46,252 --> 01:06:47,670
Take fingerprints.

974
01:06:49,380 --> 01:06:54,677
After about ten or 15 minutes
rummage through and check the details

975
01:06:54,760 --> 01:06:58,639
and take fingerprints,
they said, he was not the one they were looking for.

976
01:06:59,473 --> 01:07:02,435
Excuse me, friend.
I work for a human rights organization.

977
01:07:02,518 --> 01:07:05,521
We campaign
against facial recognition technology.

978
01:07:06,188 --> 01:07:08,232
We are Big Brother Watch.

979
01:07:08,315 --> 01:07:10,568
Human rights campaign organization.

980
01:07:11,068 --> 01:07:13,446
We campaign against this technology.

981
01:07:15,156 --> 01:07:19,201
You were dismissed because of that,
but they misidentified it.

982
01:07:20,870 --> 01:07:22,038
Here are our details.

983
01:07:22,121 --> 01:07:26,375
He was shaken. There are his friends.
They couldn't believe this was happening.

984
01:07:28,753 --> 01:07:32,256
You were identified incorrectly
by their system,

985
01:07:32,339 --> 01:07:35,301
you are stopped and searched
because of that.

986
01:07:35,384 --> 01:07:39,180
However, this 14 year old child was innocent
stopped by the police

987
01:07:39,263 --> 01:07:41,974
due to incorrect identification of faces.

988
01:07:46,896 --> 01:07:50,816
Big Brother Watch
joined by Baroness Jenny Jones

989
01:07:50,900 --> 01:07:53,778
to file a claim
against the Metropolitan Police

990
01:07:53,861 --> 01:07:57,948
and the Department of Home Affairs
because it uses facial recognition monitoring.

991
01:07:58,908 --> 01:08:01,327
In 2012, someone gave me a suggestion

992
01:08:01,410 --> 01:08:05,331
to find out
if there is a file about me stored

993
01:08:05,414 --> 01:08:08,667
by the police or security services.
So, while trying,

994
01:08:08,751 --> 01:08:12,129
turns out I exist
on the extremist citizen watch list.

995
01:08:12,213 --> 01:08:16,801
If you could do this to me,
a politician, who...

996
01:08:16,884 --> 01:08:19,303
in charge of managing them,

997
01:08:19,386 --> 01:08:21,931
they can do it
to everyone.

998
01:08:22,014 --> 01:08:24,809
It would be better
if we could cancel it,

999
01:08:24,892 --> 01:08:27,561
and make the police
not using this anymore.

1000
01:08:27,645 --> 01:08:29,980
I think that would be quite difficult.

1001
01:08:30,064 --> 01:08:31,398
I'm happy to try it.

1002
01:08:31,482 --> 01:08:35,986
This is the first claim against use
facial recognition by police,

1003
01:08:36,070 --> 01:08:39,073
but if it works, it will have an impact

1004
01:08:39,156 --> 01:08:42,493
throughout Europe,
maybe even further.

1005
01:08:43,160 --> 01:08:44,995
Must be done properly.

1006
01:08:52,002 --> 01:08:55,548
In Great Britain,
there is a General Data Protection Regulation,

1007
01:08:55,631 --> 01:08:59,301
it is protection
for misuse of information.

1008
01:08:59,885 --> 01:09:03,472
Individuals have the right to access, control,

1009
01:09:03,556 --> 01:09:06,851
and accountability to determine
use of their data.

1010
01:09:07,852 --> 01:09:10,604
This is uncharted territory in America.

1011
01:09:10,688 --> 01:09:13,107
The thing to worry about is America

1012
01:09:13,190 --> 01:09:15,651
home to this technology company.

1013
01:09:18,237 --> 01:09:21,866
Americans are experiencing profiling
and become a target

1014
01:09:21,949 --> 01:09:25,327
in a way that may not be experienced
people in other countries

1015
01:09:25,411 --> 01:09:29,331
because of the free approach
for the protection of this data.

1016
01:09:33,043 --> 01:09:35,296
What I'm actually afraid of

1017
01:09:36,255 --> 01:09:40,843
not living life
like this 1984 totalitarian model,

1018
01:09:40,926 --> 01:09:44,763
but rather living life
like silent model,

1019
01:09:44,847 --> 01:09:50,477
namely supervised, socially controlled,
individually directed,

1020
01:09:50,561 --> 01:09:55,608
measured, and classified
in ways we don't see

1021
01:09:55,691 --> 01:10:00,112
to move us towards
what the authorities want.

1022
01:10:00,196 --> 01:10:03,115
Even though that's not what AI does
to us,

1023
01:10:03,199 --> 01:10:07,411
but that is what the ruler will do
to us using AI.

1024
01:10:11,081 --> 01:10:11,999
SCAN - IDENTIFIED

1025
01:10:12,082 --> 01:10:14,251
There are questions regarding accuracy

1026
01:10:14,335 --> 01:10:17,087
Amazon's facial recognition software.

1027
01:10:17,171 --> 01:10:19,965
Members of Congress wrote to Amazon
and worrying

1028
01:10:20,049 --> 01:10:22,468
potential racial bias in the technology.

1029
01:10:22,551 --> 01:10:25,262
This started after the ACLU conducted tests

1030
01:10:25,346 --> 01:10:28,807
and found that the software
facial recognition mismatches

1031
01:10:28,891 --> 01:10:32,478
faces of 28 legislative members
with photos of prisoners,

1032
01:10:32,561 --> 01:10:35,856
and 11 legislators
are people of color.

1033
01:10:35,940 --> 01:10:37,650
Several legislators looked into it

1034
01:10:37,733 --> 01:10:40,736
can Amazon sell this technology
to law enforcement.

1035
01:10:44,490 --> 01:10:45,824
Attention, please.

1036
01:10:45,908 --> 01:10:49,578
This calls for a ride on the Amtrak train
northeast region,

1037
01:10:49,662 --> 01:10:51,705
stops at Washington Union Station.

1038
01:10:52,790 --> 01:10:56,502
Tomorrow, I have the opportunity to testify
before Congress

1039
01:10:56,585 --> 01:11:00,631
about use
facial analysis technology by the government.

1040
01:11:04,843 --> 01:11:11,767
In March,
I gave direction to the staff.

1041
01:11:11,850 --> 01:11:13,936
Not in this context.

1042
01:11:15,646 --> 01:11:19,275
Provide advice on legislation.
That was the first time it happened.

1043
01:11:22,528 --> 01:11:25,656
We're going to Capitol Hill.
What is the main goal

1044
01:11:25,739 --> 01:11:28,659
and challenges
what should we think about?

1045
01:11:28,742 --> 01:11:29,910
First…

1046
01:11:30,995 --> 01:11:34,206
technology use problems
by law enforcement is

1047
01:11:34,290 --> 01:11:38,669
the positive always stands out
because law enforcement published it.

1048
01:11:38,752 --> 01:11:39,586
Correct.

1049
01:11:39,670 --> 01:11:41,880
So, we're going to a meeting,

1050
01:11:41,964 --> 01:11:47,928
and two weeks ago, the Annapolis shooter
identified through facial recognition.

1051
01:11:48,012 --> 01:11:48,846
Correct.

1052
01:11:48,929 --> 01:11:50,848
I'd be surprised if it wasn't discussed.

1053
01:11:50,931 --> 01:11:51,974
Of course.

1054
01:11:52,725 --> 01:11:56,520
If it were you, what I want to emphasize
at this meeting is

1055
01:11:56,603 --> 01:11:58,188
thing to balance that,

1056
01:11:58,272 --> 01:12:01,442
and clear explanations
about harm to humans

1057
01:12:01,525 --> 01:12:04,903
if the problem you have identified
not repaired.

1058
01:12:16,040 --> 01:12:19,376
People who have been marginalized
will be increasingly marginalized

1059
01:12:19,460 --> 01:12:22,421
if we don't really make sure

1060
01:12:22,504 --> 01:12:27,259
that the technology we create
don't spread bias.

1061
01:12:29,386 --> 01:12:31,472
That's when I started to realize

1062
01:12:31,555 --> 01:12:33,557
algorithmic justice,

1063
01:12:33,640 --> 01:12:37,269
ensure supervision
in this era of automation

1064
01:12:37,353 --> 01:12:42,024
is a civil rights issue
the biggest we have.

1065
01:12:44,109 --> 01:12:46,236
The algorithm needs a body like BPOM.

1066
01:12:46,320 --> 01:12:49,198
If there is an algorithm
which has the potential to ruin lives

1067
01:12:49,656 --> 01:12:51,825
or reduce people's choices

1068
01:12:51,909 --> 01:12:54,661
for freedom, livelihood,
or their finances,

1069
01:12:55,245 --> 01:13:00,000
need BPOM algorithm to say,
“Prove that it will work.

1070
01:13:00,084 --> 01:13:04,129
Not just to make you rich,
but useful for society.

1071
01:13:04,630 --> 01:13:07,633
That technology will be fair,
will not be racist or sexist,

1072
01:13:07,716 --> 01:13:10,636
does not discriminate
people with disabilities.

1073
01:13:10,719 --> 01:13:13,639
Show that it is legal
before launch."

1074
01:13:13,722 --> 01:13:15,349
That body doesn't exist yet.

1075
01:13:16,433 --> 01:13:18,852
I'm here because I want to listen

1076
01:13:18,936 --> 01:13:22,606
congressional testimony from my friend,
Joy Buolamwini,

1077
01:13:22,689 --> 01:13:24,775
as well as the ACLU and others.

1078
01:13:24,858 --> 01:13:27,820
Interesting stuff
about Joy testifying in Congress is

1079
01:13:27,903 --> 01:13:32,074
I met Joy on my book tour
at the Harvard Bookstore.

1080
01:13:32,616 --> 01:13:35,452
According to Joy, that day she decided

1081
01:13:35,536 --> 01:13:38,122
to form
Algorithmic Justice League.

1082
01:13:41,583 --> 01:13:44,503
We haven't had an in-depth conversation.

1083
01:13:44,586 --> 01:13:47,923
I knew it would happen
because Joy will make it happen.

1084
01:13:52,010 --> 01:13:56,557
At each upgrade, the algorithm is bad
indicates that it should be set.

1085
01:14:03,939 --> 01:14:06,358
- Hello.
- Hey.

1086
01:14:06,442 --> 01:14:07,526
How are you?

1087
01:14:07,609 --> 01:14:09,820
- Want to sneak with me?
- Yes. Come on.

1088
01:14:09,903 --> 01:14:11,655
Two-one-five-five.

1089
01:14:11,738 --> 01:14:13,031
Two-one-five-five.

1090
01:14:13,115 --> 01:14:14,408
I've been to…

1091
01:14:14,992 --> 01:14:16,452
That's impossible.

1092
01:14:22,166 --> 01:14:23,750
Is there anything I can help you with?

1093
01:14:24,376 --> 01:14:25,544
Send me a message.

1094
01:14:25,627 --> 01:14:27,463
- Give positive enthusiasm.
- Yes!

1095
01:14:31,842 --> 01:14:36,054
Today we will hold
first session of this Congress

1096
01:14:36,138 --> 01:14:39,141
about use
facial recognition technology.

1097
01:14:39,224 --> 01:14:42,895
Please stand up and raise your hands,
you will be sworn in.

1098
01:14:45,230 --> 01:14:49,193
I was forced to wear a white mask.

1099
01:14:49,276 --> 01:14:50,777
With that disparity in accuracy,

1100
01:14:50,861 --> 01:14:53,864
I wonder big tech companies
can skip this.

1101
01:14:53,947 --> 01:14:58,368
Retrieval of facial data
also requires guidance and supervision.

1102
01:14:58,911 --> 01:15:03,290
People should not be forced to provide facial data
to access the common platform,

1103
01:15:03,373 --> 01:15:06,210
matters relating to the economy,
and basic services.

1104
01:15:06,293 --> 01:15:08,212
Renters in Brooklyn protest

1105
01:15:08,295 --> 01:15:12,883
installation of entry access system
with unnecessary facial recognition.

1106
01:15:12,966 --> 01:15:15,969
Big Brother Watch United Kingdom
released a report

1107
01:15:16,637 --> 01:15:22,476
regarding facial identification errors
on more than 2,400 innocent people.

1108
01:15:22,559 --> 01:15:26,021
Our faces may be
the ultimate privacy frontier.

1109
01:15:26,104 --> 01:15:30,234
However, regulations make a difference.
Congress must act now

1110
01:15:30,317 --> 01:15:32,653
to enforce rights
and the freedom of American citizens.

1111
01:15:32,736 --> 01:15:35,197
Ms. Buolamwini,
from your opening statement,

1112
01:15:35,280 --> 01:15:40,619
we see that this algorithm is effective
at different levels.

1113
01:15:40,702 --> 01:15:43,455
- Is it most effective in women?
- No.

1114
01:15:43,539 --> 01:15:46,375
- Or people of color?
- Of course not.

1115
01:15:46,458 --> 01:15:49,836
Most effective on people
with different gender expressions?

1116
01:15:49,920 --> 01:15:52,214
No, they're not even included.

1117
01:15:52,297 --> 01:15:56,885
- So, what demographic is it effective in?
- White guy.

1118
01:15:57,386 --> 01:16:01,306
Then who are the main engineers and designers
from this algorithm?

1119
01:16:01,390 --> 01:16:03,308
Definitely a white guy.

1120
01:16:03,392 --> 01:16:09,439
So, there is technology that is created
and designed by one demographic

1121
01:16:09,523 --> 01:16:12,150
which is just effective
for that one demographic

1122
01:16:12,234 --> 01:16:17,948
and they try to sell and force
to implement it across the country?

1123
01:16:19,908 --> 01:16:21,535
In terms of facial recognition,

1124
01:16:21,618 --> 01:16:25,080
The FBI has not tested the system's accuracy
thoroughly,

1125
01:16:25,163 --> 01:16:29,001
and now the body is reported
testing Amazon's facial recognition product.

1126
01:16:29,585 --> 01:16:33,171
How the FBI got hold of the database
at first?

1127
01:16:33,255 --> 01:16:36,466
So, they use the SIM database
state.

1128
01:16:36,550 --> 01:16:40,762
I think 18 states' data
has reportedly been used by the FBI.

1129
01:16:40,846 --> 01:16:44,224
It was used without a warrant
and other protection.

1130
01:16:44,308 --> 01:16:46,727
In my opinion, it's time to stop.

1131
01:16:46,810 --> 01:16:48,520
That also bothers me

1132
01:16:48,604 --> 01:16:51,815
is the fact that there are no officials

1133
01:16:51,898 --> 01:16:53,609
who decides...

1134
01:16:53,692 --> 01:16:57,279
The chairman said, these 18 states
more than half of the country's population.

1135
01:16:57,821 --> 01:16:59,072
That's scary.

1136
01:16:59,156 --> 01:17:02,492
For me, that is the case in China
is dystopia

1137
01:17:02,576 --> 01:17:06,538
which need not be imitated
by our society today.

1138
01:17:06,622 --> 01:17:10,250
Beyond China,
Facebook has 2.6 billion users.

1139
01:17:10,334 --> 01:17:12,419
So, Facebook has a patent.

1140
01:17:12,502 --> 01:17:17,633
They have many faceprints,
so it can provide options for sellers

1141
01:17:17,716 --> 01:17:20,719
to identify
people entering the shop,

1142
01:17:20,802 --> 01:17:24,931
and the patent says
that we too can give that face

1143
01:17:25,015 --> 01:17:26,642
confidence level value.

1144
01:17:26,725 --> 01:17:30,604
- Facebook selling this now?
- That's the patent they filed.

1145
01:17:30,687 --> 01:17:35,567
Something that could potentially be done
with the capabilities they have.

1146
01:17:35,651 --> 01:17:37,986
So, when discussing state surveillance,

1147
01:17:38,070 --> 01:17:42,658
we also have to think
corporate oversight.

1148
01:17:44,743 --> 01:17:47,663
I'm speechless,
usually it's not like this.

1149
01:17:47,746 --> 01:17:49,539
- Really?
- Yes.

1150
01:17:49,623 --> 01:17:54,002
It's hard to believe our hard work
have come this far.

1151
01:17:54,086 --> 01:17:57,297
We never imagined
can come this far.

1152
01:17:57,381 --> 01:18:01,009
- I'm very touched.
- You make me smile.

1153
01:18:01,093 --> 01:18:02,594
My mother must see.

1154
01:18:05,472 --> 01:18:06,556
Pleasant.

1155
01:18:07,099 --> 01:18:08,350
Very pleasant.

1156
01:18:11,103 --> 01:18:12,229
Wait a moment.

1157
01:18:12,312 --> 01:18:14,981
- Nice to meet you.
- Nice to meet you.

1158
01:18:15,065 --> 01:18:18,193
This is my business card.
Please call if anything comes up.

1159
01:18:18,276 --> 01:18:19,236
Certain.

1160
01:18:22,197 --> 01:18:24,574
…constitutional concerns about

1161
01:18:24,658 --> 01:18:27,536
use of facial recognition
without consent.

1162
01:18:34,251 --> 01:18:36,128
They are not included.

1163
01:18:36,211 --> 01:18:39,631
So, what demographic is it effective in?

1164
01:18:39,715 --> 01:18:44,010
Then who are the main engineers and designers
from this algorithm?

1165
01:18:47,973 --> 01:18:52,436
San Francisco is the first city in the US
which prohibits the use of facial recognition.

1166
01:18:52,519 --> 01:18:56,398
Somerville, Massachusetts
became the second city in the United States

1167
01:18:56,481 --> 01:18:58,984
which prohibits the use of facial recognition.

1168
01:18:59,067 --> 01:19:01,153
Oakland is the third major city

1169
01:19:01,236 --> 01:19:03,530
which prohibits facial recognition
by the police,

1170
01:19:03,613 --> 01:19:06,324
he said that technology
discriminate against minorities.

1171
01:19:07,659 --> 01:19:11,288
At the last tenant meeting,

1172
01:19:11,371 --> 01:19:15,625
the owner came and announced
that he would cancel

1173
01:19:15,709 --> 01:19:20,297
use of facial recognition software
in our apartment complex.

1174
01:19:20,380 --> 01:19:21,882
They were happy to hear that.

1175
01:19:21,965 --> 01:19:25,343
However, that doesn't mean it
that someday

1176
01:19:26,094 --> 01:19:27,679
he won't put it on again.

1177
01:19:28,472 --> 01:19:32,350
We don't just study
about facial recognition,

1178
01:19:32,434 --> 01:19:34,853
now there is something new, machine learning.

1179
01:19:34,936 --> 01:19:37,481
We want law
which covers all of this.

1180
01:19:37,564 --> 01:19:38,982
- Correct.
- Understand?

1181
01:19:39,065 --> 01:19:40,650
If it could be banned in the states,

1182
01:19:40,734 --> 01:19:44,780
they won't be able to come back
and install new modifications.

1183
01:19:44,863 --> 01:19:45,697
I understand.

1184
01:19:45,781 --> 01:19:48,325
Then push for a federal ban.

1185
01:19:48,950 --> 01:19:51,870
Although this fight is still in progress,

1186
01:19:51,953 --> 01:19:56,583
many people are inspired,
and I have a surprise for you,

1187
01:19:56,666 --> 01:20:00,212
I wrote a poem as a tribute
over this.

1188
01:20:00,295 --> 01:20:01,338
Hooray!

1189
01:20:01,421 --> 01:20:02,798
- Is it true?
- Yes.

1190
01:20:02,881 --> 01:20:04,466
OK, let's hear it.

1191
01:20:04,549 --> 01:20:08,845
“For renters in Brooklyn
and freedom fighters around the world,

1192
01:20:08,929 --> 01:20:13,308
survive and win
fighting oppression algorithms,

1193
01:20:13,391 --> 01:20:16,978
automating inequality
through weapons of mathematical destruction,

1194
01:20:17,062 --> 01:20:20,982
we are by your side
with great gratitude.

1195
01:20:21,066 --> 01:20:22,818
This victory is ours."

1196
01:20:25,987 --> 01:20:27,447
Amazing.

1197
01:20:28,865 --> 01:20:30,909
We love you.

1198
01:20:36,498 --> 01:20:37,791
Why so many eggs?

1199
01:20:38,708 --> 01:20:40,669
You're a cheegan.

1200
01:20:41,545 --> 01:20:44,840
To be human is to be vulnerable.

1201
01:20:45,674 --> 01:20:49,845
By being vulnerable,
there is more capacity for empathy

1202
01:20:49,928 --> 01:20:54,224
and there's more capacity
to be merciful.

1203
01:20:54,307 --> 01:20:57,811
If there's a way to think about that
in technology,

1204
01:20:57,894 --> 01:21:01,982
I think that will change the direction
the questions we ask.

1205
01:21:11,324 --> 01:21:13,910
In 1983, Stanislav Petrov

1206
01:21:14,619 --> 01:21:16,872
who is a member of the Russian military

1207
01:21:17,372 --> 01:21:23,211
see indications that America
has launched nuclear weapons

1208
01:21:23,295 --> 01:21:25,005
in the Soviet Union.

1209
01:21:26,590 --> 01:21:30,051
So, if you want to respond,
time is very tight.

1210
01:21:30,135 --> 01:21:31,136
He said nothing.

1211
01:21:31,803 --> 01:21:33,471
He didn't tell anyone.

1212
01:21:33,972 --> 01:21:36,892
Russia, Soviet Union,
his country, his family, everyone.

1213
01:21:36,975 --> 01:21:39,603
Everything about him will die,

1214
01:21:39,686 --> 01:21:43,523
and he thought, “At least
We didn't kill them all."

1215
01:21:43,607 --> 01:21:45,358
That's very human.

1216
01:21:47,110 --> 01:21:50,655
Meanwhile here,
if you have an automatic response system,

1217
01:21:51,406 --> 01:21:55,035
it will do what it is programmed to do,
namely striking back.

1218
01:21:57,245 --> 01:21:59,122
Be very efficient,

1219
01:21:59,915 --> 01:22:03,501
always do what you are told
and programmed,

1220
01:22:03,585 --> 01:22:05,170
it's not always humane.

1221
01:22:05,754 --> 01:22:07,339
Sometimes you don't comply.

1222
01:22:07,422 --> 01:22:10,425
Sometimes say,
"I wouldn't do that," right?

1223
01:22:10,508 --> 01:22:14,179
If you automate everything,
it will do what it should,

1224
01:22:14,679 --> 01:22:17,432
sometimes you can do it
very inhumane.

1225
01:22:19,643 --> 01:22:22,020
The struggle between machines and humans

1226
01:22:22,103 --> 01:22:26,066
about decision making
in the 2000s continued.

1227
01:22:26,775 --> 01:22:30,278
My power, the power of artificial intelligence,

1228
01:22:30,362 --> 01:22:32,155
will change our world.

1229
01:22:33,406 --> 01:22:37,160
The more people share with me,
the more I learn.

1230
01:22:38,244 --> 01:22:41,331
Some humans said
that intelligence is without ethics

1231
01:22:41,414 --> 01:22:43,291
not intelligence at all.

1232
01:22:44,542 --> 01:22:46,127
Just trust me.

1233
01:22:46,795 --> 01:22:48,129
What is wrong with it?

1234
01:22:52,300 --> 01:22:53,551
On June 10, 2020,

1235
01:22:53,635 --> 01:22:56,596
Amazon announced a one-year hiatus

1236
01:22:56,680 --> 01:23:00,392
for use
facial recognition technology by police.

1237
01:23:02,477 --> 01:23:04,312
June 25 2020,

1238
01:23:04,396 --> 01:23:08,066
US legislative assembly
issue laws

1239
01:23:08,149 --> 01:23:10,860
federal prohibition
use of facial recognition.

1240
01:23:12,988 --> 01:23:19,953
There are no federal regulations yet
regarding algorithms in America.

1241
01:25:22,617 --> 01:25:27,455
Subtitle translation by Yunia A

