Original subtitles

RAFAEL IRIZARRY: The data sets used in this series

have been made available to you as R objects, specifically as data frames.

The US murders data, the reported heights data, the Gapminder data,

and the poll data are all examples.

These data sets come included in the dslabs package,

and we loaded them using the data function.

Furthermore, we have made the data available in what

is referred to as tidy form, a concept we define later in this course.

The tidyverse packages and functions assume that the data is tidy,

and this assumption is a big part of the reason

these packages work so well together.

We did quite a bit of work behind the scenes

to get the original raw data into the tidy tables you work with.

However, in a typical data science project,

it is much more typical for the data to be in a file, a database,

or extracted from a document, including web pages, tweets, or PDF.

In these cases, the first step is to import the data into R,

and when using the tidy verse, tidy up the data.

The first step in the data analysis process

usually involves several often complicated steps

to convert data from its raw form to the tidy form that greatly

facilitates the rest of the analysis.

We refer to this process as data wrangling.

In this course, we cover several common steps

of the data wrangling process including importing data into R from files,

tidying data, string processing, HTML parsing, working with dates and times,

and text mining.

Rarely are all these wrangling steps necessary in a single analysis,

but a data scientist will likely face them all at some point.

Some of the examples we used to demonstrate data wrangling techniques

are based on the work we did to convert the raw data into the tidy data

sets provided by the dslab packages and used in the series as examples.