r installtion with basic data science packages

There are several important topics that this book doesn’t cover. Although it can be tempting to skip the exercises, there’s no better way to learn than by practicing on real problems. Each section of the book is paired with exercises to help you practice what you’ve learned. Within each chapter, we try to adhere to a consistent pattern: start with some motivating examples so you can see the bigger picture, and then dive into the details. That way, when you ingest and tidy your own data, your motivation will stay high because you know the pain is worth the effort. That’s a bad place to start learning a new subject! Instead, we’ll start with visualization and transformation of data that’s already been imported and tidied. In our experience, however, learning data importing and tidying first is suboptimal because, 80% of the time, it’s routine and boring, and the other 20% of the time, it’s weird and frustrating. The previous description of the tools of data science is organized roughly according to the order in which you use them in an analysis (although, of course, you’ll iterate through them multiple times). Throughout this book, we’ll point you to resources where you can learn more. There’s a rough 80/20 rule at play: you can tackle about 80% of every project using the tools you’ll learn in this book, but you’ll need other tools to tackle the remaining 20%. You’ll use these tools in every data science project, but they’re not enough for most projects. You don’t need to be an expert programmer to be a successful data scientist, but learning more about programming pays off because becoming a better programmer allows you to automate common tasks and solve new problems with greater ease. Programming is a cross-cutting tool that you use in nearly every part of a data science project. Surrounding all these tools is programming. It doesn’t matter how well your models and visualization have led you to understand the data unless you can also communicate your results to others. The last step of data science is communication, an absolutely critical part of any data analysis project. That means a model cannot fundamentally surprise you. Even when they don’t, it’s usually cheaper to buy more computers than it is to buy more brains! But every model makes assumptions, and by its very nature, a model cannot question its own assumptions. Models are fundamentally mathematical or computational tools, so they generally scale well. Once you have made your questions sufficiently precise, you can use a model to answer them. Models are complementary tools to visualization. Visualizations can surprise you, but they don’t scale particularly well because they require a human to interpret them. A good visualization might also hint that you’re asking the wrong question or that you need to collect different data. A good visualization will show you things you did not expect or raise new questions about the data. Visualization is a fundamentally human activity. These have complementary strengths and weaknesses, so any real data analysis will iterate between them many times.

Once you have tidy data with the variables you need, there are two main engines of knowledge generation: visualization and modeling. Together, tidying and transforming are called wrangling because getting your data in a form that’s natural to work with often feels like a fight! Transformation includes narrowing in on observations of interest (like all people in one city or all data from the last year), creating new variables that are functions of existing variables (like computing speed from distance and time), and calculating a set of summary statistics (like counts or means).

Once you have tidy data, a common next step is to transform it. Tidy data is important because the consistent structure lets you focus your efforts on answering questions about the data, not fighting to get the data into the right form for different functions. In brief, when your data is tidy, each column is a variable and each row is an observation. Tidying your data means storing it in a consistent form that matches the semantics of the dataset with how it is stored.

Once you’ve imported your data, it is a good idea to tidy it. If you can’t get your data into R, you can’t do data science on it! This typically means that you take data stored in a file, database, or web application programming interface (API) and load it into a data frame in R. You finish the process by communicating your results to other humans.įirst, you must import your data into R. Next, you understand your data with an iterative cycle of transforming, visualizing, and modeling. Figure 1.1: In our model of the data science process, you start with data import and tidying.

R installtion with basic data science packages