This week covers:
- An intro to Git and Github for sharing code
- Command line tools
- R and the Tidyverse
Install tools: Ubuntu on Windows, GitHub for Windows, R, and RStudio
- Open http://aka.ms/wslstore and select Ubuntu on Windows
- If this seems like it's hanging, hit enter
- Create a username and password
- Updates all packages with
sudo apt-get updateandsudo apt-get upgrade
- Check that you have git under bash by typing
git --versionin the terminal - Install GitHub for Windows
- Download and install R from a CRAN mirror
- Download and install RStudio
- Open RStudio and install the
tidyversepackage, which includesdplyr,ggplot2, and more:install.packages('tidyverse', dependencies = T)
- You'll need a plain text editing program
- If you are familiar with emacs or vim, you can install them in Ubuntu with
sudo apt-get install emacsorsudo apt-get install vim - Otherwise consider Visual Studio Code, Atom, or Sublime
- Check your editor's settings for unix-friendly line endings
- Files that you create in Ubuntu on Windows get stored in a somewhat hidden location within the Windows filesystem
- To make it easier to find files you work on in Ubuntu, do the following:
- Open a bash shell
- Go to your home directory:
cd ~ - Create a symbolic link to your Documents folder:
ln -s /mnt/c/Users/<your name>/Documents ~/Documents - Change to this directory:
cd ~/Documents - Do all of your work, including the following section, from within this folder, which you'll be able to see under "Documents" in the Windows Explorer
- Sign up for a free GitHub account
- Then follow this guide to fork your own copy of the course repository
- Clone a copy of your forked repository, which should be located at
https://github.com/<yourusername>/coursework.git, to your local machine - Once that's done, create a new file in the
week1/studentsdirectory,<yourfirstname>.txt(e.g.,jake.txt) - Use
git addto add the file to your local repository - Use
git commitandgit pushto commit and push your changes to your copy of the repository - Then issue a pull request to send the changes back to the original course repository
- Finally, sync changes from the main repo to your fork with
git pull upstream master
- Codecademy's interactive introduction to git
- A full hour-long introductory video
- More resources from GitHub available here and here
- And here's a handy cheatsheet
- See this intro to the command line notebook
- Read through Lifehacker's command line primer
- See Linux Journey's shell lesson
- See this crash course for more details on commonly used commands
- Check out Software Carpentry's guide to the Unix shell
- Review this wikibook on data analysis on the command line, covering
cut,grep,wc,uniq,sort, etc - Learn awk in 20 minutes
- Check out some more advanced tools for Data Science at the Command Line
- Do Codecademy's interactive command line tutorial (the free portion)
- See these Introduction to Counting slides
- Pull changes from the msr-ds3/coursework repo:
git pull upstream master - Use the download_trips.sh file to download Citibike trip data by running
bash download_trips.shor./download_trips.sh - Fill in solutions under each comment in citibike.sh using the
201402-citibike-tripdata.csvfile
- Make sure to save your work and push it to GitHub. Do this in three steps:
git addandgit commitand new files to your local repository. (Omit large data files.)git pull upstream masterto grab changes from this repository, and resolve any merge conflicts, commiting the final results.git push origin masterto push things back up to your GitHub fork of the course repository.
- Think about how to write a
musical_pairs.shscript to determine your programming partner each day
- See the Data Wrangling in R slides
- Review intro_to_r.ipynb for an introduction to R
- Do the free portion of Codecademy's introduction to R, chapters 1, 2, and 3
- See chapters 1, 2, and 4 of R for Data Science for background on using R and Rstudio (chapter numbers correspond to the online edition)
- Use the musical pairs script to determine your programming partner each day
- Fill in solutions to the counting exercises under each comment in citibike.R
- Read chapter 5 of R for Data Science and do the following exercises:
-
References:
- Basic types: (numeric, character, logical, factor)
- Vectors, lists, dataframes: a one page reference and more details
- Cyclismo's more extensive tutorial
- Rstudio's data wrangling cheatsheet
- The tidyverse style guide
- Hadley Wickham's style guide
- The dplyr vignette
- Sean Anderson's dplyr and pipes examples (code on github)
-
Tutorials:
- DataCamp's introduction to R tutorials (or Hadley's Advanced R if you're a pro)
- DataCamp's Data Manipulation in R tutorial
- Datacamp's Introduction to the Tidyverse tutorial
- See the Data visualization slides
- Review visualization_with_ggplot2.ipynb for an introduction to data visualization with ggplot2
- Read chapter 3 of the online edition of R for Data Science and do the following exercises:
- Citibike plots
- Modify and use the
download_trips.shscript to grab all months of trip data from 2014 - Run the load_trips.R file to generate
trips.RData - Write code in plot_trips.R to create visualizations using
trips.RData
- Modify and use the
- Tutorials:
- DataCamp's Data Visualization with ggplot2 (part 1) tutorial
- References:
- RStudio's ggplot2 cheatsheet
- Sean Anderson's ggplot2 slides (code) for more examples
- The R Graphics Cookbook
- The official ggplot2 docs
- Videos on Visualizing Data with ggplot2
- Review combine_and_reshape_in_r.ipynb on joins with dplyr and reshaping with tidyr
- Finish up the Citibike plotting exercises in plot_trips.R, including the plots that involve reshaping data
- Read chapters 12 and 13 of R for Data Science on tidyr and joins
- Do the following exercises from R for Data Science:
- Do part 1 of Datacamp's Cleaning Data in R tutorial
- Additional references:
- The tidyr vignette on tidy data
- The dplyr vignette on two-table verbs for joins
- A visual guide to joins
- Read Chapter 27 of R for Data Science on Rmarkdown
- Do the following exercises: