DSCI 521: Computing Platforms for Data Science

How to install, maintain, and use the data scientific software “stack”. The Unix operating system, integrated development environments, and problem solving strategies.

Course Learning Objectives

  • Use the Unix command line to navigate their computer’s filesystem.
  • Define and distinguish between absolute file paths and relative file paths.
  • Effectively use local and remote version control software (e.g., Git and GitHub) to organize projects and manage file versions.
  • Create, edit and run reproducible literate Python and R code documents (e.g., reports and presentations) using Jupyter and RMarkdown.
  • Write and edit Markdown and in-line LaTeX syntax within literate code documents.
  • Define and correctly use a project working directory.
  • Diagnose and troubleshoot programming and development environment problems, and explain how such problems can be avoided.

Lesson Learning Objectives and Readings

Learning objectives are grouped by week. Some weeks split one set of objectives across several lessons, and some lessons carry more than one set.

Orientation

JupyterLab Orientation

  1. Launch JupyterLab
  2. Use the Notebook interface inside Jupyterlab
  3. Know your way around the JupyterLab user interface

RStudio Orientation

  1. Launch RStudio
  2. Use the .R script and RMarkdown .Rmd interface inside RStudio
  3. Know your way around the RStudio user interface

Positron Orientation

  1. Launch Positron and open an assignment as a folder
  2. Know your way around the Positron user interface
  3. Start, switch, and restart an R or Python session
  4. Run code from a script, a Jupyter notebook, and a Quarto document

Week 1

MDS Software and Bash

  1. Recognize the directory hierarchy as it is commonly represented in diagrams, paths, and file explorer software.
  2. Distinguish common operators and representations of the different filesystem elements typically used in Bash.
  3. Explore the filesystem using Bash commands as ls, pwd and cd.
  4. Translate an absolute path into a relative path and vice versa.
  5. Use command-line arguments to produce alternative outputs of commands.
  6. Create, edit, move, and delete files and folders using the command line and Positron.

Git, GitHub, and SSH

  1. Implement SSH authentication
  2. Differentiate between the use of GitHub as a remote hosting service for version control and Git as a version control system.
  3. Create a Git repository.
  4. Implement Git’s clone, add, status, commit, pull, and push operations on the command line and their equivalent use in Positron.
  5. Understand what implies using the staging area in a Git workflow.
  6. Recognize the commit as the primary building block for storing a project version characterized by an attached message and a hash that serves as a unique identifier.

Readings

Week 2

Quarto and GitHub Pages

  1. Use the quarto terminal command to create different quarto projects
  2. Create and edit a Quarto website
  3. Discover how GitHub can be used to serve static websites
  4. Modify a GitHub repository to publish a website

Quarto, RMarkdown, and Jupyter Slides

  1. Understand how Quarto extend R Markdown documents functionalities.
  2. Explore different data science products to communicate your results: slides, books and websites.
  3. Create slides using Jupyter Notebook and Quarto slides with reveal.js
  4. Recognize Jupyter Book, bookdown, and Quarto book as long-form output formats.
  5. Sharing rendered HTML files publicly via GitHub Pages.

Readings

Week 3

Git: History, Conflicts, and Ignores

  1. Explore the Git history via git log in the terminal and GitHub.
  2. Compare commits using git diff in the terminal and GitHub.
  3. Solve merge conflicts at the command line and in Positron.
  4. Save transitory changes with git stash.
  5. Manage to avoid pushing specific local files by including a .gitignore.
  6. Differenciate among different ways to restore your project history (git reset --hard/--soft, git revert) when working on an older version of a project.

RStudio Projects and Literate Programming

  1. Create RProjects in RStudio using here to define robust file paths.
  2. Detect the basic components of a dynamic document in Jupyter Notebooks and in R Markdown.
  3. Explain markdown usage in relation to dynamic documents.
  4. Differentiate between code chunks and code cells in RMarkdown and Jupyter Notebooks.
  5. Select appropriate code chunk options for RMarkdown.
  6. Use semantic line breaks for version control files.
  7. Specify metadata in the YAML header block.

Readings

Week 4

Virtual Environments: uv and renv

  1. Understand what a computational environment is and how it can ensure the reproducibility of a project
  2. Differentiate Python, PyPI, pip, venv, and uv
  3. Distinguish between the packages a project declares (pyproject.toml, DESCRIPTION) and the packages a project locks (uv.lock, renv.lock)
  4. Manage packages and environments in Python using uv
  5. Manage packages and environments in R using renv

Organization of Data Science Projects

  1. Describe how a data analysis project is organized, and how that organization differs from a code project.
  2. Distinguish raw data from processed data, and explain why raw data files are never overwritten.
  3. Apply the three principles of a good file name: machine readable, human readable, and plays well with default ordering.
  4. Explain how informative file names and locations make a project self-documenting.

Introduction to Regular Expressions (RegEx)

  1. Understand the basic syntax and functionality of regular expressions (regex) for pattern matching.
  2. Explore the use of special characters, ranges, and anchors in regex to match specific patterns within text.
  3. Apply regex to search, extract, and manipulate data in various formats using practical examples.
  4. Use regular expressions to navigate and organize files within the filesystem.

Asking Effective Questions

  1. Explain why a well-posed question gets a faster and better answer.
  2. Construct a minimal reproducible example (MRE) for a programming problem.
  3. Describe a problem clearly, including what you have already tried and the full error message.

Readings

Attributions

Materials were inspired, re-used and re-mixed from the following sources:

License

The UBC Master of Data Science DSCI 521: Computing Platforms for Data Science course materials here are licensed under the Creative Commons Attribution 2.5 Canada License (CC BY 2.5 CA). If re-using/re-mixing please provide attribution and link to this webpage.