Experiments and Causal Inference

Admin

• Problem sets and slides are posted on Canvas.
• About me

Lessons

Each lesson will have a set of readings that you are expected to read before the class session. Readings include Colab notebooks, sections of textbooks, and course notes. We will use the Colab notebooks during class and in the problem sets. The course notes highlight the key concepts of the lesson with my own commentary. The supplemental resources are materials that may be helpful if you need to review or want more detail.

Lesson 1: Causality fundamentals

• Readings
  • Notes: Causality fundamentals
  • Colab notebook: Law of large numbers and selection bias
  • SCMixtape: "Potential outcomes causal model" (pages 81-103)
• Supplemental resources
  • MWCCI, Part I "Counterfactual Causality and Empirical Research in the Social Sciences"

Lesson 2: Random sampling and statistical tests

• Readings
  • Notes: Data and uncertainty
  • Colab notebook: Simulating and analyzing random variables with Python
  • PEEGES, Part II, Chapter 3: "Power analysis and the detection of effects" (pages 47-67)
  • HFC, Sections 3.2 and 3.3, Sampling and error rates (pages 26-39)
• Supplemental resources
  • LLPython, Chapters 1-5: For general review of simulation and numerical computing in Python.
  • HFC, Chapters 1-3. Chapter 1 is a short review of probability. Chapters 2 and 3 analyze experiment design using the framework of signal detection theory. If you think having a slightly different perspective on the problem is helpful, I recommend reading this.

Lesson 3: Regression for estimation in experiments

• Readings
  • Colab notebook: Experiments and regression
  • SCMixtape, "Properties of Regression" (pages 35-65)
• Supplemental resources
  • MWCCI, Chapters 4 (matching) and 5 (regression)

Lesson 4: Practical complications and biased ATE estimates

• Readings
  • Notes: Treatment noncompliance
  • Colab notebook

Lesson 5: Practical complications and inference

• Readings
  • Colab notebook
  • HFC, Chapter 5, The multiple testing problem (pages 63-66)
  • HFC, Section 10.4, Excess successes from optional stopping (pages 117-120)

Lesson 6: Decision-making and experiments

• Readings
  • Colab notebook
  • Decision strategies (bonus material) [Colab notebook]
  • Reading: Kim Jae. (2015) "How to Choose the Level of Significance: A Pedagogical Note."
• Supplemental reading: PIDT, sections 13.1.{1,2,3} "Measuring the value of information", 13.2.3 "Hypothesis testing"

Reference books

Digital copies are available from the library. Open access texts can be downloaded from the publisher free of charge.

• SCMixtape = Scott Cunningham. Causal Inference: The Mixtape. (Open access)
• MWCCI = Morgan and Winship. Counterfactuals and Causal Inference. Cambridge University Press, 2007.
• PEEGES = Paul Ellis. The Essential Guide to Effect Sizes: Statistical Power, Meta-Analysis, and the Interpretation of Research Results. Cambridge University Press, 2010.
• PIDT = Giovanni Parmigiani and Lurdes Inoue. Decision Theory: Principles and Approaches. Wiley, 2009.
• HFC = Herzog, Francis, and Clark. Understanding Statistics and Experimental Design. Springer, 2019. (Open access)
• LLPython = Linge and Langtangen. Programming for Computations - Python: A gentle introduction to numerical simulations with Python 3.6 Springer, 2020. (Open access)