Using Difference-in-Differences to identify causal effects of COVID-19 policies

This paper reviews the challenges of performing Difference in differences analysis for studying the impact of COVID-19 policies.

  1. Packaged policies: multiple, similar policies are enacted around the same time and it is difficult to attribute the effect to one policy.
  2. Reverse causality: the policy can be a reaction of the disease prevalence and stress to the health systems.
  3. Voluntary precautions: people take precautions before policy gets enacted.
  4. Anticipation: behaviors may change in response to information about the policy itself. (e.g., stockpiling & shopping before lockdown)
  5. Spillovers: disease prevalence is correlated geographically and spill-over to nearby regions. Policies that help treatment group may also help the control group.
  6. Variation in policy timing:
  7. Measurement and scaling of the dependent variable:

So the recommendations are: 1. Estimate dynamics 2. Choose the control group wisely 3. be careful with regression DD 4. Sign the bias 5. Be clear about what is knowable