Using Difference-in-Differences to identify causal effects of COVID-19 policies
This paper reviews the challenges of performing Difference in differences analysis for studying the impact of COVID-19 policies.
- Packaged policies: multiple, similar policies are enacted around the same time and it is difficult to attribute the effect to one policy.
- Reverse causality: the policy can be a reaction of the disease prevalence and stress to the health systems.
- Voluntary precautions: people take precautions before policy gets enacted.
- Anticipation: behaviors may change in response to information about the policy itself. (e.g., stockpiling & shopping before lockdown)
- Spillovers: disease prevalence is correlated geographically and spill-over to nearby regions. Policies that help treatment group may also help the control group.
- Variation in policy timing:
- Measurement and scaling of the dependent variable:
So the recommendations are: 1. Estimate dynamics 2. Choose the control group wisely 3. be careful with regression DD 4. Sign the bias 5. Be clear about what is knowable