Regression analysis

A way to think and generalize the statistical models is breaking them into a stochastic and a systematic component.

Yif(θi,α)(stochastic)θi=g(xi,β)(systematic) \begin{aligned} Y_i \sim f(\theta_i, \alpha) \quad &\text{(stochastic)} \\ \theta_i = g(x_i, \beta)\quad &\text{(systematic)} \end{aligned}

where there is always an estimation uncertainty – the lack of knowledge of β\beta and α\alpha – and fundamental uncertainty (stochastic component).


Marginal effect

Marginal effect refers to “the slope of the regression surface with respect to a given covariate” (Leeper, 2017). In other words, it is about how much the dependent variable changes when we change the given independent variable. Of course this is not necessarily a constant. The slope can change based on the value of the independent variables. Thus we consider a representative marginal effect. A common method is calculating the “average marginal effects (AMEs)”, which is the mean of the marginal effect for every data point. Alternatively, the marginal effect can be calculated at the means of independent variables, “marginal effects at means (MEMs)”, or at a representative values (MERs). AMEs are most ‘data-driven’ and reflect the full data distribution.