“Why ‘Does this Effect Generalize’ Is a Bad Question and What to Ask Instead” by Aaron Kaufman (NYU Abu Dhabi), on June 4, 4pm-5:30pm

Aaron Kaufman (NYU Abu Dhabi) on June 4, 4pm-5:30pm — Political Methodology / Causal Inference Joint Workshop

Prof. Aaron Kaufman (NYU Abu Dhabi) will present his work in our Political Methodology Workshop (joint with Waseda Causal Inference Workshop series) on June 4. The workshop is open to everyone and no pre-regisration is required. Please join us! Details are below.

Presenter: Aaron Kaufman (NYU Abu Dhabi) https://aaronrkaufman.com/

Date and Time: June 4 (Thursday), 4 – 5:30pm

Location: Building 10, Room 103, Waseda Campus

Title: Why ‘Does this Effect Generalize’ Is a Bad Question and What to Ask Instead

Abstract:
Social scientists often ask whether experimental results generalize across contexts, and the question is usually posed in terms of whether average treatment effects replicate. This paper argues that for many theories, the more relevant question is whether a parsimoniously parameterized model of effects and effect heterogeneity yields stable results across settings. We call this problem parametric generalizability. The paper develops a causal framework that distinguishes unconditional generalizability of marginal effects from conditional generalizability of treatment effects given effect modifiers. It then shows how conditional generalizability can be operationalized in three ways: nonparametric comparison of conditional effects, reweighting to target populations, and parametric projection of effect heterogeneity onto a low-dimensional model. The parametric approach yields a practical set of diagnostics for multi-site experiments: site-by-site tests of the theoretical coefficient pattern, cross-site tests of coefficient homogeneity, and equivalence-style confidence regions describing the range of coefficient variation across contexts. This shifts the inferential target from asking whether effects are identical within agnostically defined regions of the covariate space to asking whether a parsimonious theoretical approximation yields stable results and is thus portable.

Language: English