Research & Software
Current Projects
-
Joint Sensitivity Analysis for Multiple Assumptions: Unpacking Racial Disparity in Police Use of Force
Joint work with Tom Leavitt and Luke Miratrix on a joint sensitivity analysis for bias in police encounters and stops, applied to NYPD Stop, Question, and Frisk data (2003-2013). We show how dependence between encounter bias and stop bias changes inference about racial disparities in police use of force, and we assess robustness as violations increase. Revise and resubmit at the Journal of the American Statistical Association.
-
Powerful Detection of Effects in Block-Randomized Experiments by Testing Hypotheses in Order
Joint work with Nuole Chen and David Kim, including the manytestsr R package. We present a method based on the Sequential Intersection Union Principle and energy or distance test statistics to detect where policy effects appear in block-randomized experiments while controlling family-wise error. We compare to block-by-block testing with p-value adjustment and show improved power in simulations and 25 education trials. In preparation.
-
Randomization Tests for Distributions of Individual Treatment Effects Combining Multiple Rank Statistics
Joint work with Xinran Li, David Kim, and Yongchang Su. We develop a combined Stephenson rank test that avoids choosing a single subset size, controls false positives, and improves power for concentrated effects and outliers. We derive permutation distributions for randomized studies with and without blocks. In preparation.
-
Experimental Reasoning in Process Tracing: A Method for Calculating P-Values for Qualitative Causal Inference
Joint work with Matias Lopez. We adapt Fisher's urn model to qualitative process tracing to compute p-values for evidence supporting one theory over another. The method includes sensitivity analysis for observation bias and a framework for weighing evidence strength, illustrated with simulations and replications. Under review.
-
The Causal Inference for Social Impact (CISIL) Data Challenge
- [link]
Joint work with Carrie Cihak, Betsy Rajala, Quinn Waeiss, Ryan Moore, Laura Stoker, Laura Feeney, Ben Hansen, Crystal Hall, and Anjali Chainani. We recruited 30 teams to evaluate transportation policy impacts in King County, WA to study how analytic decisions affect results and policy conclusions. Launched in February 2022 with 31 teams from 10 countries. In preparation.
Software
-
RItools
- [code]
This R package implements randomization inference methods for assessing balance in matched or stratified observational studies or randomized studies or causal effects in complete, blocked/stratified, and/or cluster-randomized studies. It implements the $d^2$ test for omnibus tests of the null hypothesis of no relationship between any covariate and treatment (for balance tests) or tests of the null hypothesis of no effects on any of multiple outcomes from a single treatment.
-
manytestsr (development)
- [code]
A package to do many tests to localize causal effects in (sets of) experimental blocks
-
DrBristol (development)
- [code]
A set of functions to compute p-values and perform sensitivity analysis, adapting Fisher’s p-value test to case studies and process tracing following López and Bowers (2025). It uses unbiased and biased urn models to draw null distributions in the absence of randomization.
-
CMRSS (development)
- [code]
R package for conducting randomization inference for quantiles of individual treatment effects, using combined rank sum statistics, both for completely randomized and stratified randomized experiments