Model Uncertainty and Robustness

Empirical results can be very dependent on model specification. How can analysts show model robustness in their research?

NEW articles:

We Ran 9 Billion Regressions: Eliminating False Positives Through Computational Model Robustness.” 2018. With John Muñoz. Sociological Methodology. Vol 48(1): 1-33.

Symposium article, with commentary by Bruce Western and Robert O’Brien.

Download the replication package

Rejoinder: Can We Weight Models by Their Probability of Being True?” 2018. With John Muñoz. Sociological Methodology. Vol 48(1): 43–51.

Model Uncertainty and the Crisis in Science.” 2018. Socius.

 

 Model Uncertainty and Robustness: A Computational Framework for Multi-Model Analysis.  Sociological Methods and Research. February 2017. Vol. 46(1): 3 – 40. (Lead article, with Katherine Holsteen)

robustness

Download: This do file installs our Stata program, loads in data sets, and replicates all the analyses in the paper.

OR: Paste the following command into Stata:

do http://web.stanford.edu/~cy10/public/mrobust/install_mrobust.do

OR: type “ssc describe mrobust” in Stata

 

The multi-model analysis framework allows researchers to

(1) estimate thousands of regression models across combinations of control variables, functional forms, alternative variable definitions, and estimation commands;

(2) display the resulting modeling distribution of estimates; and

(3) discover which aspects of model specification have the most influence on an estimate of interest.

This allows analysts to clarify and demonstrate which modeling assumptions are essential to their empirical findings, and which are not.

 

This is part of an enduring theme in my research, focusing on robust results. My earlier research:

Model Uncertainty in Sociological Research: An Application to Religion and Economic Growth. American Sociological Review. June 2009.

 

New working paper, applying model robustness methods: Patients as Consumers in the Market for Medicine: The Halo Effect of Hospitality (with Xinxiang Chen)