A Shiny interactive web application to quantify how robust inferences are to potential sources of bias (sensitivity analysis)

Written by: Josh Rosenberg

Primary Source: Joshua M. Rosenberg, January 17, 2018

We are happy to announce the release of an interactive web application, Konfound-It, to make it easy to quantify the conditions necessary to change an inference. For example, Konfound-It generates statements such as “XX% of the estimate would have to be due to bias to invalidate the inference” or “an omitted variable would have to be correlated with the outcome and predictor of interest (e.g., treatment) at ZZ to invalidate the inference.” Thus, Konfound-It provides a precise language for debating causal inferences.

Who would use such an approach in the course of their work? * Researcher receiving a revise and resubmit decision for a manuscript who is asked to carry out sensitivity analyses or alternative analyses as part of the revision of their manuscript. * Practitioners (including policymakers) seeking to understand whether evidence in a research report is strong enough to support action in their contexts * Data analysts and scholars debating whether a research community should make an inference in light of an estimated effect * Those analyzing observational data seeking to understand how trustworthy an inference is relative to potential omitted variables * Those analyzing data from a randomized controlled trial (RCT) trying to understand how generalizable the results from the study are to other contexts * Those needing to characterize the strengths of their research for the media / a wider audience, including audiences of teachers and administrators

Konfound-It takes four values – the estimated effect (such as an unstandardized regression coefficient), its standard error, the number of observations, and the number of covariates. Konfound-It returns output in the forms of publishable statements as well as figures to support the interpretation of the output. As you can see in the figure below, the Konfound-It interface is easy-to-use (and can be accessed from both a computer browser as well as mobile devices) and provides links to additional resources and a State procedure and R package for further use.

 

Konfound-It interactive web application

 

To learn more, check out Konfound-It here: http://konfound-it.com. If you have any questions or feedback, please do not hesitate to contact Ken Frank (and Joshua Rosenberg and Ran Xu) at kenfrank@msu.edu.

 

The following two tabs change content below.
Joshua M. Rosenberg is a Ph.D. student in the Educational Psychology and Educational Technology program at Michigan State University. In his research, Joshua focuses on how social and cultural factors affect teaching and learning with technologies, in order to better understand and design learning environments that support learning for all students. Joshua currently serves as the associate chair for the Technological Pedagogical Content Knowledge (TPACK) Special Interest Group in the Society for Information Technology and Teacher Education. Joshua was previously a high school science teacher, and holds degrees in education (M.A.) and biology (B.S.).