Improving Reproducible Research Practices in Schools of Public Health

Funnel with words pouring in - transparency, open access, corroboraion, replications, statistical power, results reporting, methodological training

We are pleased to announce a one-day symposium on Improving Reproducible Research Practices in Schools of Public Health. This symposium will be held at the Yale School of Public Health, New Haven, Connecticut on April 16th, 2018, and is organized by the Department of Environmental Health Sciences at the Yale School of Public Health and the Meta-Research Innovation Center at Stanford (METRICS).

Over the past few years, there has been an ongoing concern in the scientific community about the reliability and reproducibility of research. Numerous reports suggest that many experimental findings across various biomedical fields lack reproducibility. Questionable research practices, failure to adhere to rigorous scientific methods, and the pressure of academics to publish or perish are all likely to contribute to the growing reproducibility crisis. Although many scientific investigators are able to self-regulate and self-monitor their research, the responsibility for research findings ultimately rests with both the investigator and their host institution.

As the movement to facilitate and demand reproducibility in the scientific community continues to expand, it is important to start a discussion about the role that schools of public health and medicine can play when it comes to supporting, teaching, and promoting open science and reproducible research practices. In particular, there are numerous important questions that should be discussed. These include:

  • What policies can institutions adopt that promote the dissemination of statistical plans (protocols) and raw data? 
  • Are there certain performance metrics that can be used to evaluate students and faculty, that recognizes and rewards reproducible research practices? 
  • How can institutions ensure a high level of statistical literacy and numeracy for all investigators and teams performing biomedical research? 
  • Are there uniform policies regarding the expectation for research quality that research laboratories across all institutions can adopt? 
  • How should graduate students learn about transparency and reproducibility?
METRICS Logo

This one day symposium will bring together key stakeholders in science, including experts from schools of public health and medicine, and will be an ideal platform for discussing ways to promote reproducibility and transparency. The ultimate goal is to develop action steps that can be taken to train the next generation of public health and biomedical graduate students. 

April 16, 2018 • 9 am - 5 pm
Hope Lecture Hall 216
315 Cedar Street, New Haven, Connecticut

Register

Organizing Committee

  • John Ioannidis, Co-Director Meta-research Innovation Center at Stanford (METRICS) 
  • Vasilis Vasiliou, Chair, Environmental Health Sciences Yale School of Public Health 
  • Joshua Wallach, Postdoctoral Research Fellow, Collaboration for Research Integrity and Transparency (CRIT) and the Center for Outcomes Research and Evaluation (CORE), Yale-New Haven, Hospital

Tentative Schedule - April 16, 2018

9:00 AM: Opening remarks by Vasilis Vasiliou
Department Chair and Professor, Yale School of Public Health
Yale University

9:10 AM: Welcome address from Sten Vermund  
Dean, Professor of Epidemiology, Yale School of Public Health
Yale University

9:20 AM: Improving Research Practices: Overview
Presented by  John Ioannidis
Professor of Medicine, of Health Research and Policy, of Biomedical Science, and of Statistics, and Co-Director of the Meta-Research Innovation Center at Stanford (METRICS)
Stanford University

9:50 AM: Statistical and Data Errors: Prevent, Detect, Admit, Correct
Presented by David Allison
Provost Professor of Epidemiology and Biostatistics, Dean and Provost Professor, Bloomington School of Public Health
Indiana University

10:10 Morning Break

10:30 AM: Exploring new types of research indicators based on open source information
Presented by Kevin Boyack
President of SciTech Strategies, Inc.

10:50 AM: Computational Reproducibility
Presented by Victoria Stodden
Associate Professor, School of Information Sciences, The iSchool at Illinois
University of Illinois

11:10 Open Discussion with Harlan Krumholz, John Ioannidis, and Joshua Wallach (50 minutes) 

Harlan Krumholz
Harold H. Hines, Jr. Professor of Medicine (Cardiology) and Professor in the Institute for Social and Policy Studies, of Investigative Medicine and of Public Health (Health Policy); Director, Center for Outcomes Research and Evaluation, Yale-New Haven Hospital; Co-Director, Robert Wood Johnson Foundation Clinical Scholars Program, Yale University
 
12:00 Lunch

12:45: What resources can Schools of Public Health utilize to promote transparency and reproducibility?

Joshua Wallach, Vasilis Vasiliou, and John Ioannidis
Moderators

Library Resources (15 minutes)
Kate Nyhan, MLS
Research and Education Librarian, Cushing/Whitney Medical Library
Yale University

Center for Open Science (15 minutes)
Matthew Spitzer
Community Manager, Center for Open Science

The Yale University Open Data Access (YODA) Project (15 minutes)
Joseph Ross  
Associate Professor of Medicine (General Medicine), Institute for Social and Policy Studies, and of Public Health (Health Policy)
Yale University

30-minute discussion

2:00 PM: How should faculty/graduate students learn about transparency and reproducibility?

Melinda Pettigrew, Associate Dean, Yale School of Public Health, and Vasilis Vasiliou
Moderators

Donna Spiegelman
Professor of Epidemiologic Methods, Harvard School of Public Health
Harvard University

Limor Peer
Associate Director for Research, Institution for Social and Policy Studies, Office of the Provost
Yale University

David Jett
Program Director, Division of Translational Research
Director, NIH Countermeasures Against Chemical Threats (CounterACT) Program

David Allison
Provost Professor of Epidemiology and Biostatistics, Dean and Provost Professor Bloomington University
Indiana University

Open discussion

3:15-3.30: break

3:30: Are there uniform policies regarding the expectation for research quality that Schools of Public Health can adopt?
Open discussion with all participants (John Ioannidis, Vasilis Vasiliou, Sten Vermund, Joshua Wallach, moderators)

  • What can be done to promote reproducible practices in schools of public health?
  • What are resources that can be shared?
  • What are practices that can be widely endorsed?
  • What are obstacles and how can they be overcome?
  • What structures need to be in place, perhaps be created?
  • What research or observatory-type of input would be useful for guidance?
  • Are there specific assessments or studies that would be worthwhile conducting across multiple schools of public health?


4:20-4:40 PM Session 5: Action items to summarize

rev. 4.10.2018