It is estimated that more than $200 billion is spent on biomedical research every year. The return on that investment is too low, agree a group of experts who gathered at the Yale School of Public Health to discuss reproducibility and transparency in research last week.
Improving Reproducible Research Practices in Schools of Public Health
We are pleased to announce a one-day symposium on Improving Reproducible Research Practices in Schools of Public Health. This symposium will be held at the Yale School of Public Health, New Haven, Connecticut on April 16th, 2018, and is organized by the Department of Environmental Health Sciences at the Yale School of Public Health and the Meta-Research Innovation Center at Stanford (METRICS).
Over the past few years, there has been an ongoing concern in the scientific community about the reliability and reproducibility of research. Numerous reports suggest that many experimental findings across various biomedical fields lack reproducibility. Questionable research practices, failure to adhere to rigorous scientific methods, and the pressure of academics to publish or perish are all likely to contribute to the growing reproducibility crisis. Although many scientific investigators are able to self-regulate and self-monitor their research, the responsibility for research findings ultimately rests with both the investigator and their host institution.
As the movement to facilitate and demand reproducibility in the scientific community continues to expand, it is important to start a discussion about the role that schools of public health and medicine can play when it comes to supporting, teaching, and promoting open science and reproducible research practices. In particular, there are numerous important questions that should be discussed. These include:
- What policies can institutions adopt that promote the dissemination of statistical plans (protocols) and raw data?
- Are there certain performance metrics that can be used to evaluate students and faculty, that recognizes and rewards reproducible research practices?
- How can institutions ensure a high level of statistical literacy and numeracy for all investigators and teams performing biomedical research?
- Are there uniform policies regarding the expectation for research quality that research laboratories across all institutions can adopt?
- How should graduate students learn about transparency and reproducibility?
This one day symposium will bring together key stakeholders in science, including experts from schools of public health and medicine, and will be an ideal platform for discussing ways to promote reproducibility and transparency. The ultimate goal is to develop action steps that can be taken to train the next generation of public health and biomedical graduate students.
April 16, 2018 • 9 am - 5 pm
Hope Lecture Hall 216
315 Cedar Street, New Haven, Connecticut
- John Ioannidis, Co-Director Meta-research Innovation Center at Stanford (METRICS)
- Vasilis Vasiliou, Chair, Environmental Health Sciences Yale School of Public Health
- Joshua Wallach, Postdoctoral Research Fellow, Collaboration for Research Integrity and Transparency (CRIT) and the Center for Outcomes Research and Evaluation (CORE), Yale-New Haven, Hospital
Tentative Schedule - April 16, 2018
9:00 AM: Opening remarks by Vasilis Vasiliou
Department Chair and Professor, Yale School of Public Health
9:10 AM: Welcome address from Sten Vermund
Dean, Professor of Epidemiology, Yale School of Public Health
9:20 AM: Improving Research Practices: Overview
Presented by John Ioannidis
Professor of Medicine, of Health Research and Policy, of Biomedical Science, and of Statistics, and Co-Director of the Meta-Research Innovation Center at Stanford (METRICS)
9:50 AM: Statistical and Data Errors: Prevent, Detect, Admit, Correct
Presented by David Allison
Provost Professor of Epidemiology and Biostatistics, Dean and Provost Professor, Bloomington School of Public Health
10:10 Morning Break
Exploring new types of research indicators based on open source information
Presented by Kevin Boyack
President of SciTech Strategies, Inc.
10:50 AM: Computational Reproducibility
Presented by Victoria Stodden
Associate Professor, School of Information Sciences, The iSchool at Illinois
University of Illinois
11:10 Open Discussion with Harlan Krumholz, John Ioannidis, and Joshua Wallach (50 minutes)
Harold H. Hines, Jr. Professor of Medicine (Cardiology) and Professor in the Institute for Social and Policy Studies, of Investigative Medicine and of Public Health (Health Policy); Director, Center for Outcomes Research and Evaluation, Yale-New Haven Hospital; Co-Director, Robert Wood Johnson Foundation Clinical Scholars Program, Yale University
12:45: What resources can Schools of Public Health utilize to promote transparency and reproducibility?
Joshua Wallach, Vasilis Vasiliou, and John Ioannidis
Library Resources (15 minutes)
Kate Nyhan, MLS
Research and Education Librarian, Cushing/Whitney Medical Library
Center for Open Science (15 minutes)
Community Manager, Center for Open Science
The Yale University Open Data Access (YODA) Project (15 minutes)
Associate Professor of Medicine (General Medicine), Institute for Social and Policy Studies, and of Public Health (Health Policy)
2:00 PM: How should faculty/graduate students learn about transparency and reproducibility?
Melinda Pettigrew, Associate Dean, Yale School of Public Health, and Vasilis Vasiliou
Professor of Epidemiologic Methods, Harvard School of Public Health
Associate Director for Research, Institution for Social and Policy Studies, Office of the Provost
Program Director, Division of Translational Research
Director, NIH Countermeasures Against Chemical Threats (CounterACT) Program
Provost Professor of Epidemiology and Biostatistics, Dean and Provost Professor Bloomington University
3:30: Are there uniform policies regarding the expectation for research quality that Schools of Public Health can adopt?
Open discussion with all participants (John Ioannidis, Vasilis Vasiliou, Sten Vermund, Joshua Wallach, moderators)
- What can be done to promote reproducible practices in schools of public health?
- What are resources that can be shared?
- What are practices that can be widely endorsed?
- What are obstacles and how can they be overcome?
- What structures need to be in place, perhaps be created?
- What research or observatory-type of input would be useful for guidance?
- Are there specific assessments or studies that would be worthwhile conducting across multiple schools of public health?
4:20-4:40 PM Session 5: Action items to summarize