Reproducibility of Research: Issues and Proposed Remedies


  • Tuesday, March 7, 2017
    7:00 PM  -  8:30 PM
    Speaker Welcome Dinner  (SpeakerWelcomeDinner)

    For invited speakers and panelists at The Library, St. Gregory Hotel.

    There is no formal program, just an opportunity to meet the other speakers and talk to PNAS staff about the submission and review process for publication.

  • Wednesday, March 8, 2017
    8:30 AM  -  12:10 PM
    Session I

    8:30 AM  Colloquium Call to Order
                    David Allison, University of Alabama at Birmingham

    8:35 AM Welcoming Remarks
                   Marcia McNutt, President, National Academy of Sciences

    8:50 AM  Remembering Stephen Feinberg, Robert Groves, Provost, Georgetown University

    9:00 AM  Framing the Issues, Victoria Stodden, University of Illinois

    Part 1 – Taxonomy of Issues & Challenges to Improving Research Reproducibility

    9:25 AM Issues of Design & Measurement, Kay Dickersin, Johns Hopkins University

    10:15 AM Break 

    10:35 AM  Issues of Analysis, David B. Allison, University of Alabama at Birmingham

    11:20 AM Taking Stock/Making Change: Surveying Practices, Assessing Perspectives, and Re-training Personnel as methods of supporting reproducibility in a learning health system, Madhu Mazumdar, Icahn School of Medicine at Mount Sinai

    12:10 PM  -  1:10 PM
    Mar 8 Lunch
    Buffet lunch at the NAS Building included with registration.
    1:10 PM  -  5:00 PM
    Session II

    1:10 PM Incomplete Reporting in Primary Scientific Literature, Lehana Thabane, McMaster University

    2:00 PM Honest learning for the healthcare system: large-scale evidence from real-world data, David Madigan, Columbia University

    2:50 PM Break

    3:10 PM Distortion of research results in primary scientific literature and beyond , Isabelle Boutron, Centre d'Epidémiologie Clinique, Hôpital Hôtel Dieu  

    4:00 PM PANEL DISCUSSION – Moderator: Andrew W. Brown, University of Alabama at Birmingham
    Panelists: Inder Verma, PNAS; Phil Campbell, Nature; Kelvin Droegemeier, Oklahoma University; Veronique Kiermer, PloS; Ginny Barbour, COPE

    5:00 PM  -  7:00 PM
    Annual Arthur M. Sackler Lecture

    5:00 PM         Reception


    6:00 PM         17th Annual Arthur M. Sackler Public Lecture
                           The Challenge of Reproducibility in the Biomedical Sciences

                           presented by Randy Schekman, University of California, Berkeley

    Keynote Speaker:
  • Thursday, March 9, 2017
    8:00 AM  -  12:00 PM
    Session III



    8:00 AM Raw Data Depositing and Sharing, Victoria Stodden, University of Illinois 


    8:45 AM The Role of Registries and Repositories, Roberta W. Scherer, Johns Hopkins University


    9:30 AM Statistical power and evidence in the psychological literature, Joachim Vandekerckhove, University of California, Irvine


    10:15 AM Break


    10:30 AM Rigorous Assessment of Whether A Result can be Said to Have Been Replicated, Yoav Benjamini, Tel Aviv University


    11:15 AM Training the Current and the Next Generations in Statistics, Emery Brown, Massachusetts Institute of Technology

    12:00 PM  -  1:00 PM
    Mar 9 Lunch
    Lunch at the NAS Building included with registration.
    1:00 PM  -  6:00 PM
    Session IV

    1:00 PM Biomedical replication studies, Elizabeth Iorns, Science Exchange


    1:45 PM Agents of Cultural Behavior Change, Susan T. Fiske, Princeton University


    2:30 PM Post-Publication Peer-Review and Certificate Systems, Hilda Bastian, National Center for Biotechnology Information, NIH


    3:15 PM Break


    3:30 PM Science Journalism and Public Dialog, Trevor Butterworth, Sense About Science USA


    4:15 PM Government Perspectives, Catherine Woteki , Former Under Secretary for Research, Education and Economics, USDA


    5:00 PM PANEL DISCUSSION – Dialogue and Reactions on Day 2 Topics  Moderator: Kathryn A. Kaiser, University of Alabama at Birmingham

    Panelists:  Daniele Fanelli, Stanford University; Jeffrey Flier, Harvard Medical School

  • Friday, March 10, 2017
    8:20 AM  -  12:00 PM
    Session V



    8:20 AM The Problem with Remedies: The Need to Study Unintended Consequences, Richard M. Shiffrin, Indiana University, Bloomington


    9:10 AM Establishing and Validating Ongoing Monitoring Systems for the Quality of Individual Research Reports, Tania B. Huedo-Medina, University of Connecticut


    10:00 AM Break


    10:20 AM Minimizing Inadvertent Distortion of Science by Scientists and Media, Kathleen Hall Jamieson, University of Pennsylvania


    11:10 AM Systems Matter: Research Environments and Institutional Integrity, C. K. Gunsalus, University of Illinois

    12:00 PM  -  1:00 PM
    March 10 lunch
    Lunch buffet at the NAS Building included in registration
    1:00 PM  -  4:00 PM
    Session VI

    1:00 PM Testing Interventions at the Levels of the Journal, David Moher, University of Ottawa

    1:50 PM Improving Reproducibility, Brian Nosek, University of Virginia, Center for Open Science (COS) 

    2:40 PM Break

    3:00 PM New statistical approaches to reproducibility, Giovanni  Parmigiani, Dana Farber Cancer Institute, Harvard University

    3:50 PM Sharing scientific data and replicability, Keith Baggerly, MD Anderson Cancer Center

    4:40 PM Closing Remarks, Richard Shiffrin, Indiana University, Bloomington