A Reproducible Paper

• Type of event: Seminar
• Institutions: Faculty of Psychology and Human Movement Science
• Funding period: 01.09.2024 to 30.09.2025
• Short title: Repro

Orientation of the project

Research should be reproducible. The same analysis of the same data should yield the same result. This sounds obvious at first, but many studies show that a large proportion of scientific literature in various disciplines is not reproducible. Research data and analysis codes are often not readily shared, making it virtually impossible for other researchers to independently verify published scientific results. In some cases, even the scientists who conducted the research are unable to reproduce their own results, even shortly after the work has been published. The public's investment in a particular research study is often limited to a single article, often behind a paywall, in PDF format on the website of a for-profit publisher.

The reproducible conduct of research is hampered by at least two factors: First, incentives in academic research are not conducive to reproducibility, as the quantity of publications and the impact factor of the journals in which they are published are often still given greater weight than the quality, robustness, and reproducibility of the work when it comes to hiring and promotion. Second, it is indeed difficult to make research reproducible, and early-career researchers often do not receive targeted and comprehensive training in the relevant practices and tools.

^{Scott Graham / unsplash}

Reproducing the same results on another computer is often not a trivial problem. For example, it not only requires that all code and data be available in an accessible format, but also that the same software (or computational environment) can be recreated. More broadly, managing research in a reproducible and transparent way along the entire research lifecycle, from initial idea to publication and beyond, is a considerable challenge. Fortunately, scientists can learn about practices and tools from other disciplines, particularly software engineering, that have significantly professionalized collaborative work on digital objects like code and data. Among several things, this involves tracking changes in digital objects using version control systems like Git, good practices for code and data management, as well as creating stable and transportable computational environments using software containers like Docker.

This course provided an introduction to the tools and procedures that enable young scientists to make their research reproducible.

Review and results

Following the successful approach of our previous course on “Version Control of Code and Data with Git”, we focused on the development of an online learning resource, with the preliminary title “The Repro Book” (made available here), modeled after our “Version Control Book”. This online guide was adjusted to the structure of the seminar and serve as the course’s textbook. During the course, participants were ntroduced to a new concept in each session and then implemented the newly learned tool or practice with hands-on exercises. These exercises focused on working on a small-scale research project from conception to dissemination, using the methods for reproducible research introduced in the course.

The main outcome of the teaching project was that course participants had the opportunity to learn about reproducible work in a scientific context, in particular through practical application in a small fictional research project. Through practical exercises and learning about concepts such as metadata, community guidelines, renv, good coding practices, and literate programming using the Quarto, R, and RStudio software, they acquired in-depth knowledge of how to structure a research project in a way that is easily reproducible.

Tips from lecturers for lecturers

The central element of digital competence development was the most transparent, public, and collaborative development of teaching and learning resources possible with Git and GitHub, as well as with the Open Science Framework, within the time available. Individual teaching content was applied to the creation of teaching content. This approach follows good scientific principles of transparency, openness, and reproducibility.

The teaching project also contributed to the development of the teacher's didactic skills. Various didactic methods were tested as part of the teaching project, ranging from partner and individual work to quizzes, demonstrations, and exercises. A conscious focus was always placed on creating plenty of time and space for implementation, during which participants could apply what they had learned using concrete exercises and examples.

Persons involved

Dr. Lennart Wittkuhn

Researcher

Specialisation:

• Cognitive neuroscience, in particular learning and change mechanisms

Kontakt