Welcome to the Price Statistics Reproducibility Project

Welcome to the project aiming to help researchers in the field of price statistics make their projects more reproducible and help push open science in the discipline.

Why care about open science?

When only the research paper is available, we struggle to understand and easily apply the findings as we cannot reproduce the results or reuse the code. We also cannot expand the research into new directions and get insight into related but unanswered questions and are slower at learning from each other. In a sense, individually each researcher is less well off, and collectively, the discipline is slower at getting to a consensus on an approach.

The idea of this project is to help researchers by providing approachable guidance on how all digital artifacts for their research projects (such as code) can be published openly, as well as help bring together all the open datasets that are part of the discipline so that input data for projects can be open and not the proprietary datasets that are often the default.

How does it work in practice?

The aim can be visualized using the (input) data and analysis quadrant commonly applied for this case (see Figure 1). With proprietary data and non-open analysis (i.e. code and other digital artifacts), generalizability is the default. However, if the dataset used for the research is shared and the analysis is open, the method can be reproduced exactly. The goal of the project is to help make this happen. The backup is replicability, where the analysis is shared but different datasets are used (such as because the data cannot be shared).¹

Figure 1: Definitions of Reproducible Research, inspired by the Turing Way.

Visually, the status quo is a shift (see Figure 2). If a research compendium (code and related objects) is published alongside the research and (ideally) open data are used, the goal can be achieved.

Figure 2: Shifting from limited to fuller reproducibility (summarizing the main materials for a research, inspired by Bontemps and Orozco (2020))

How the reproducibility project helps you?

Making guidance on reproducibility available

The project thus helps make:

• Recommendations on how to approach reproducibility and structure your research compendium.
• Recommendations on structured ways to reference digital objects.
• Recommendations on how to cite others’ code and how to cite open datasets in your research.

Cataloguing all open datasets applicable to the discipline

The project also makes resources available, such as:

• The Price Statistics Open Data Catalogue.
• Guidance on how the Price Statistics Open Data Catalogue works.
• Guidance on how to register open dataset to the Price Statistics Data Catalogue.

Interested about the reproducibility?

We recommend other guidance related to the topic but not specific to price statistics (on whom we mostly base our discipline-specific guidance):

• The Turing Way is a wonderful and comprehensive guide for open research.
• The Reproducible Analytical Pipelines (or RAP) is another very applicable paradigm for reproducibility, although has come to be applied within official statistics as a way to operationalize production processes. Many UN classes are nowadays teaching these principles as a basic minimum standard within official statistics and have been recommended by the ONS for price statistics.²³

Footnotes

1. See Definitions in The The Turing Way for more details.

2. For instance see sessions by UN ESCAP, UN SIAP, or the University of Luxembourg as example classes.

3. See Price and Marques (2023) for an overview of RAP to price statistics.