Journals in Economic Sciences: Paying Lip Service to Reproducible Research?

Paying Lip Service to Reproducible Research?

Authors

  • Sven Vlaeminck ZBW – German National Library for Economics / Leibniz Information Centre for Economics in Hamburg/Germany
  • Felix Podkrajac Library and Information System of the Carl von Ossietzky University of Oldenburg

DOI:

https://doi.org/10.29173/iq6

Keywords:

reproducible research, economics, journals, data policies, data archives,

Abstract

The findings of numerous replication studies in economics have raised serious concerns regarding the credibility and reliability of published applied economic research. Literature suggests that economic research often is not replicable because (i) only a small proportion of journals in the field have implemented functional policies on the disclosure of employed datasets and program code, (ii) authors frequently do not comply with these data policies and (iii) editorial offices do not ensure that these policies are enforced. In this paper, we focus on the aspect last mentioned. We empirically evaluate 599 articles published in 37 journals with a data availability policy. We present the share of articles that fall under a data policy, because replication data is needed to verify the published results. Afterwards, we check the journal data archives and supplemental information section of each article for the availability of replication files. For a reduced sub-sample of 245 data-based articles, we check in depth whether the replication files we found are compliant with the requirements of the journal’s respective data policy. Thereby, we are able to determine how much journals in economic sciences enforce their data policies. Our findings suggest a mixed picture: While some journals achieve high compliance rates, a significant share of journals only sporadically provides replication files for data-based research papers. Overall, 47.5% of all articles analysed honour the data policy of the respective journal. Our findings also provide evidence that voluntary data policies are not effective in fostering replicable research.

Downloads

Published

2017-12-10 — Updated on 2022-09-28

Versions

How to Cite

Vlaeminck, S., & Podkrajac, F. (2022). Journals in Economic Sciences: Paying Lip Service to Reproducible Research? Paying Lip Service to Reproducible Research?. IASSIST Quarterly, 41(1-4), 16. https://doi.org/10.29173/iq6 (Original work published December 10, 2017)