Journals in Economic Sciences: Paying Lip Service to Reproducible Research?
Paying Lip Service to Reproducible Research?
Keywords:reproducible research, economics, journals, data policies, data archives,
The findings of numerous replication studies in economics have raised serious concerns regarding the credibility and reliability of published applied economic research. Literature suggests that economic research often is not replicable because (i) only a small proportion of journals in the field have implemented functional policies on the disclosure of employed datasets and program code, (ii) authors frequently do not comply with these data policies and (iii) editorial offices do not ensure that these policies are enforced. In this paper, we focus on the aspect last mentioned. We empirically evaluate 599 articles published in 37 journals with a data availability policy. We present the share of articles that fall under a data policy, because replication data is needed to verify the published results. Afterwards, we check the journal data archives and supplemental information section of each article for the availability of replication files. For a reduced sub-sample of 245 data-based articles, we check in depth whether the replication files we found are compliant with the requirements of the journal’s respective data policy. Thereby, we are able to determine how much journals in economic sciences enforce their data policies. Our findings suggest a mixed picture: While some journals achieve high compliance rates, a significant share of journals only sporadically provides replication files for data-based research papers. Overall, 47.5% of all articles analysed honour the data policy of the respective journal. Our findings also provide evidence that voluntary data policies are not effective in fostering replicable research.
- 2022-09-28 (2)
- 2017-12-10 (1)
How to Cite
Copyright (c) 2017 Sven Vlaeminck, Felix Podkrajac
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
This license lets others remix, tweak, and build upon your work non-commercially, and although their new works must also acknowledge you and be non-commercial, they don’t have to license their derivative works on the same terms.
The Creative Commons-Attribution-Noncommercial License 4.0 International applies to all works published by IASSIST Quarterly. Authors will retain copyright of the work. Your contribution will be available at the IASSIST Quarterly website when announced on the IASSIST list server.