Provenance metadata for statistical data: An introduction to Structured Data Transformation Language (SDTL)
Keywords:metadata, provenance, statistical data
Structured Data Transformation Language (SDTL) provides structured, machine actionable representations of data transformation commands found in statistical analysis software. The Continuous Capture of Metadata for Statistical Data Project (C2Metadata) created SDTL as part of an automated system that captures provenance metadata from data transformation scripts and adds variable derivations to standard metadata files. SDTL also has potential for auditing scripts and for translating scripts between languages. SDTL is expressed in a set of JSON schemas, which are machine actionable and easily serialized to other formats. Statistical software languages have a number of special features that have been carried into SDTL. We explain how SDTL handles differences among statistical languages and complex operations, such as merging files and reshaping data tables from “wide” to “long”.
How to Cite
Copyright (c) 2020 George Alter, Darrell Donakowski, Jack Gager, Pascal Heus, Carson Hunter, Sanda Ionescu, Jeremy Iverson, H.V. Jagadish, Carl Lagoze, Jared Lyle, Alexander Mueller, Sigbjørn Revheim, Matthew A. Richardson, Risnes Ørnulf, Karunakara Seelam, Dan Smith, Tom Smith, Jie Song, Yashas Jaydeep Vaidya, Ole Voldsater
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
"This license lets others remix, tweak, and build upon your work non-commercially, and although their new works must also acknowledge you and be non-commercial, they don’t have to license their derivative works on the same terms."