Reducing the Effort for Systematic Reviews in Software Engineering
release_zmu437hrebectmmb6v32gqyp7u
by
Francesco Osborne, Henry Muccini, Patricia Lago, Enrico Motta
2019
Abstract
Context. Systematic Reviews (SRs) are means for collecting and synthesizing
evidence from the identification and analysis of relevant studies from multiple
sources. To this aim, they use a well-defined methodology meant to mitigate the
risks of biases and ensure repeatability for later updates. SRs, however,
involve significant effort. Goal. The goal of this paper is to introduce a
novel methodology that reduces the amount of manual tedious tasks involved in
SRs while taking advantage of the value provided by human expertise. Method.
Starting from current methodologies for SRs, we replaced the steps of
keywording and data extraction with an automatic methodology for generating a
domain ontology and classifying the primary studies. This methodology has been
applied in the Software Engineering sub-area of Software Architecture and
evaluated by human annotators. Results. The result is a novel Expert-Driven
Automatic Methodology, EDAM, for assisting researchers in performing SRs. EDAM
combines ontology-learning techniques and semantic technologies with the
human-in-the-loop. The first (thanks to automation) fosters scalability,
objectivity, reproducibility and granularity of the studies; the second allows
tailoring to the specific focus of the study at hand and knowledge reuse from
domain experts. We evaluated EDAM on the field of Software Architecture against
six senior researchers. As a result, we found that the performance of the
senior researchers in classifying papers was not statistically significantly
different from EDAM. Conclusions. Thanks to automation of the less-creative
steps in SRs, our methodology allows researchers to skip the tedious tasks of
keywording and manually classifying primary studies, thus freeing effort for
the analysis and the discussion.
In text/plain
format
Archived Files and Locations
application/pdf 3.7 MB
file_pr3co6bdrjff3eudzd3snk2nym
|
arxiv.org (repository) web.archive.org (webarchive) |
1908.06676v1
access all versions, variants, and formats of this works (eg, pre-prints)