Evaluating software architecture evaluation methods: An internal replication

Abstract:

Context: The size and complexity of software systems along with the demand for ensuring quality requirements have fostered the interest in software architecture evaluation methods. Although several empirical studies have been reported, the actual body of knowledge is still insufficient. To address this concern, we presented a family of four controlled experiments that compares a recently proposed method, the Quality-Driven Architecture Derivation and Improvement (QuaDAI) method against the well-known Architecture Tradeoff Analysis Method (ATAM). Objective: To provide further evidence on the efficiency, effectiveness, and perceived satisfaction of participants using these two software architecture evaluation methods. We report the results of a differentiated internal replication study. Method: The same materials used in the baseline experiments were employed in this replication but the participants were sixteen practitioners. In addition, we used a simpler design to reduce the treatments' application sequences. Results: The participants obtained architectures with better quality when applying QuaDAI, and they found this method to be more useful and likely to be used than ATAM, but no difference in terms of efficiency and perceived ease of use were found. Conclusions: The results are in line with the baseline experiments and support the hypothesis that QuaDAI achieve better results than ATAM when performing architectural evaluations; however, further work is need to improve the methods usability.

Año de publicación:

2017

Keywords:

• Experiment replication
• Software architecture evaluation

Fuente:

scopus