Semi-proving: an integrated method based on global symbolic evaluation and metamorphic testing
ISSTA '02 Proceedings of the 2002 ACM SIGSOFT international symposium on Software testing and analysis
Metamorphic Testing of Programs on Partial Differential Equations: A Case Study
COMPSAC '02 Proceedings of the 26th International Computer Software and Applications Conference on Prolonging Software Life: Development and Redevelopment
COMPSAC '03 Proceedings of the 27th Annual International Conference on Computer Software and Applications
The Art of Software Testing
A Metamorphic Approach to Integration Testing of Context-Sensitive Middleware-Based Applications
QSIC '05 Proceedings of the Fifth International Conference on Quality Software
Towards a Metamorphic Testing Methodology for Service-Oriented Software Applications
QSIC '05 Proceedings of the Fifth International Conference on Quality Software
MuJava: a mutation system for java
Proceedings of the 28th international conference on Software engineering
Feature-driven requirement dependency analysis and high-level software design
Requirements Engineering
Self-Metamorphic-Testing Components
COMPSAC '06 Proceedings of the 30th Annual International Computer Software and Applications Conference - Volume 02
Automated analysis of feature models: challenges ahead
Communications of the ACM - Software product line
Generic semantics of feature diagrams
Computer Networks: The International Journal of Computer and Telecommunications Networking
Verifying feature models using OWL
Web Semantics: Science, Services and Agents on the World Wide Web
Automated error analysis for the agilization of feature modeling
Journal of Systems and Software
ICST '09 Proceedings of the 2009 International Conference on Software Testing Verification and Validation
On guiding the augmentation of an automated test suite via mutation analysis
Empirical Software Engineering
Automatic system testing of programs without test oracles
Proceedings of the eighteenth international symposium on Software testing and analysis
S.P.L.O.T.: software product lines online tools
Proceedings of the 24th ACM SIGPLAN conference companion on Object oriented programming systems languages and applications
Inferring information from feature diagrams to product line economic models
Proceedings of the 13th International Software Product Line Conference
SAT-based analysis of feature models is easy
Proceedings of the 13th International Software Product Line Conference
Generic feature-based software composition
SC'07 Proceedings of the 6th international conference on Software composition
Automated analysis of feature models 20 years later: A literature review
Information Systems
Automated diagnosis of feature model configurations
Journal of Systems and Software
Automated Test Data Generation on the Analyses of Feature Models: A Metamorphic Testing Approach
ICST '10 Proceedings of the 2010 Third International Conference on Software Testing, Verification and Validation
Feature model based on description logics
KES'06 Proceedings of the 10th international conference on Knowledge-Based Intelligent Information and Engineering Systems - Volume Part II
Automated reasoning on feature models
CAiSE'05 Proceedings of the 17th international conference on Advanced Information Systems Engineering
Feature models, grammars, and propositional formulas
SPLC'05 Proceedings of the 9th international conference on Software Product Lines
BeTTy: benchmarking and testing on the automated analysis of feature models
Proceedings of the Sixth International Workshop on Variability Modeling of Software-Intensive Systems
Using regression testing to analyze the impact of changes to variability models on products
Proceedings of the 16th International Software Product Line Conference - Volume 1
Strategies for testing products in software product lines
ACM SIGSOFT Software Engineering Notes
Reverse engineering feature models with evolutionary algorithms: an exploratory study
SSBSE'12 Proceedings of the 4th international conference on Search Based Software Engineering
A formal framework for software product lines
Information and Software Technology
Hi-index | 0.00 |
Context: A feature model (FM) represents the valid combinations of features in a domain. The automated extraction of information from FMs is a complex task that involves numerous analysis operations, techniques and tools. Current testing methods in this context are manual and rely on the ability of the tester to decide whether the output of an analysis is correct. However, this is acknowledged to be time-consuming, error-prone and in most cases infeasible due to the combinatorial complexity of the analyses, this is known as the oracle problem. Objective: In this paper, we propose using metamorphic testing to automate the generation of test data for feature model analysis tools overcoming the oracle problem. An automated test data generator is presented and evaluated to show the feasibility of our approach. Method: We present a set of relations (so-called metamorphic relations) between input FMs and the set of products they represent. Based on these relations and given a FM and its known set of products, a set of neighbouring FMs together with their corresponding set of products are automatically generated and used for testing multiple analyses. Complex FMs representing millions of products can be efficiently created by applying this process iteratively. Results: Our evaluation results using mutation testing and real faults reveal that most faults can be automatically detected within a few seconds. Two defects were found in FaMa and another two in SPLOT, two real tools for the automated analysis of feature models. Also, we show how our generator outperforms a related manual suite for the automated analysis of feature models and how this suite can be used to guide the automated generation of test cases obtaining important gains in efficiency. Conclusion: Our results show that the application of metamorphic testing in the domain of automated analysis of feature models is efficient and effective in detecting most faults in a few seconds without the need for a human oracle.