Framework to enhance natural language requirements for automated test case generation
Loading...
Date
2021
Authors
Journal Title
Journal ISSN
Volume Title
Publisher
Universiti Teknologi Malaysia
Abstract
Software testing plays an essential role in ensuring the conformance of the software system to the specified requirements. However, software testing is a challenging, labor-intensive, and time-consuming process that takes 40-70% of development time, effort, and cost. Test engineers need to identify all test execution scenarios from requirements and determine the runtime conditions that trigger these scenarios. The given requirement specifications are provided in natural language (NL) that tends to be ambiguous, inconsistent, and incomplete. As a consequence of these problems, 56% of errors during the software testing phase are traced back to requirements. Therefore, the aim of this study was to propose a framework to Enhance the Natural language requirements for automated Test Case Generation (ENTCG). The ENTCG framework is comprised of three models for the enhancement of NL requirements and one model for the test case generation from enhanced natural language requirements (ENLR). The proposed framework employed NL Processing to build Use Case Test Models and captured control flow from ENLR. The ENTCG framework supported the generation of system-level test cases from ENLR to reduce manual effort and ensure the requirement coverage. User stories and test scenarios expressed in NL acted as the input for test case generation. The first step of using this framework was the preprocessing of the NLR text. After preprocessing, the first model is proposed to detect the NL's terminological ambiguity whilst the second and third models were developed to detect the NL requirements' inconsistencies and incompleteness. A mechanism was proposed to map the requirements frame to a formal representation using semantic role labeling. The ENTCG framework was compared with existing related studies through experiments using the selected case study datasets. The results obtained from the feature extraction from the NLR phase's enhancement indicate that the ENTCG framework outperformed comparably with related works regarding precision, recall, and F-Measure. Semantic role labeling was utilized to improve the extraction technique's performance, the concept, and the process of effective test case generation from NLR analysis. The ENTCG framework was evaluated using a group of 326 NL requirements and 625 NL requirements from the PROMISE dataset. The comparison with existing studies shows that the proposed framework achieves precision and recall of 74.2% and 77.1%, respectively. The corresponding F1-score was 75.1%. The logical inconsistencies detection model was compared with the relevant model. The ambiguity detection model incompleteness and logical detection models outperform the existing models when it comes down to finding suitable diagram pairs for the use cases. Likewise, results show that the test case generation model could process all the case studies correctly and systematically to generate 30 test suits and 389 test cases. The ENTCG framework enables the analyst to detect ambiguity and incompleteness from fundamental requirements efficiently. The ENTCG framework also helps software testers efficiently design test cases from the need to improve software development quality and make the right decision at an early stage of development.
Description
Thesis (Ph.D (Computer Science))
Keywords
Natural language processing, Computational linguistics