Wang, XiaoyinTanofsky, Philip2024-03-082024-03-0820149781303921056https://hdl.handle.net/20.500.12588/5704This item is available only to currently enrolled UTSA students, faculty or staff. To download, navigate to Log In in the top right-hand corner of this screen, then select Log in with my UTSA ID.Software testing performs a vital function in the software development lifecycle of quality assurance. Due to the time-intensive and often prohibitive costs of software testing, research efforts attempt to identify and improve approaches to automated test-case generation. Research efforts for these automated techniques provide evaluations of effectiveness and performance of the generated test cases, but rarely compare the effectiveness against test suites used in popular, real-world projects, which are developed in part by professional software developers and testers. Without the direct comparison of the two types of test suites, the strengths and weaknesses of the automatically-generated test cases are difficult to determine, which makes the added benefit or potential replacement value of the automatically-generated test cases hard to establish. Our research provides an empirical study of automatically-generated test suites and the test suites in practice by comparing their respective test sufficiency and test efficiency. The study provides insight into the similarities and differences between the two types of test suites. We begin by locating test cases in practice which are test cases included with the real-world project and developed by software developers and testers. Then, the second test suite is built from test cases created by a mature off-the-shelf, automated test-generation tool using dynamic symbolic execution (i.e., KLEE). The results of the study show the automatically-generated test suites typically outperform the code coverage of the test suites in practice but achieve poorer results in killing mutants and solving hard-testing problems. The automatically-generated test suites exhibit stronger data validation capabilities, while the test suites in practice present better focus on meaningful paths of code execution.82 pagesapplication/pdfGNU CoreutilsKLEEsoftware engineeringsoftware testingsoftware verificationComputer scienceComputer software -- VerificationComputer software -- DevelopmentThesis