When Simple is Better than Complex: Coverage and Mutation for DL Testing

Speaker: JInhan Kim, Università della Svizzera italiana

Abstract: In the context of traditional software testing, coverage testing has served as a foundational step for mutation testing, implying that the latter builds upon the insights gained from coverage analysis. This trade-off is well-known: coverage testing offers a rapid and cost-effective assessment, whereas mutation testing, despite its higher cost, excels in quantifying the effectiveness of test sets. Interestingly, this paradigm appears to extend to the testing of deep learning systems, where coverage techniques provide a straightforward yet efficient evaluation, and mutation testing, though costly, delivers more comprehensive insights. Despite the plethora of proposed coverage criteria and mutation methods, a comparative analysis between these approaches remains unexplored. This raises the question: Given that mutation testing has traditionally been regarded as a more robust testing methodology compared to coverage testing, can it maintain its superiority in the realm of deep learning? In this talk, I will introduce and discuss several coverage and mutation methods with the aim of empirically exploring and addressing these questions.

Biography: Jinhan Kim is currently a postdoc at USI, working with Prof. Paolo Tonella. He completed his Ph.D. degree from KAIST under the supervision of Dr. Shin Yoo, where he focused on software engineering research, particularly on mutation testing, fault localization, and deep learning system testing.

Chair: Francesco Bresciani

*************************
In February 2019, the Software Institute started its SI Seminar Series. Every Thursday afternoon, a researcher of the Institute will publicly give a short talk on a software engineering argument of their choice. Examples include, but are not limited to novel interesting papers, seminal papers, personal research overview, discussion of preliminary research ideas, tutorials, and small experiments.
On our YouTube playlist you can watch some of the past seminars. On the SI website you can find more details on the next seminar, the upcoming seminars, and an archive of the past speakers.