Can ChatGPT advance software testing intelligence? An experience report on metamorphic testing
CoRR(2023)
摘要
While ChatGPT is a well-known artificial intelligence chatbot being used to
answer human's questions, one may want to discover its potential in advancing
software testing. We examine the capability of ChatGPT in advancing the
intelligence of software testing through a case study on metamorphic testing
(MT), a state-of-the-art software testing technique. We ask ChatGPT to generate
candidates of metamorphic relations (MRs), which are basically necessary
properties of the object program and which traditionally require human
intelligence to identify. These MR candidates are then evaluated in terms of
correctness by domain experts. We show that ChatGPT can be used to generate new
correct MRs to test several software systems. Having said that, the majority of
MR candidates are either defined vaguely or incorrect, especially for systems
that have never been tested with MT. ChatGPT can be used to advance software
testing intelligence by proposing MR candidates that can be later adopted for
implementing tests; but human intelligence should still inevitably be involved
to justify and rectify their correctness.
更多查看译文
关键词
metamorphic testing,advance software testing intelligence,chatgpt
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要