State of the Fuzz : An Analysis of Black-Box Vulnerability Testing

Black-box vulnerability testing is a favored behavioral and functionality testing for finding vulnerabilities when no internal information regarding the system is available to the tester. Traditional blackbox fuzzers are oblivious to the state changes generated by random and unexpected inputs which makes them not only inefficient, but inept in testing stateful applications. On the other hand, modern automated scanners have shifted towards guided input generation using state-aware testing which aims to create input samples more efficiently and estimate the state changes by utilizing the outputs as a feedback mechanism. Similarly, differential black-box testing techniques have taken state-aware approaches and evolutionary input generation into account to limit the number of generated inputs and increase the code-coverage. This paper surveys three different black-box testing techniques, aiming to present an organized overview of the approaches which systematically improve automated black-box testing and differential fuzzing. It first provides an overview of required preliminaries and terminologies which are used throughout the paper. Next, it highlights the challenges of each technique, the problems they are aiming to solve as well as the proposed solutions and their evaluation. Finally a discussion of current issues, limitations, and a summary of future research direction are discussed.