What is fuzz testing?
The search for bugs is usually done in an orderly manner. Fuzz testing, on the other hand, is characterized by adding random records. In this way, errors can be discovered that conventional test mechanisms remain hidden.
In IT, a genesis myth is not necessarily one of the basic requirements for well-implemented methods. Nevertheless, there is a very clear history of the origin of what is known as fuzz testing.
In the late 1980s, Barton Miller, a computer science professor in Madison, Wisconsin, used a landline connection to work from home at his Unix terminal at the university. During a thunderstorm, the signals did not come through the line as expected and the software received quasi random inputs between thunder and lightning. Even robust programs collapsed from the random inputs, to Miller’s greatest surprise.
Miller and his students distilled this experience into a test procedure known as fuzzing or fuzz testing. The stability of a program was and is tested via random, unstructured data input if the user input leaves certain schemes. The process is still popular in software development today, but it is not free from difficulties.
The benefits of fuzz testing
If programs are only tested along their limits, security gaps can persist. An error caused by unexpected input is a potential security hole that could be exploited at any time.
In fact, hackers and cybercriminals use fuzz testing and similar methods more often to find vulnerabilities in systems. Accordingly, software developers should also integrate fuzzing into their test cycle in order to find weak points in their programs even under stress.
In practice, fuzz testing has proven to be particularly suitable for finding more serious security gaps. The fuzzing method delivers the best results in conjunction with black box testing or as part of a beta test. Precisely because cyber criminals often use similar methods in practice, programmers should anticipate them through intensive fuzz testing. However, a major disadvantage of fuzzing is the cost.
In principle, fuzz testing – as with Miller’s thunderstorm – is also possible through the use of completely randomly generated inputs. However, the most likely weaknesses of a system can hardly be identified in this way. There tends to be an infinite number of random entries, but in order to condense them to a limited test period, fuzz testing requires a little more manual skill on the part of the developer .
This is how fuzz testing works in practice
The practical application of fuzzing follows four easy-to-repeat steps.
- First a correct input file is prepared and opened.
- In the second step, any part of the file is exchanged for random data and …
- … then opened by the program.
- Finally, developers need to record which part of the system has given way.
This last part can be quite laborious, in many cases it is the easiest way to record passes and fails. Parts of the fuzz test can be written manually in Perl or AppleScript, for example.
Different variants of fuzzers
Programs that enable automatic fuzzing are called fuzzers. These are typically based on three different methodologies:
- Mutation-based fuzzers: Here, correct data sets are changed in order to generate new test data. Individual files or bits are adapted and changed and automated logs of randomly generated errors are created.
- Generation-based fuzzers: This model does not fall back on correct data sets, but instead creates inputs generated based on the input specifications. Each input is completely rewritten by the fuzzer.
- Protocol-based fuzzers: The most efficient fuzzers are found in protocol-based fuzzers. The prerequisite for functionality in this case is that programmers * can use syntax, grammar or robustness tests as the basis for model-based test generation. Protocol-based fuzzers require deeper knowledge of the protocol format to be tested and require sophisticated specifications.
Various fuzzer tools are available for programmers, such as Peach Fuzzer, the Webscarab written in Java , Spike Proxy or OWASP WSFuzzer or the open source project American Fuzzy Loop.
Fuzz testing as part of the test cycle
Fuzz testing has proven to be a particularly efficient addition to security tests to identify memory leaks and assertion failures or to check for incorrect entries. In C and C ++ in particular, memory management is a system-critical point due to the manual control over memory, and it deserves appropriate attention in security tests.
Despite the required expertise, the time and financial expenditure, dealing with fuzz testing is definitely advisable, but should only ever be understood as part of a comprehensive security check. Incidentally, the name is derived from the English term “fuzzy”, which translates as “unclear, blurred” and already gives a clear indication of the methodology of fuzz testing.
