The Relevance of Classic Fuzz Testing ⁉️
A paper (here) by Barton P. Miller, Senior Member, IEEE, Mengxiao Zhang, and Elisa R. Heymann.
The paper talks about if the classic, basic fuzz technique is still useful and applicable in the face of the incredible progress in fuzz testing techniques and tools.
"As a side note, we tested the limited number of utilities available in a modern programming language (Rust) and found them to be of no better reliability than the standard ones". Apparently, most programmers still make many minor mistakes in programming, which is not entirely unavoidable given the size of projects developed today, and using a modern memory-safe language like Rust can not solve all the problems, but then again, keeping some pain points in mind can make your program robust.
In section 4.2 of the PDF file, the examples provided highlight various issues uncovered through fuzz testing, including:
Buffer overflows
Memory corruption
Pointer misuse
Array errors
Failure to check return values
Careless error handling
Complex state in loops
Errors related to line-editing and history functionality
These examples showcase the diverse range of vulnerabilities and bugs that can be identified through fuzz testing, emphasizing the importance of this technique in software testing and security assessment. For more detailed insights into each issue, you may refer to the PDF document (linked at start).
Such a case was also seen when steam rm -rfd a user's system github thread here. Such issues can be avoided with fuzzing to test out unexpected scenarios in your program's runtime behaviour.
I'd like to end by highlighting LLMs can also take good care of handling unexpected scenarios, but you'd need something specifically trained for this. ChatGPT, atleast the free one, is not that good at writing robust code, in this context.
