Home   >   CSC-OpenAccess Library   >    Manuscript Information
A Study on Using Code Coverage Information Extracted from Binary to Guide Fuzzing
Baoying Lou, Jia Song
Pages - 200 - 209     |    Revised - 30-11-2020     |    Published - 31-12-2020
Volume - 14   Issue - 5    |    Publication Date - December 2020  Table of Contents
MORE INFORMATION
KEYWORDS
Code Coverage, Fuzzing, Software Testing, Binary Analysis, Test Case Generation.
ABSTRACT
Code coverage is commonly used in software testing because it tells which portion of code has been tested or not. Fuzzing is one of the most popular and powerful solutions to find software vulnerabilities. And code coverage information is used in several fuzzing techniques to guide the testing. Coverage-guided fuzzer is efficient and effective by tracking and utilizing code coverage feedback. In practice, when the source code of a target application is not provided, we have to focus on the binary files and fuzz the executable files. This paper briefly introduces fuzzing techniques and the common code coverage measurement criteria. Then the paper give a comprehensive review and summary of the ways to gather coverage information, including source code instrumentation, dynamic instrumentation, static instrumentation, emulation, debugger, and hardware feature. Their advantages and disadvantages are discussed. Few studies have been conducted on the techniques that fuzzers extract code coverage information from binary files and use it to guide fuzzers in next step. Therefore this paper also provides a summary of how fuzzers utilize code coverage feedback information and what are the strengths and limitations of each of them.
A. Henderson, H. Yin, G. Jin, H. Han, Deng H. “VDF: Targeted Evolutionary Fuzz Testing of Virtual Devices”. In: Dacier M., Bailey M., Polychronakis M., Antonakakis M. (eds) Research in Attacks, Intrusions, and Defenses. RAID 2017. Lecture Notes in Computer Science, vol 10453. Springer, Cham, 2017.
A. Yeboah-Ofori, “Software Reliability and Quality Assurance Challenges in Cyber Physical Systems Security ”, International Journal of Computer Science and Security (IJCSS), Volume (14) : Issue (3) : 2020.
Abdullah Alashjaee*, Salahaldeen Duraibi*, Jia Song, “Dynamic Taint Analysis Tools: A Review”, International Journal of Computer Science and Security (IJCSS) Volume-13 Issue- 6, 2019.
C. Kaner, "Software negligence and testing coverage”. Software QA Quarterly, vol.2, #2, pp.18, 1995.
D. C. D’Elia, "SoK: Using Dynamic Binary Instrumentation for Security”. In ACM Asia Conference on Computer and Communications Security (AsiaCCS ’19), July 9–12, 2019, Auckland, New Zealand. ACM, New York, NY, USA, 2019.
D. C. D’Elia, E. Coppa, S. Nicchi, F. Palmaro, and L. Cavallaro, "SoK: Using Dynamic Binary Instrumentation for Security (And How You May Get Caught Red Handed). In ACM Asia Conference on Computer and Communications Security (AsiaCCS ’19), July 9–12, 2019.
E. Guler, C. Aschermann, Ali Abbasi, and Thorsten Holz. “AntiFuzz: Impeding Fuzzing Audits of Binary Executables”. In USENIX Security Symposium, 2019.
Emma, “EMMA: Frequently asked questions”. http://emma.sourceforge.net/faq.html, Jan, 1, 2006 [Dec 4, 2020].
G. J. Myers. "The Art of Software Testing," Second Edition. 2004.
H. A. Salem, J. Song, “A Review on Grammar-Based Fuzzing Techniques”, International Journal of Computer Science & Security (IJCSS), 2019.
H. Al Salem and J. Song, "A Review on Grammar-Based Fuzzing Techniques", International Journal of Computer Science & Security (IJCSS), Volume (13), Issue (3), 2019.
H. Peng and Y. Shoshitaishvili and M. Payer. "T-Fuzz: Fuzzing by Program Transformation", 2018 IEEE Symposium on Security and Privacy (SP), pages 697-710.
Honggfuzz, https://github.com/google/honggfuzz, Dec, 2, 2020 [Dec 4, 2020].
Intel Developer Zone, “Online Guide for the Intel Joule Module”. https://software.intel.com/en-us/node/721535 [Dec 4, 2020].
J. Salwan, "Dynamic Binary Analysis and Instrumentation Covering a function using a DSE approach", 2015.
Jun Li, Bodong Zhao, and Chao Zhang. 2018. “Fuzzing: a Survey”. Cybersecurity, 2018
lcamtuf's blog, “Binary fuzzing strategies: what works, what doesn't”. https://lcamtuf.blogspot.com/2014/08/binary-fuzzing-strategies-what-works.html [Dec 4, 2020].
M. A. Laurenzano, M. M. Tikir, L. Carrington and A. Snavely, "PEBIL: Efficient static binary instrumentation for Linux," 2010 IEEE International Symposium on Performance Analysis of Systems & Software (ISPASS), White Plains, NY, 2010.
M. Smithson, K. ElWazeer, K. Anand, A. Kotha, R. Barua, "Static Binary Rewriting without Supplemental Information”. WCRE, Koblenz, Germany, 2013.
M. Young, “The Technical Writer's Handbook”. Mill Valley, CA: University Science, 1989.
M. Zalewski, “American fuzzy lop”, https://lcamtuf.coredump.cx/afl/ [Dec 4, 2020].
Microsoft Security Engineering, https://www.microsoft.com/en-us/securityengineering/sdl [Dec 4, 2020].
Mohd. Ehmer Khan, “Different Software Testing Levels for Detecting Errors ”, International Journal of Software Engineering (IJSE), Volume (2) : Issue (4) : 2011.
N. Stephens and John Grosen and C. Salls and Andrew Dutcher and Ruoyu Wang and Jacopo Corbetta and Yan Shoshitaishvili and C. Krugel and G. Vigna. "Driller: Augmenting Fuzzing Through Selective Symbolic Execution", NDSS, 2016.
P. Chen and H. Chen. "Angora: Efficient Fuzzing by Principled Search" In IEEE Symposium on Security and Privacy (S&P), 2018.
P. S. Kochhar, F. Thung, D. Lo. "Code Coverage and Test Suite Effectiveness: Empirical Study with Real Bugs in Large Systems," In IEEE Symposium on Security and Privacy, 2015.
R. Freingruber, "The art of fuzzing", SEC Consult, 2017.
RAPITA Systems, “Code coverage without instrumentation”. https://www.rapitasystems.com/blog/code-coverage-without-instrumentation [Dec 4, 2020].
S. Gan, C. Zhang, X. Qin, X. Tu, K. Tu, K. Li, Z. Pei, Z. Chen, "CollAFL: Path Sensitive Fuzzing”. IEEE Symposium on Security and Privacy (SP), 2018.
S. Nagy and M. Hicks, "Full-Speed Fuzzing: Reducing Fuzzing Overhead through Coverage-Guided Tracing," 2019 IEEE Symposium on Security and Privacy (SP), San Francisco, CA, USA, 2019.
S. Rawat, V. Jain, A. Kumar, L. Cojocar, C. Giuffrida and H. Bos, "VUzzer: Application- aware Evolutionary Fuzzing" NDSS, Feb. 2017.
S. Schumilo, C. Aschermann, R. Gawlik, S. Schinzel, T. Holz, "kAFL: Hardware-Assisted Feedback Fuzzing for OS Kernels", 26th USENIX Security Symposium, 2017.
Salahaldeen Duraibi*, Abdullah Alashjaee*, Jia Song, “A Survey of Symbolic Execution Tools”, International Journal of Computer Science and Security (IJCSS) Volume-13 Issue-6, 2019.
Testing Brain, “Statement Coverage in software testing”. https://www.testingbrain.com/whitebox/statement-coverage.html [Dec 4, 2020].
V. Zhao, "Evaluation of Dynamic Binary Instrumentation Approaches: Dynamic Binary Translation vs. Dynamic Probe Injection “ Honors Thesis Collection. 575. 2018.
Wikipedia, Fuzzing, https://en.wikipedia.org/wiki/Fuzzing, Dec. 3, 2020 [Dec 4, 2020].
Y. Wang, X. Jia, Y. Liu, K. Zeng, T. Bao, D. Wu, and P. Su. “Not all coverage measurements are equal: Fuzzing by coverage accounting for input prioritization”. In NDSS’ 20, 2020.
“gamozolabs / mesos”. https://github.com/gamozolabs/mesos [Dec 4, 2020].
“QEMU documentation”. https://www.qemu.org/docs/master/ [Dec 4, 2020].
Miss Baoying Lou
Computer Science Department, University of Idaho, Moscow, ID, 83844 - United States of America
lou1407@vandals.uidaho.edu
Dr. Jia Song
Computer Science Department, University of Idaho, Moscow, ID, 83844 - United States of America