Home   >   CSC-OpenAccess Library   >    Manuscript Information
Full Text Available

This is an Open Access publication published under CSC-OpenAccess Policy.
Publications from CSC-OpenAccess Library are being accessed from over 74 countries worldwide.
A Study on Using Code Coverage Information Extracted from Binary to Guide Fuzzing
Baoying Lou, Jia Song
Pages - 200 - 209     |    Revised - 30-11-2020     |    Published - 31-12-2020
Volume - 14   Issue - 5    |    Publication Date - December 2020  Table of Contents
Code Coverage, Fuzzing, Software Testing, Binary Analysis, Test Case Generation.
Code coverage is commonly used in software testing because it tells which portion of code has been tested or not. Fuzzing is one of the most popular and powerful solutions to find software vulnerabilities. And code coverage information is used in several fuzzing techniques to guide the testing. Coverage-guided fuzzer is efficient and effective by tracking and utilizing code coverage feedback. In practice, when the source code of a target application is not provided, we have to focus on the binary files and fuzz the executable files. This paper briefly introduces fuzzing techniques and the common code coverage measurement criteria. Then the paper give a comprehensive review and summary of the ways to gather coverage information, including source code instrumentation, dynamic instrumentation, static instrumentation, emulation, debugger, and hardware feature. Their advantages and disadvantages are discussed. Few studies have been conducted on the techniques that fuzzers extract code coverage information from binary files and use it to guide fuzzers in next step. Therefore this paper also provides a summary of how fuzzers utilize code coverage feedback information and what are the strengths and limitations of each of them.
1 P. S. Kochhar, F. Thung, D. Lo. "Code Coverage and Test Suite Effectiveness: Empirical Study with Real Bugs in Large Systems," In IEEE Symposium on Security and Privacy, 2015.
2 G. J. Myers. "The Art of Software Testing," Second Edition. 2004.
3 H. Al Salem and J. Song, "A Review on Grammar-Based Fuzzing Techniques", International Journal of Computer Science & Security (IJCSS), Volume (13), Issue (3), 2019.
4 Wikipedia, Fuzzing, https://en.wikipedia.org/wiki/Fuzzing, Dec. 3, 2020 [Dec 4, 2020].
5 R. Freingruber, "The art of fuzzing", SEC Consult, 2017.
6 Microsoft Security Engineering, https://www.microsoft.com/en-us/securityengineering/sdl [Dec 4, 2020].
7 E. Guler, C. Aschermann, Ali Abbasi, and Thorsten Holz. “AntiFuzz: Impeding Fuzzing Audits of Binary Executables”. In USENIX Security Symposium, 2019.
8 M. Zalewski, “American fuzzy lop”, https://lcamtuf.coredump.cx/afl/ [Dec 4, 2020].
9 Honggfuzz, https://github.com/google/honggfuzz, Dec, 2, 2020 [Dec 4, 2020].
10 S. Rawat, V. Jain, A. Kumar, L. Cojocar, C. Giuffrida and H. Bos, "VUzzer: Application- aware Evolutionary Fuzzing" NDSS, Feb. 2017.
11 N. Stephens and John Grosen and C. Salls and Andrew Dutcher and Ruoyu Wang and Jacopo Corbetta and Yan Shoshitaishvili and C. Krugel and G. Vigna. "Driller: Augmenting Fuzzing Through Selective Symbolic Execution", NDSS, 2016.
12 H. Peng and Y. Shoshitaishvili and M. Payer. "T-Fuzz: Fuzzing by Program Transformation", 2018 IEEE Symposium on Security and Privacy (SP), pages 697-710.
13 C. Kaner, "Software negligence and testing coverage”. Software QA Quarterly, vol.2, #2, pp.18, 1995.
14 Testing Brain, “Statement Coverage in software testing”. https://www.testingbrain.com/whitebox/statement-coverage.html [Dec 4, 2020].
15 Emma, “EMMA: Frequently asked questions”. http://emma.sourceforge.net/faq.html, Jan, 1, 2006 [Dec 4, 2020].
16 V. Zhao, "Evaluation of Dynamic Binary Instrumentation Approaches: Dynamic Binary Translation vs. Dynamic Probe Injection “ Honors Thesis Collection. 575. 2018.
17 M. Young, “The Technical Writer's Handbook”. Mill Valley, CA: University Science, 1989.
18 D. C. D’Elia, "SoK: Using Dynamic Binary Instrumentation for Security”. In ACM Asia Conference on Computer and Communications Security (AsiaCCS ’19), July 9–12, 2019, Auckland, New Zealand. ACM, New York, NY, USA, 2019.
19 J. Salwan, "Dynamic Binary Analysis and Instrumentation Covering a function using a DSE approach", 2015.
20 D. C. D’Elia, E. Coppa, S. Nicchi, F. Palmaro, and L. Cavallaro, "SoK: Using Dynamic Binary Instrumentation for Security (And How You May Get Caught Red Handed). In ACM Asia Conference on Computer and Communications Security (AsiaCCS ’19), July 9–12, 2019.
21 M. A. Laurenzano, M. M. Tikir, L. Carrington and A. Snavely, "PEBIL: Efficient static binary instrumentation for Linux," 2010 IEEE International Symposium on Performance Analysis of Systems & Software (ISPASS), White Plains, NY, 2010.
22 M. Smithson, K. ElWazeer, K. Anand, A. Kotha, R. Barua, "Static Binary Rewriting without Supplemental Information”. WCRE, Koblenz, Germany, 2013.
23 “QEMU documentation”. https://www.qemu.org/docs/master/ [Dec 4, 2020].
24 “gamozolabs / mesos”. https://github.com/gamozolabs/mesos [Dec 4, 2020].
25 Intel Developer Zone, “Online Guide for the Intel Joule Module”. https://software.intel.com/en-us/node/721535 [Dec 4, 2020].
26 RAPITA Systems, “Code coverage without instrumentation”. https://www.rapitasystems.com/blog/code-coverage-without-instrumentation [Dec 4, 2020].
27 S. Gan, C. Zhang, X. Qin, X. Tu, K. Tu, K. Li, Z. Pei, Z. Chen, "CollAFL: Path Sensitive Fuzzing”. IEEE Symposium on Security and Privacy (SP), 2018.
28 A. Henderson, H. Yin, G. Jin, H. Han, Deng H. “VDF: Targeted Evolutionary Fuzz Testing of Virtual Devices”. In: Dacier M., Bailey M., Polychronakis M., Antonakakis M. (eds) Research in Attacks, Intrusions, and Defenses. RAID 2017. Lecture Notes in Computer Science, vol 10453. Springer, Cham, 2017.
29 lcamtuf's blog, “Binary fuzzing strategies: what works, what doesn't”. https://lcamtuf.blogspot.com/2014/08/binary-fuzzing-strategies-what-works.html [Dec 4, 2020].
30 P. Chen and H. Chen. "Angora: Efficient Fuzzing by Principled Search" In IEEE Symposium on Security and Privacy (S&P), 2018.
31 S. Schumilo, C. Aschermann, R. Gawlik, S. Schinzel, T. Holz, "kAFL: Hardware-Assisted Feedback Fuzzing for OS Kernels", 26th USENIX Security Symposium, 2017.
32 H. A. Salem, J. Song, “A Review on Grammar-Based Fuzzing Techniques”, International Journal of Computer Science & Security (IJCSS), 2019.
33 A. Yeboah-Ofori, “Software Reliability and Quality Assurance Challenges in Cyber Physical Systems Security ”, International Journal of Computer Science and Security (IJCSS), Volume (14) : Issue (3) : 2020.
34 Mohd. Ehmer Khan, “Different Software Testing Levels for Detecting Errors ”, International Journal of Software Engineering (IJSE), Volume (2) : Issue (4) : 2011.
35 Jun Li, Bodong Zhao, and Chao Zhang. 2018. “Fuzzing: a Survey”. Cybersecurity, 2018
36 Y. Wang, X. Jia, Y. Liu, K. Zeng, T. Bao, D. Wu, and P. Su. “Not all coverage measurements are equal: Fuzzing by coverage accounting for input prioritization”. In NDSS’ 20, 2020.
37 S. Nagy and M. Hicks, "Full-Speed Fuzzing: Reducing Fuzzing Overhead through Coverage-Guided Tracing," 2019 IEEE Symposium on Security and Privacy (SP), San Francisco, CA, USA, 2019.
38 Salahaldeen Duraibi*, Abdullah Alashjaee*, Jia Song, “A Survey of Symbolic Execution Tools”, International Journal of Computer Science and Security (IJCSS) Volume-13 Issue-6, 2019.
39 Abdullah Alashjaee*, Salahaldeen Duraibi*, Jia Song, “Dynamic Taint Analysis Tools: A Review”, International Journal of Computer Science and Security (IJCSS) Volume-13 Issue- 6, 2019.
Miss Baoying Lou
Computer Science Department, University of Idaho, Moscow, ID, 83844 - United States of America
Dr. Jia Song
Computer Science Department, University of Idaho, Moscow, ID, 83844 - United States of America