Discovering Vulnerabilities In The Wild: An Empirical Study

Abstract

There is little or no information available on what actually happens when a software vulnerability is detected. We performed an empirical study on reporters of the three most prominent security vulnerabilities: buffer overflow, SQL injection, and cross site scripting vulnerabilities. The goal was to understand the methods and tools used during the discovery and whether the community of developers exploring one security vulnerability differs---in their approach---from another community of developers exploring a different vulnerability. The reporters were featured in the SecurityFocus repository for 12 month periods for each vulnerability. We collected 127 responses. We found that the communities differ based on the security vulnerability they target; but within a specific community, reporters follow similar approaches. We also found a serious problem in the vulnerability reporting process that is common for all communities. Most reporters, especially the experienced ones, favor full-disclosure and do not collaborate with the vendors of vulnerable software. They think that the public disclosure, sometimes supported by a detailed exploit, will put pressure on vendors to fix the vulnerabilities. But, in practice, the vulnerabilities not reported to vendors are less likely to be fixed. Ours is the first study on vulnerability repositories that targets the reporters of the most common security vulnerabilities, thus concentrating on the people involved in the process; previous works have overlooked this rich information source. The results are valuable for beginners exploring how to detect and report security vulnerabilities and for tool vendors and researchers exploring how to automate and fix the process.