July 16, 2018

by Rahul Pandita and Todd Erickson

Phase Change research scientist Dr. Rahul Pandita recently had two co-written papers published in well-known research journals. The first paper, “Are vulnerabilities discovered and resolved like other defects?,” was published in the June 2018 volume of the Empirical Software Engineering: An International Journal and presented as a Journal First Paper at the 40th International Conference on Software Engineering (ICSE) in Gothenburg, Sweden.

The paper was co-written with Patrick Morrison, Dr. Xusheng Xiao, Dr. Ram Chillarege, and Dr. Laurie Williams. Patrick Morrison is a Ph.D. candidate in the Computer Science Department at North Carolina State University. Dr. Xusheng Xiao is an assistant professor in the Department of Electrical Engineering and Computer Science at Case Western University.

Dr. Ram Chillarege is the founder and president of Chillarege Inc. Dr. Laurie Williams is a professor, and the department head, at the North Carolina State University Department of Computer Science.

The paper

The goal of the project’s research was to determine if security defects (referred to as vulnerabilities in the paper) are discovered and resolved by different software-development practices in comparison to non-security defects. If true, technical leaders could use the distinction to drive security-specific software development process improvements.

The research consisted of extending Orthogonal Defect Classification (ODC), which is a well-established scheme for classifying software defects, to study process-related differences between vulnerabilities and non-security defects, and thereby creating ODC + Vulnerabilities (ODC+V). This new classification was applied to 583 vulnerabilities and 583 defects across 133 releases of three open-source projects – the Firefox web browser, phpMyAdmin, and Google’s Chrome web browser.

The study found that compared with non-security defects, vulnerabilities are found much later in the development cycle and are more likely to be resolved through changes to conditional logic. The results indicate opportunities may exist for more efficient vulnerability detection and resolution.

The paper was accepted by the 40th International Conference on Software Engineering (ICSE) that was held in Gothenburg Sweden, between May 27 and June 3, as part of the *ICSE 2018* Journal First Papers track. Dr. Williams presented it on May 31, 2018.

But wait, there’s more

The second paper, “Mapping the field of software life cycle security measures,” is scheduled to be published in the October 2018 issue of Information and Software Technology. It was co-written with Patrick Morrison, Dr. Laurie Williams, and David Moye, a program site lead with Aelius Exploration Technologies LLC.

The authors suspected that a catalog of software-development life cycle security metrics could assist practitioners in choosing appropriate metrics, and researchers in identifying opportunities for security measurement refinement.

They conducted a systematic mapping study, beginning with 4,818 papers and focusing on 71 papers reporting on 324 unique security metrics. For each metric, the researchers identified the subject being measured, how the metric had been validated, and how the metric was used. Then they categorized the metrics and included examples of metrics for each category.

The research found that approximately 85% of the security metrics studied were proposed and evaluated solely by their authors, leaving room for replication and confirmation through field studies. Approximately 60% of the metrics were empirically evaluated by their authors or others.

They concluded that the primary application of security metrics to the software development lifecycle is studying the relationship between properties of source code and reported vulnerabilities. This suggests that researchers need to refine vulnerability measurements and give greater attention to metrics for the requirement, design, and testing phases of development.

Rahul Pandita is a senior research scientist at Phase Change. He earned his Ph.D. in computer science from North Carolina State University. You can reach him at [email protected].

Todd Erickson is a tech writer at Phase Change. You can reach him at [email protected].