CVSS system criticized for failure to address real-world impact

JFrog argues vulnerability risk metrics need complete revamp

ANALYSIS Weaknesses in the existing CVSS scoring system have been highlighted through new research, with existing metrics deemed responsible for “overhyping” some vulnerabilities.

So-called “overinflated” ratings are potentially eating up the limited time of cybersecurity teams who may then not be focused on the bugs most likely to impact their organizations in favor of issues deemed critical across the board.

Catch up with the latest security vulnerability research and analysis

An analysis, put together by security tools vendor JFrog, involved accessing the popular Common Vulnerability Scoring System (CVSS), an open industry standard framework for assessing the severity of security problems. The system is managed by the non-profit Forum of Incident Response and Security Teams (FIRST) with the National Vulnerability Database (NVD) providing CVSS scores for confirmed vulnerabilities.

JFrog’s analysis, which focused on accessing the impact of security bugs in open source software, concluded that public CVSS impact metrics may be oversimplifying the risk posed by vulnerabilities because it lacks context, among other factors.

Critical assessment

According to the report (PDF), “Analysis of Open Source Security Vulnerabilities Most Impactful to DevOps and DevSecOps Teams”, there is a “discrepancy” between public severity ratings and the internal JFrog assessments of the top 50 CVEs of 2022.

JFrog’s security researchers say that in ‘most’ cases, the company’s own CVE (Common Vulnerabilities and Exposures) severity assessment is lower than the rating assigned in the NVD – “meaning oftentimes these vulnerabilities are being overhyped”.

For example, a buffer overrun in X.509 certificate verification, CVE-2022-3602 (CVSS 7.5), was of grave concern – until the release of the exploit’s technical details, which showed there was a marginal real-world impact, according to the researchers.

In total, 64% of the top 50 CVEs were given a lower JFrog severity rating, whilst 90% received a lower or equal rating.

Context dependant

JFrog says that many NVD security ratings were “undeserved” as they were not as simple to exploit as reported. Furthermore, many of the analyzed vulnerabilities required complex configuration environments or particular conditions for a successful attack.

Another criticism made by the cybersecurity firm is that there may be a lack of context when assigning CVE attack complexity metrics. For example, considering how potentially vulnerable software is deployed, the network environment, how the software is used, or whether a vulnerable API could end up parsing untrusted data should all be analyzed. The result is that severity ratings might either be set either too high or too low.

Misdirecting priorities risk

JFrog also observed that 10 of the most prevalent vulnerabilities in 2022 impacting the enterprise tended to have low severity ratings and so are either regarded as a lower priority for enterprise IT teams and open source project maintainers – so remediation work is either delayed or (worse) entirely disregarded.

If a bug is considered too small to bother with, developers may not create a patch, which JFrog says can only increase the number of affected systems over time. In contrast, if a CVSS rating is high but the real-world impact is considered minuscule, the threat level could be faulted as misleading.

Speaking to The Daily Swig, Shachar Menashe, senior director of security research at JFrog, said the best solution would be to update the CVSS standard to contain fields that would provide more context, such as exploitability in default configurations and whether or not there are context-dependent attack vectors.

Menashe added:

Sign up to Daily Swig Deserialized, our new fortnightly rundown of web security, bug bounty, and hacking culture news

Many cybersecurity experts acknowledge that the existing CVSS system is limited, and experience is what fills the gaps during vulnerability assessments. The quantitative research carried out by JFrog backs the “gut feeling” of many infosec professionals that the current vulnerability scoring system needs a revamp.

FIRST responder

Asked about JFrog’s critique, Chris Gibson, executive director of FIRST, said that in general, “scoring providers supply ‘reasonable worst case’ base scores, and rely on the consumers to mitigate (lower) the final score”.

Temporal threat information, asset criticality, compensating controls – such as firewall filters – and other environmental scores are “designed to lower the score to a more apt, applicable level, according to Gibson.

“Third parties, such as JFrog, can help consumers by providing threat intelligence (temporal score), allowing the use of the full CVSS score to better track patch priority and technical risk.”

Asked about potential improvements, Gibson said CVSS v4.0 is “coming soon” and will include a method for product developers to provide supplementary urgency ratings, leading to “a more accurate representation of the urgency of the vulnerability in their implementation, rather than relying on the OSS library provider’s worst-case scoring”.

“The CVSS system can be useful as long as its shortcomings are kept in mind. For instance, CVSS might score a vulnerability without considering important contextual factors like the environment in which it was found and the potential commercial or operational impact.”

Prashanth Samudrala, VP of product management at AutoRABIT, told The Daily Swig: “The system relies on currently available information, which means it could make decisions based on incomplete or incorrect information. While the CVSS system can be helpful, it should be used alongside other means of evaluation to get an accurate assessment.”

YOU MAY ALSO LIKE ‘Most web API flaws are missed by standard security tests’ – Corey J Ball on securing a neglected attack vector