4.4 False Positives & Negatives

Let's now talk about the concept of false positives and false negatives, which are critical to understand in the context of smart contract audits or security.

False Positives

False positives are findings which flag the presence of vulnerabilities, but which in fact are not vulnerabilities. They could arise due to incorrect assumptions or simplifications in analysis which do not correctly consider all the factors required for the actual presence of vulnerabilities.

False positives require further manual analysis on findings to investigate if they are indeed false positives or if they are true positives. A high number of false positives increases the manual effort required in verification and also lowers the confidence in the accuracy of findings from the earlier automated analysis.

On the flip side, true positives might sometimes be incorrectly classified as false positives, which leads to the vulnerabilities behind those findings being ignored and left behind in the code instead of being fixed, and may end up getting exploited later.

False Negatives

On the other hand false negatives are missed findings that should have indicated the presence of vulnerabilities, but which are in fact not reported at all. Such false negatives again could be due to incorrect assumptions or inaccuracies in analysis which did not correctly consider the minimum factors required for the actual presence of vulnerabilities.

False negatives, per definition, are not reported or even realized unless a different analysis reveals their presence, or the vulnerabilities are realized when they're exploited. A high number of false negatives lowers the confidence in the effectiveness of the earlier manual or automated analysis. In contrast, true negatives are findings that are analyzed and dismissed which are in fact not vulnerabilities

So these concepts of true positives, false positives, true negatives and false negatives come up often in smart contract auditing and in security in general, and therefore this terminology (the distinction between these types) should be well understood.