Getting 100% code coverage doesn't eliminate bugs
Achieving 100% code coverage doesn't ensure bug-free software. A blog post illustrates this with a critical bug missed despite full coverage, leading to a rocket explosion. It suggests alternative approaches and a 20% coverage minimum.
Read original article
Achieving 100% code coverage does not guarantee bug-free software, as demonstrated by a one-line code example provided in a blog post. The post emphasizes that while some may believe 100% coverage is the ultimate goal for bug elimination, it is not a reliable indicator of software quality. The example highlights a scenario where despite achieving full code coverage and extensive testing, a critical bug causing a divide-by-zero error went unnoticed until a rocket launch resulted in an explosion. The author argues that focusing solely on increasing code coverage to reduce bugs is a misconception and suggests considering alternative approaches like formal verification and proof checkers for mission-critical software. Additionally, the post recommends aiming for a minimum code coverage of 20% based on the Pareto Principle, which suggests that a small portion of code is responsible for the majority of bugs. Junior developers and misguided team managers are mentioned as potential audiences who may benefit from understanding the limitations of code coverage as a metric for software quality.
Related
I've stopped using box plots (2021)
The author critiques box plots for being unintuitive and prone to misinterpretation, advocating for simpler alternatives like strip plots to effectively communicate distribution insights without confusing audiences.
My weekend project turned into a 3 years journey
Anthony's note-taking app journey spans 3 years, evolving from a secure Markdown tool to a complex Electron/React project with code execution capabilities. Facing challenges in store publishing, he prioritizes user feedback and simplicity, opting for a custom online deployment solution.
I found an 8 years old bug in Xorg
An 8-year-old Xorg bug related to epoll misuse was found by a picom developer. The bug caused windows to disappear during server lock, traced to CloseDownClient events. Despite limited impact, the developer seeks alternative window tree updates, emphasizing testing and debugging tools.
8 commentsJacoco measures both line and branch coverage, same as Jest.
Preferred thing to do is establish a baseline, and then
* Write tests for new features going forward.
* Write tests for fixes to production incidents.
Don't let things get worse than baseline. (The number will vary from repo to repo.)
And those bugs are easier to find than the ones that might still exist if you have 100% coverage.
No one ever anywhere (except for numerous strawmen) has claimed 100% coverage "will eliminate all bugs and produce the perfect software."
> This article is useless. Everybody knows that 100% code coverage doesn’t eliminate bugs.
A double-secret strawman is still a strawman, dude.
> Junior developers certainly don’t know this. Misguided team managers don’t know it as well.
Great. Do they read your clickbait? If so, do they believe it (on authority, I guess)? Because unless they do, you're certainly a problem and not part of a solution.
Like money doesn't give happiness... but it helps.
I doubt this is a common misunderstanding among any but the most junior of devs.
Related
I've stopped using box plots (2021)
The author critiques box plots for being unintuitive and prone to misinterpretation, advocating for simpler alternatives like strip plots to effectively communicate distribution insights without confusing audiences.
My weekend project turned into a 3 years journey
Anthony's note-taking app journey spans 3 years, evolving from a secure Markdown tool to a complex Electron/React project with code execution capabilities. Facing challenges in store publishing, he prioritizes user feedback and simplicity, opting for a custom online deployment solution.
I found an 8 years old bug in Xorg
An 8-year-old Xorg bug related to epoll misuse was found by a picom developer. The bug caused windows to disappear during server lock, traced to CloseDownClient events. Despite limited impact, the developer seeks alternative window tree updates, emphasizing testing and debugging tools.