June 22nd, 2024

Coverage at a Crossroads

Coverage.py is evolving to reduce execution-time overhead by adopting SlipCover's low-overhead approach for code coverage. Python 3.12's sys.monitoring improves line coverage, but challenges remain for branch coverage. SlipCover's method shows promise, requiring adjustments for optimal results.

Read original articleLink Icon
Coverage at a Crossroads

Coverage.py is undergoing changes to reduce execution-time overhead by leveraging new Python features. The current method involves a trace function for each line of Python execution, leading to inefficiencies for statement coverage. SlipCover offers a low-overhead alternative by directly tracking code execution without a trace function. Python 3.12 introduced sys.monitoring for event-based line execution tracking, reducing overhead for line coverage but posing challenges for branch coverage due to event handling limitations. SlipCover's approach of using line events for branch coverage presents a promising solution, although it requires adjustments in data collection and analysis. Coverage.py is exploring ways to adapt SlipCover's techniques while addressing issues like multi-arc branches and static code analysis complexities. Collaboration and feedback are sought to enhance coverage.py's performance and efficiency.

Link Icon 1 comments
By @ofek - 5 months
Very interesting; I'm quite excited for these future performance improvements! In small-medium sized projects the time isn't very noticeable but larger projects definitely are noticeable in CI.

edit: I should say for larger test suites rather than the project itself, but larger projects do tend to have larger test suites.