In the world of software development, one of the most critical moment is the decision made before release. Those, who responsible for the release of a new version must determine whether the software is ready to be delivered to users or additional work is needed. Many factors infuelnce this decision, but one of the most important metric is test coverage. Why do so few companies rely on this metric, and does test coverage determine that a software is ready to go to market?
The role of code coverage in the Go/No-Go decision
As we have discussed in previous articles, code coverage plays a crucial role in the Go/No-Go decision. It is a key metric that measures how much of the software's source code has been tested. Typically expressed as a percentage, the higher the number, the more of the code has been covered by tests. When measuring test coverage, various dimensions can be considered such as line, branch, or function coverage.
High test coverage indicates that the tests have thoroughly examined the source code's functionality. This also suggests that the risks of hidden bugs and unexpected behavior have been minimized by testers and developers. On the other side, low code coverage suggests that significant parts of the code have not been tested. This carries significant risk, especially in the case of critical systems or major changes, such as the introduction of a new feature.
During the Go/No-Go decision process, insights into code coverage enable development teams to objectively assess the current state of the software. If necessary, further testing can be carried out to reduce risks. Therefore, test coverage is not just a technical metric but a strategic tool that facilitates the release of high-quality software, while protecting the company's reputation and minimizing financial losses.
How TestNavigator helps?
With TestNavigator, you can easily see how well a version has been tested before release and understand the risks of sending it to the customer. Releasing an inadequately or poorly tested update can lead to enormous damages, which can be measured in significant financial losses.
In July 2024, the CordStrike company released an insufficiently tested update, made computers running the software stop working. Estimates suggest that the bug made 8.5 million computers unusable, causing over $10 billion in financial damage to customers. For example, this incident could have been avoided with TestNavigator. Before the release, the company’s decision-makers would have received clear percentage-based data about the system's test coverage and the untested parts of the code.
Pre-release testing with TestNavigator
In the TestNavigator system, when creating a test cycle, you can set "Exit criteria" for full coverage, change coverage, and the number of test cases run. It's important to note that you can set separate "Exit Criteria" for the test coverage of the whole source code and for the newly changed parts of the code. It is known, that the biggest risks usually come from the sections that have been updated since the last version.
Once the "Exit Criteria" have been defined, the testing process begins. Upon completion, the system assigns a "GO" or "No-GO" status to the version intended for release. If the system indicates a "No-GO" status, additional test cases must be run to ensure the version meets the desired level of test coverage, thereby ensuring a safe release.
TestNavigator is the solution
TestNavigator is not just a tool for supporting testing. It is a critical component in the release process, helping to minimize risks and ensuring that versions are only released if they meet strict testing criteria.
The Go/No-Go status offers a clear and transparent decision point, enabling companies to deliver reliable and stable software to customers. In addition it is avoiding the financial losses and damage to customer trust that can result from severe software errors.