Coverage in Hardware verification

In this article, we will learn one of the crucial parameters of verification which helps us identify the completeness of verification. In previous articles, we have learned that System Verilog provides a whole lot of features focused on verification. But how we can determine that our verification is complete? We can generate an infinite number of random scenarios for design under test, but we need to stop our verification at some point of time and consider the design to be verified. Coverage gives us that metric which can be used to say verification is complete. This article will be more focused on the concepts of coverage and thus will be same for HDLs like System Verilog, VHDL, etc.

What is Coverage in Verification?

Coverage is a metric that measures how well the design under test (DUT) has been verified by the testbench. Coverage helps us identify the gaps in the verification process and to ensure that the DUT meets the specifications and requirements.

Coverage can be classified into two main types: code coverage and functional coverage. Code coverage helps us know how much code of the design has been executed from the TB stimulus. Functional coverage on the other hand helps us analyze if the TB stimulus is covering all the functionalities as per the spec. We will see these in detail in the next section.

Why is Coverage Important?

Coverage is important because it provides feedback on the quality and completeness of the verification process. For example, coverage can tell us whether we are generating error stimulus from TB, whether we are generating some corner case stimulus in which DUT may not work properly, etc.

Without coverage, it is difficult to know if the testbench has covered all the features and scenarios of the DUT, or if there are any missing or redundant tests. Coverage also helps optimize the verification process by reducing the number of tests or increasing the efficiency of the tests.

Code Coverage

Code coverage is a measure of how much of the code in the DUT has been executed by the testbench. It also helps us detect any dead or unreachable code in DUT. This helps optimize the design to remove any redundant code. After synthesis each line of code in design will be converted to gates or flops. Thus, optimizing design helps power and area efficiency as well.

Code coverage can also help us optimize the verification process by reducing the number of tests or increasing the efficiency of the tests. With code coverage we can know if some codes have been executed or not by our stimulus. If there are multiple stimulus executing same code, we can remove some redundant tests, similarly we can add new stimulus if some code is not getting executed.

Different types of code coverage

A code can contain different types of logic, like simple statement, if-else, expression or even FSM logic. Thus, code coverage cannot be calculated generically for the entire code. In HDL verification, we divide the code coverage in following types:

Statement Coverage

The percentage of executable statements in the DUT that have been executed at least once by the testbench. This helps us understand if there are some unreachable code or if there are some redundant code in design.

Branch coverage

The percentage of branches (if-else, case, etc.) in the DUT that have been taken by the testbench. Basically, it covers whether all the branches in if-else construct are getting executed or not. Branch coverage implies statement coverage, but not vice versa.

Expression coverage

The percentage of expressions (logical, arithmetic, etc.) in the DUT that have been evaluated to all possible values by the testbench. This basically covers whether the operands in an expression have taken all possible values or not.

In most cases operands or the variables which are part of the expression would be limited to some valid values and thus, it would not be possible to cover this completely. In such cases, we add an exclusion indicating that it is expected that the variable will take some valid values only.

Toggle coverage

The percentage of bits or signals in the DUT that have changed their values from 0 to 1 or 1 to 0 by the testbench. Toggle coverage generally slows down the simulation time for a complex design and thus as a trade-off it can sometimes be disabled.

Also toggle coverage is more prone to false coverage, as most of the nets or register will change its value on the clock edge, when DUT comes out of reset. Thus, there will be some toggle coverage even without any stimulus.

Finite state machine (FSM) coverage

The percentage of states and transitions in the FSMs in the DUT that have been visited by the testbench. This coverage includes 2 parts:

States: This indicates whether all the states of the FSM have been visited by the DUT or not. If some state is not being visited, then either TB needs new stimulus or there is some redundant state in Design is not being used.
Transitions – This covers all the possible transitions from one state to another. In FSM we know that there can be various paths through which a state can move to another state. Covering states only won’t tell us whether all paths have been verified or not.

Example FSM diagram with multiple transition paths for some states

Code coverage can be collected using tools such as simulators or code analyzers. Thus, code coverage does not require any extra coding as needed in functional coverage. Tool automatically generates a code coverage report if code coverage commands are used in simulation.

Functional Coverage

Functional coverage is a measure of how well the functionality of the DUT has been verified by the testbench. Functional coverage is based on the specifications and requirements of the DUT, not on its implementation.

In System Verilog, functional coverage can be defined by using cover-points and bins, which specify what values or ranges of values of certain signals or variables are of interest for verification. For example, a cover-point can be defined for a signal that represents an opcode, and bins can be defined for each valid opcode value. We will see more about cover groups and cover points in the next article. In this article we will limit our discussion to understanding the concept of functional verification.

Functional coverage cannot be automatically generated by a tool. All the coverage related codes are added as part of the verification plan. While analyzing the functional coverage we related all the cover points and cover groups created to a certain feature. Thus, it gives us overall coverage with regards to the functionality as per the spec.

Functional coverage can be collected using tools such as System Verilog. Functional coverage can help to ensure that all the features and scenarios of the DUT have been tested by the testbench.

Coverage Analysis

Till now we have seen different coverages and how they are generated by the tool. But analyzing coverage reports is also a crucial aspect to deciding whether verification is complete or not.

We will try to understand the analysis using 2 common scenarios:

Scenario 1 – code coverage is more than functional coverge

In this scenario, let us assume that we have 100% code coverage, but functional coverage is low (around 80-90%). We can have following possibilities:

Design code may not have implemented all the features. This is why we are hitting code coverage of 100% but functional coverage is low.
Cover points may not be written properly in TB and thus we are now hitting functional coverage numbers. Functional coverage needs to be re-analyzed.

Scenario 2 – code coverage is less than functional coverage.

In this scenario, let us assume that we have approx. 80-90% code coverage, but functional coverage is 100%. We can have following possibilities:

Design code has some redundant code which may not be part of spec. Thus, the code is not reachable with the stimulus.
Some cover points might have been missed in TB. Which means TB has not implemented all the features as per spec.
There may be some bug in the in how functional coverage is being monitored or captured in the TB. Due to this some of the cover points can get falsely covered. Thus, from the above 2 scenarios we can say that we need both the coverage metric to identify whether verification has been completed or not. Relying on any one of the coverages will give the wrong impression of coverage.

Conclusion

In this article, we have learned about the concept and importance of coverage in hardware verification. Coverage is a metric that measures how well the design under test (DUT) has been verified by the testbench. We have seen the two main types of coverage: code coverage and functional coverage. Code coverage tells us how much of the code in the DUT has been executed by the testbench, while functional coverage tells us how well the functionality of the DUT has been tested by the testbench. We have also discussed how to analyze the coverage reports and identify the gaps in the verification process.

Coverage is essential for ensuring the quality and completeness of the verification process and optimizing the design and testbench. We hope this article has helped you understand the basics of coverage and its role in hardware verification. In the next article we will learn more on the cover groups and functional coverage.