ASIC Verfication Do I need to verify all possible combinations?

Question

I'm currently doing ASIC Black box Verification.

Suppose I got a module with 200 input ports with 12 bit width each and a one output port with 64 bit width. Lets say, its pure combinational inside.

                                  [11:0]      +------------+
                 inputport 0   ------/------> |            |
                      .        -------------> |            |       [63:0]
                      .        -------------> |     duv    | --------/----> outputport
                      .        -------------> |            |
                 inputport 200 -------------> |            |
                                              +------------+

To test the outputport, do I need to test all possible combinations of all 200 input ports?

If no, then what is a good way in verifying such module?

Is there a good way in monitoring all of the corner cases?

Will the bug(s) will eventually come out during the process?

Answer 1

OK, black box is not a problem, as long as you know the functionality. You'll need something like 64 logical functions in (maximum) 2400 variables.

Worst case you have an untestable product. That's for instance if each of the 64 outputs indeed depends on all input states. I imagine values of different ports are compared for equality, or greater than, and those results are used in a logical combination. In that case you could test for values around the comparison value, which can dramatically decrease the number of combinations.

Say output Y should be high if port A > port B. You could apply the value 0x25 to port B, and check what the output is for values 0x24, 0x25 and 0x26 of port A. Repeat for 0x211 and 0xE5A (I'm just making up numbers). Then you have reduced 2\$^{24}\$ = 16 777 216 combinations to 9.

You may have a problem here. Even if that's indeed the comparison you need to make, the result may be internal, and only come out after some logic function with similar internal results. That's how you make an untestable product.

I would have split up the design into several ASICs, where intermediate results go from one ASIC to the next. This way you might only have to test \$O(\sqrt{N})\$ combinations, instead of \$O(N)\$. 1000 billion tests may be impossible, but 1 million is not that large a number. It may also decrease I/O number if there are mutually independent functions.

A solution for the single chip might be to have test pads for intermediate results on the die, so that they can be tested on the wafer with flying probe testing.

Answer 2

You have an absurd number of possible input sets in your input space. You can't test all of them, period. You have to test a subset of the input sets and use some worthwhile heuristic to choose the sets you do test.

Here's what your testing must do (this is derived from DO-254 test recommendations that my company uses):

• Each output must change at least once
• Each input must change at least once

That's a minimum, not a maximum. In cases like this we perform Minimum Condition Decision Coverage (MCDC) analysis to see which other input sets we need to test. I've always explained it like this:

Pretend your black box is an AND gate with 200 inputs. You can't test every single input set, so you test a subset of the whole input space. First, you follow the rules I gave above - you need to test at least two cases: one which causes the output to be 0/False and another that causes the output to be 1/True. For the AND gate this is easy: all 0/False and all 1/True. That proves your output can change. The next thing you do is to choose input sets such that there is an input set for which each input is the deciding factor in the output. For a 200 input AND gate this gives you 200 different input sets: the first one has all inputs True except for input #1, the second has all inputs True except for #2 and so on. Using this methodology you verify that each input has an independent effect on the output. If any input being False doesn't produce a False output then you know you have a problem.

You said your logic is all combinatorial. This is preferable to a whole lot of latches/flip flops but still not perfect. You have to work backwards from the output to determine input sets for which each individual input has a deciding effect on the output. If you logic is complicated then you'll have a lot of work ahead of you, but this is the quickest and most comprehensive way of testing your logic black box.

As far as the practical aspects you're on your own. The company I work for uses custom hardware to do in-system pin-level black box testing of FPGAs (and theoretically ASICs if you could provide us a part to put on a board) at operational speeds. It's comprehensive and automated but each test fixture is custom for the given project, so using this approach requires a contract and not just buying off the shelf items and combining them yourself.

One of the problems you'll have that is alleviated by our approach is simply connecting all the wires. I guess most people would suggest an obscenely large logic analyzer but you've got thousands of pins so you'd have to test subsets of pins at a time. One of the positive aspects of the approach we use is that it monitors all pins at all times so it lets you seen that the outputs you expect to change do change, but also that the outputs that shouldn't change don't either. You can't get that if you use a logic analyzer with fewer pins that the number of inputs/outputs you have.