When talking about black-box and white-box testing, the term “box” refers to the system under test or SUT where the “color of the box” refers to the visibility that a tester has to the internal details of the SUT. When we talk about black-box testing, the tester judges the quality and correctness of the system without seeing inside this box meaning that the tester has and needs no details of the internal workings of the SUT in order to test the system. White-box testing, in contrast, assumes that the tester has a full visibility to the box being tested and how the system internally operates.
Black-box testing validates whether the given system conforms to its design and functional specification. The test cases that a model-based testing tool like Conformiq Designer generates are black-box by nature which means that they depend on the model and the interfaces of the system under test, but not on the internal structure of the implementation. One does not require an understanding how the system has been architected internally in order to create a system model with Conformiq Designer.
In order to describe the degree to which the source code has been tested, it is quite common to use code coverage as a measure. A system with high code coverage is considered to be more thoroughly tested than one with low code coverage, therefore lowering the change of containing a software defect. Code coverage is often the applied methodology in white-box testing.
In model-based testing, as a testing heuristic, one often also uses various coverage metrics to measure the quality of generated test suite with respect to the model from which the test suite was derived. Assuming that the model represents the aspects of the functional specification accurately, we can use the coverage figures reported by an MBT tool to argue about the coverage of the functional specification; these coverage figures help us understand to what degree the functional specification is covered and tested by the generated test suite.
However, a generated black-box test suite with a demonstrably good coverage of the functional specification (whether implicit or explicit) does not necessary mean a high white-box coverage measurement. This is because there can be disproportionate amounts of code in the system that are either unused or dead code or that are related to unspecified functionality. It is known that good coverage of functional requirements does not always lead to high white-box coverage due to the aforementioned issues. On the other hand, good white-box coverage can be of only limited value if the tests are derived from the code itself. The key point of model-based black-box testing is that the system is judged against an independent reference.
Therefore one should always be careful to cast premature judgments’ after seeing poor white-box coverage figures when running black-box tests and should start by asking all kinds of questions about the uncovered code in the implementation since it does not seem to be related to any specified behavior.