As detailed in a blog post I wrote a couple of months ago, the core of Conformiq DesignerTM is a custom crafted semantics driven, symbolic state space exploration algorithm for test generation from system models (because this is really the only known solution that robustly generates both test inputs and outputs from a system model without user intervention). On a very high level, this algorithm selects which execution paths in the model to expand and to what extent and Conformiq has been carefully tuned our implementation during the last decade to make the operation more efficient, robust, comprehensive, and user tunable.
Because large real world test generation problems are computationally very complex and can take long to process on a single PC, this test generation algorithm has been carefully crafted for fully distributed parallel test generation, allowing users automatically to split test generation tasks across multiple cores and PCs in, for example, a public or private cloud / cluster environment or just a large server. This technology enables users to create computational clusters for rapid test generation that can be shared across the users. However even without access to a cloud or a cluster environment, users can still gain the benefits of parallel test generation with Conformiq Designer because even on a single PC setup it automatically maximizes the use of computation resources available locally on the workstation to speed up the test generation process by distributing computation work over all the processor cores available on the local host.
On the surface, this may seem like a simple thing and not a huge technical effort to create, but that is wrong because the real trick is to do it deterministically. In the context of highly distributed test generation, guaranteeing deterministic test generation results is an extremely challenging technical issue to solve. Conformiq Designer is guaranteed to generate test cases deterministically, regardless of the number of processor cores, their speed, and load. Multiple test generation runs on the same model result in generation of the same test case set. Without being deterministic, the tool would be like a slot machine so every time you generate tests you could get completely different test coverage and test cases.
To provide some technical architecture detail, Conformiq uses a master/slave architecture and CORBA as middleware. The slaves are responsible for expanding a given part of the symbolic state space and reporting all their findings back to the master component which then coordinates the work of the slaves by assigning them more work. Since the Conformiq test generation server may be accessed from multiple users, the server also handles the distribution of the computation resources between any concurrently running tasks. This architecture involves high performance load balancing and recovery, meaning that the physical location and number of processor cores actually used may vary while test generation is running. The tool will always balance the available computational resources between all ongoing computation tasks submitted from users.
This parallel test generation algorithm parallelizes relatively efficiently, providing savings of up to 90% of test generation time by scaling from one to sixteen processor cores, and even more efficiency gain when deployed on a computation cluster with tens or even hundreds of cores. This is important as it improves the productivity of automated test design by cutting down wait time before users see their newly generated tests. As this is typically an iterative process to achieve a validated model and desired test coverage and is the most time consuming part of the automated test generation process, delays during this part of the MBT process are multiplied with every iteration. Therefore, distributed test design processing is a key to MBT success.