Challenge to Our Competitors: Provide Free Evaluation Downloads
Conformiq Qtronic has been available on the web for completely free, even anonymous, evaluation download since the tool was commercially released. However, our customers have had big difficulties in getting the tools from our competitors for evaluation.
One said that “we are evaluating also the Leirios tool… even though they do not provide an evaluation [download]”. A Finnish researcher from the governmental research institution writes in his blog that people from T-Vec answered that “they do not provide evaluations for students and neither to Finland”.
Reactive Systems, however, provides an evaluation download.
As far as we know, the only tool that can claim to be on par with Conformiq Qtronic with regards to generation of test data is, however, the Leirios Test Designer… so we pose this concrete challenge to our competitors and especially the French company: make an evaluation copy of your tool available so that users of model-based testing can make cost-efficient and informed purchase decisions! The commercial model-based testing market is still in quite early stages, and people need to spend lots of effort in understanding the globally available offering… by providing free evaluation downloads you would contribute significantly to awareness about model-based testing and the maturity of tools.
There are many dimensions in which the model-based testing users need to be able to compare the tools, including:
- The ease of adoption and use of the system modeling languages. There is much difference in the solutions; Qtronic uses C#/Java + UML, Leirios tool uses (as far as we know) UML + OCL, Reactis is based on Simulink and Stateflow…
- The speed of test case generation. There is almost no information about how fast the tools are for generating test cases, even though it is known that computationally intensive algorithms are required.
- The quality of generated test cases. Once the test case generation runs the important question is what you get out of it. This is very context-dependent so it is very important for the model-based testing users to experiment with test case generation in their own context.
- Availability of online testing. Online testing provides benefits over offline test case generation when system models are nondeterministic. Nobody can neglect the fact that both the academic community as well as the commercial companies (in addition to Conformiq, for example Microsoft in their SpecExplorer tool) have recognized the importance of online testing.
- User experience in general.
So please answer to our challenge; provide your tools freely available for evaluation for the better of the model-based testing users as well as for the advancement of the model-based testing market!