Model Driven Testing - The Weblog » 2007

Conformiq Qtronic vs. Conformiq Test Generator

Tuesday, July 17th, 2007

As you can see from our web pages our product offering currently consists of two products: Qtronic and Conformiq Test Generator (aka CTG).

I’m often asked about the difference between the two. This is especially the case with those who have used CTG in the past and now try to bend their minds around Qtronic “modus operandi”.

Conformiq Test Generator represents our 2nd generation test generation technology and was released in autumn 2002 (this was preceded by Swiftest, which we consider the 1st generation). Qtronic with roughly six months in the market, on the other hand, represents the 3rd generation “state-of-the-art” model-driven test automation. At this point it should be made clear that Qtronic as we know it today is not a successor to CTG in the sense that as a CTG-user you could just take your existing CTG test model and migrate to Qtronic.

The fundamental difference between Qtronic and CTG is in what the model used as the basis for test generation describes.

With CTG you define a test model, which is essentially a description of the environment of the System Under Test. If you were testing a server your test models would define the functionality of the client that interacts with the server. In the end this boils down to UML-based graphical test scripting.

Qtronic, on the other hand, takes in the design model/behavioral description of the SUT itself, thus implementing true model-driven test generation. Returning to our client-server example with Qtronic you would define the behavior of the server itself in order to test the server.

In a way you could see these two models as mirror images of each other; where a design model has an input, a test model has an output and vice versa (this is a simplification because in the case of non-deterministic models such simple “inversion” doesn’t work). The virtue of Qtronic is that it carries out this (mathematically challenging) “inversion” of a system design into test cases, which with CTG, given the same system design spec, has to be done manually by the test model designer. Qtronic hence eliminates the need for test design.

From a process point of view you could say that CTG is a more traditional testing tool whereas Qtronic carries through the different phases of a software process by providing support for automatic deployment of early-stage design artefacts not only in development but also in testing. And well-defined designs specs of any system make for excellent documentation of the said system as well.

Posted in Uncategorized | Comments Off

ICST Call for Contributions

Wednesday, July 4th, 2007

An new series of IEEE international conferences launches next April in Norway on the subject area of “software testing, verification and validation”. I post the call for contributions on our Model Driven Testing Blog because model-based testing is one of the key areas for this new conference. The call is below:

CALL FOR CONTRIBUTIONS

1st IEEE International Conference on Software Testing, Verification and Validation (ICST 2008)

Sponsor: IEEE Computer Society Technical Committee on Software Engineering (TCSE)

April 9-11, 2008, Lillehammer, Norway

http://www.cs.colostate.edu/icst2008/index.html

There has been very strong activity in the field of testing and validation research, as reflected in recent years by numerous workshops dealing with testing in different areas. The new IEEE International Conference on Software Testing Verification and Validation (ICST) will offer an open forum for software testing, verification and validation research and its transfer to practice. Among other things, it aims at stimulating scientific research on model-based software testing, domain specific testing, empirical studies of testing techniques, and the technology transfer of research results to software development practices.

We invite original, high-quality research papers and industrial experience papers in all areas of software testing, verification and validation.

Topics of interest include, but are not limited to:

- Software testing theory and practice
- Model-based testing
- Domain specific testing including, but not limited to,
security testing, web services testing, database testing,
and OO software testing
- Verification & Validation
- Quality Assurance
- Model checking
- Empirical studies
- Metrics
- Inspections
- Tools
- Testability and diagnosability
- Design for testability
- Testing education
- Testing in multidisciplinary applications
- Embedded software
- Technology transfer
- Model-Driven Engineering
- Novel approaches to software reliability assessment

Conference Chair: Lionel Briand
Program Chairs: Rob Hierons, Aditya Mathur
Workshop Chair: Benoit Baudry
Industry Chair: Per Runeson, Clay Williams

Important dates:
Submission of abstracts: October 5, 2007
Submission of full papers: October 12, 2007
Notification: January 4, 2008
Camera-ready: February 1, 2008

For ICST 2008 submission guidelines and all additional up to date information, please visit http://www.cs.colostate.edu/icst2008/index.html

Posted in Uncategorized | No Comments »

Challenge to Our Competitors: Provide Free Evaluation Downloads

Tuesday, July 3rd, 2007

Conformiq Qtronic has been available on the web for completely free, even anonymous, evaluation download since the tool was commercially released. However, our customers have had big difficulties in getting the tools from our competitors for evaluation.

One said that “we are evaluating also the Leirios tool… even though they do not provide an evaluation [download]”. A Finnish researcher from the governmental research institution writes in his blog that people from T-Vec answered that “they do not provide evaluations for students and neither to Finland”.

Reactive Systems, however, provides an evaluation download.

As far as we know, the only tool that can claim to be on par with Conformiq Qtronic with regards to generation of test data is, however, the Leirios Test Designer… so we pose this concrete challenge to our competitors and especially the French company: make an evaluation copy of your tool available so that users of model-based testing can make cost-efficient and informed purchase decisions! The commercial model-based testing market is still in quite early stages, and people need to spend lots of effort in understanding the globally available offering… by providing free evaluation downloads you would contribute significantly to awareness about model-based testing and the maturity of tools.

There are many dimensions in which the model-based testing users need to be able to compare the tools, including:

The ease of adoption and use of the system modeling languages. There is much difference in the solutions; Qtronic uses C#/Java + UML, Leirios tool uses (as far as we know) UML + OCL, Reactis is based on Simulink and Stateflow…
The speed of test case generation. There is almost no information about how fast the tools are for generating test cases, even though it is known that computationally intensive algorithms are required.
The quality of generated test cases. Once the test case generation runs the important question is what you get out of it. This is very context-dependent so it is very important for the model-based testing users to experiment with test case generation in their own context.
Availability of online testing. Online testing provides benefits over offline test case generation when system models are nondeterministic. Nobody can neglect the fact that both the academic community as well as the commercial companies (in addition to Conformiq, for example Microsoft in their SpecExplorer tool) have recognized the importance of online testing.
User experience in general.

So please answer to our challenge; provide your tools freely available for evaluation for the better of the model-based testing users as well as for the advancement of the model-based testing market!

Posted in Uncategorized | No Comments »

Archives

Categories

Archive for July, 2007

Conformiq Qtronic vs. Conformiq Test Generator

ICST Call for Contributions

Challenge to Our Competitors: Provide Free Evaluation Downloads