I arrived yesterday to Tallinn, Estonia by a catamaran from the south coast of Finland, a 90 minutes ride that was surprisingly smooth, to take part in the combined TESTCOM/FATES 2007 conference (co-located with FORTE).

Today morning I gave my invited talk on “Implementing Conformiq Qtronic”, in which I surveyed the basic architecture of our model driven testing tool and the different challenges we have met while working with the implementation.

I think that the two important messages I tried to put forth were that (1) it is possible to do successful model driven testing with infinite state space models and (2) that it requires much more than a test generation algorithm to make a commercial-quality model driven testing solution.

After my talk, the program has continued with the presentations of peer-reviewed articles and at least today the focus has been pretty much on finite state machines… with all (due) respect, I believe it is about the time for the research community to step out of this FSM sandbox and start to look around for the real world!

I gave today a tutorial on “model driven generation of TTCN-3 test cases” at the TTCN-3 User Conference 2007. The audience at the Ericsson building in Kista, Stockholm was maybe from fifty to one hundred people.

In this blog entry I will go through some of the questions people asked from the audience.

Q: “You said that model driven testing eliminates the risk of incorrect test cases because test cases are generated by an algorithm instead of a human. But the model itself is designed by a human, so isn’t the risk there still?”

A: “Yes, the model is designed by a human being and there is no way around this. But the model is nearer to the functional requirements than the test cases and is a smaller artifact than the corresponding test case library in general. Therefore reviewing the model and ensuring its correctness takes less time and incurs less costs.”

Q: “Does your approach also help in generating the test data as opposed to test behavior?”

A: “In our approach there is really no clear separation between test data and test behavior. If your system’s behavior depends on the data it receives, then you must model the data, and once you do this you get test data also in the test cases. Our solution has full support for the generation of test data because without it you cannot realize the desired behavior in any form either.”

Q: “Is the TTCN-3 code [shown in demonstration] completely computer-generated? Also the comments?”

A: “Yes. Of course our product cannot comment intelligently on the meaning of the model, but for example if you include links to your system requirements in the model, these links are visible also as comments in the generated test cases.”

Q: “You did not mention state space explosion in your talk.”

A: “It is certainly true that you can create models that will choke up our solution, there is no doubt to that. But on the other hand the testing heuristics we use (e.g. boundary value analysis of condition coverage) scale up linearly or low-degree polynomially in the size of the model, instead of e.g. exponential growth. This means that the expected number of test cases is manageable. Also, because our tool uses internally symbolic methods for state space exploration, the state space explosion problem takes a different form in our context.”

For more information about TTCN-3, check out www.ttcn-3.org.

Hi there! This entry comes from the Cisco Toolapalooza 2007, Cisco’s by invitation trade show for testing tool vendors related to Cisco’s business.

It’s always striking how the people on the Silicon Valley (and U.S. in general) are open and direct and up-to-the-point when compared to their European colleagues. It’s definitely a different atmosphere—the sense of efficiency, if you wish, is everywhere in the air.

This event was shadowed, most unfortunately, by a horrendous massacre at the Virginia Tech, certainly the number one news in the States for many days in a row. The country is showing respect to the murdered and their families and the flags are half-way down everywhere.

Our presentation on our offering is still upcoming in a few hours, and there will be also a presentation on model-based testing by, it seems, Cisco people themselves. It makes sense as the guys here are always scanning for new ideas to make their life better and their products more competitive. Certainly many things could and should be learned by the European industry from the attitude here. No wonder that States is the #1 power in software industry.

Here at the end a picture of my colleague happy at the stand…

I spent the last weekend in sunny Braga, Northern Portugal. The occasion was the Model Based Testing 2007 workshop, held in conjunction with the ETAPS conference. The country was beautiful, weather pleasant and food very cheap (at least when compared with the Finnish price level).

On the scientific level, the quality of the papers was mixed. Maybe one of the highlights was the presentation by Swedish researchers about efficient test generation for timed systems. This reasearch was, of course, coupled with the well-known UPPAAL toolset and its derivatives.

On Sunday morning I gave my invited talk on the use of model-based testing in standardization. This joint presentation by Mr. Stephan Schulz from ETSI (European Telecommunications Standards Institute) was more about pragmatics than theory. In particular, in the presentation I summarized how the application of model-based testing in standardization differs from applying it in, say, in-house software development.

Currently there are so many small workshops and conferences touching model-based testing (e.g. MBT, A-MOST, MoDeVa, FATES, M-TOOS) that one could guess some consolidation will take place in the future. Maybe a conference dedicated to model-based testing..? Whatever “model-based testing” then means. I proposed in the workshop that one interpretation of “model-based testing” in the research world could be “algorithmic methods for black-box testing that are based on behavioral models of systems under test”, and this was received quite well.