Quite often when people talk about MBT (Model Based Testing) they intuitively assume that model is always something with “boxes and arrows”; that is the model in MBT needs to be graphical and it is constructed out of some kind of nodes or vertices which are then connected using arrows of some sort. To some, model may simply mean a state chart or an activity diagram – period. It’s not really difficult to understand why people make this kind of assumption, especially in the case of MBT, because the vast majority of the MBT tools adhere to this assumption. The models, in case of MBT, indeed are something with “boxes and arrows” and actually in the vast majority of the cases they are some type of state charts.
There are multiple reasons why MBT tools in most cases are using state charts as the modeling formalism. One of them is that test generation from a state chart that represents either the test cases themselves or the testing logic and the anticipated usage of the SUT (system under test) is close to trivial. This is especially true if the MBT solutions do not take the test data into account.
The approach chosen by Conformiq differs from these traditional MBT approaches substantially; the whole test generation process of Conformiq test generation technology is driven by the semantics (‘meaning”) of the model. Even though there can be state charts in a Conformiq model, the graphical structure of the state chart is not used in any fashion to guide test generation, but only the logical meaning of the model. And this is in stark contrast with simpler state chart driven test generation approaches where the structure of a (typically only one) state chart is used to generate a sequence of tests that correspond to different paths through the state chart. This unfortunately is too often an overlooked aspect but a really important one to understand, as only the fully semantics driven approach can tolerate models where the high-level control flow is deeply dependent on data values.
It may be interesting to know that Conformiq models are internally compiled into a variant of LISP that is then processed by the test generation algorithm. The Conformiq test generation algorithm could not care less whether the model was described using a state chart, an activity diagram or even using a pure full-blown textual programming language such as Java. What matters is the logic of the model. This makes the Conformiq approach to model based test generation really unique and versatile, letting the user really model using their familiar concepts and formalisms.
To highlight this independence of the modeling formalism, Conformiq over the years has integrated its test generation platform with multiple different modeling formalisms and below are examples of those:
- In Conformiq Designer, systems are modeled as possibly multi-threaded computer programs that interact with an external, unspecified environment by message passing. These model programs are written in a combination of Java code and UML state charts. Java code is used to describe how data works in the system, to declare data types and classes, express arithmetics and conditional rules and so on, whereas the UML state charts are used to capture high-level control flow and life cycle of objects as well as event loop structures and high-level logic of message, event and timeout processing. Together, object-oriented Java code and graphical UML state charts form a powerful modeling language for the purposes of test generation.
- In Conformiq Creator, systems are modeled using activity diagrams and domain specific modeling concepts. These models are similar to Conformiq Designer in that they interact with an unspecified environment via message passing where the messages and their types are selected from a collection of readily available domain specific concepts such as button clicks, text field inputs, and such. The beauty of Conformiq Creator compared to the Conformiq Designer approach is that modeling in Creator happens fully visually using similar native components and concepts as the target application thus requiring no programming experience by the user.
- Conformiq for iRise is very recent addition to Conformiq’s product portfolio that can be used to turn iRise visualizations directly into test cases. iRise is a requirement definition and software prototyping platform that is used to create realistic simulations of web and mobile applications, desktop software, SAP applications and more without programming. These “visualizations” are quite different from typical MBT assets as it really visually looks the “real thing” with page layout, fonts, colors and everything plus it can include things like navigations, rich interactions, business logic, media, sample data, and so on. What Conformiq for iRise allows you to do is to take these iRise visualizations (which we can also call models) and import them as-is into Conformiq, then debug, validate, and improve the iRise models, and automatically generate tests, scripts and documentation with Conformiq. Users can export test documentation for business analysts and other team members, upload documentation automatically to test management tools, use tests and scripts for test execution, and run tests on the actual application or software system under test (SUT) with Conformiq.
- MODICA is an MBT solution from Berner & Mattner that addresses a challenge of systematic test case design providing the means to derive test cases from a model of the anticipated usage of the system by leveraging the test generation technology by Conformiq in its core. Indeed, compared to approaches listed above, MODICA models are not system model; they are tester or usage models, which is quite interestingly yet another example of versatility of the Conformiq platform. MODICA builds a specific and custom tailored adaption for the automotive industry in order to make it more applicable, intuitive to use and faster to deploy. The application logic with MODICA is captured using state charts that are augmented with requirements and domain specific library concepts. Currently MODICA implements out-of-the-box integration with IBM Rational DOORS and MicroNova EXAM.
- Conformiq for Alf models are described using graphical UML elements augmented with an OMG (Object Management Group) standardized textual action language called Alf (Action Language for fUML). Conceptually Conformiq for Alf and vanilla Conformiq Designer are very closely related. The difference is that with Conformiq Designer we use Java code to describe the behavioral aspects and details of your system, while with Conformiq for Alf we use Alf code. Alf is a textual surface representation of UML modeling and its primary use is to specify executable behaviors within an overall graphical UML model. Naturally some of the benefits of using Alf—as it is standardized and used across various disciplines of behavioral modeling—are that it enables developers and testers to share the same modeling vocabulary, model reuse and refinement over the software development lifecycle plus there are also a number of modeling tools available that have native support for modeling with Alf (or they are building the support as we speak) delivering even more capabilities such as simulations and early testing.
By simply looking at the list above, it’s quite clear that model does not always refer to collection of mere boxes and arrows, but there is much more to it. I have actually seen quite a few very elegant Conformiq models that have been constructed using no visual aspects whatsoever created with a pure programming language.
A very common theme with most modeling approaches is that they include some visual aspects, and this is also true with Conformiq. However, with the Conformiq platform, this is not because the underlying technology would depend on a certain type of modeling, but because those visual model assets are something that for us humans are more intuitive and straightforward to create and much faster for other stakeholders to quickly grasp and understand. This is the reason why Conformiq has adopted this way of thinking; the models must be easy for us humans to create and understand while technology used to turn those assets in to test cases must be strong and flexible enough for dealing with real world testing challenges with mind-bogglingly complex control and data flows with ease.
It well may be that maybe we have learned to accept the limitations of traditional MBT tools “as a standard” and not realized to ask for more. In my opinion we should.