We are often asked if we execute test cases and how, so here some comments regarding that. Conformiq Designer generates tests but does not execute them. In order to automatically execute tests generated by Conformiq Designer, you need to get them out of Conformiq Designer in a format that an external test execution platform can interpret. You achieve this by using script backends. These are plug-ins that you can connect to your Conformiq Designer projects, and their purpose is to render the mathematically generated test cases in concrete, executable formats. The plug-ins connect to an open API, so they can be created freely by any user. We have also a collection of 20+ plug-ins for different test execution systems and platforms available (for free).
Here are some concrete examples how people have used Conformiq Designer and script backends to generate executable test cases and to run them successfully:
- In telecommunication networking industry, a customer uses a TTCN-3 backend to generate executable TTCN-3 test scripts that are then executed on an independent TTCN-3 execution tool. (Learn more about TTCN-3 here.) In this fashion the customer is able to share a common TTCN-3 framework between computer-generated and hand-written test cases across development units.
- Within an IT department, a customer uses a Java JUnit backend to generate executable Java code that is executed on top of the JUnit platform and that executes browser test cases using Selenium and Selenium Remote Control (learn more about Selenium). This enables the customer to harness the power of the Selenium open source web testing solution and at the same time avoid time-consuming, manual creation of test cases.
- In another IT case, a customer uses Visual Basic for Quick Test Professional backend to generate executable QTP test cases and test projects. Test cases are also maintained automatically within HP ALM (previously HP Quality Center). By doing this the customer can integrate Conformiq Automated Test Design with all the existing testing and auditing processes and can continue to leverage the expertise developed in house for the HP tool chain.
- In the mobile devices space, a customer uses a C backend to generate C code that links directly with the system under test to drive it through APIs. This enables the customer to have as lean and direct connection as possible to the system and to avoid any overhead related to interpreting test scripts or remotely driven test execution.
- In telecommunications industry, a customer uses a custom backend to generate test scripts for a home-grown test execution platform. The customer can continue to leverage the previous investments in a specialized test execution platform while changing the methodology used to create test cases.
- In an enterprise systems case, a customer generates project files for SoapUI for testing WSDL interfaces and the business logic behind them. This makes it possible for the customer to execute the WSDL tests from a professional, interactive environment while getting the test cases themselves automatically from our model-based testing tool. (Learn more about SoapUI.)
This list could be continued of course, but the point is that Conformiq Designer has been and can be used successfully across multiple different test execution platforms, thanks to the open architecture and the availability of multiple script backends for common test scripting languages and platforms. The open API prevents technology lock-in and lets our customers to decide and develop their test execution automation strategy freely without being encumbered by any restrictions from Conformiq’s side.