Since Conformiq’s Automatic Test Design™ technology development started in 2003, we have been constantly pushing the technology forward to make model-based testing applicable to an ever growing set of industrial-scale software test design problems. This section provides some highlights of how Conformiq Automatic Test Design has been implemented. More detailed information is available in our Blogs and Whitepapers.
Automated Test Design
The core of Conformiq test generation technology is a unique semantics driven, symbolic execution based, test generation algorithm. The test generation algorithm searches this part of the state space for testing goals and the number of testing goals in a model depends on the testing heuristics selected by the user. The algorithm then constructs tests from the explored part of the state space by selecting paths that lead to testing goals, then converting those paths to tests. Every input on an execution path (to the system model) becomes a test stimulus (to be sent to the real system), and every output of an execution path becomes an expected output (to be verified during test execution). This is how Conformiq automatically thinks of and designs test cases.
Conformiq is applying numerous AI techniques including Natural Language Processing (NLP) for automatic model creation. This is a capability to “reverse engineer” a formal model from existing test assets such as manually written test cases. This approach has been incorporated into the new CITA (Conformiq Intelligent Test Automation) capability that is being inserted into scripters to enable user-driven learning to greatly simplify the linkage needed between generated test stimuli (executable scripts) and the physical objects (buttons, pop-ups, fill forms, etc.) they activate during automated test execution. Further uses leveraging CITA technology are in development.
Creator enables and accelerates organizations to advance to continuous testing by providing extensive integrations with requirement/user story tools and automated test execution tools. The key is Creator’s unique capability to bridge the gap between these tools by importing requirements and using them to help automatically generate executable test scripts and validations directly using automated test execution platforms. This unique capability means there is no big manual step needed to write test scripts and think of the test validations which totally destroys any continuous integration. Creator’s end-to-end automation means there are no manual steps that would limit or slow continuous testing.
Conformiq supports testing enterprise applications “Out of the Box” and custom frameworks to automate functional test data. Using Conformiq, packaged application models can be created and tests generated. Some examples of the enterprise and packaged applications that users and services partners have tested include SAP, Oracle FLEXCUBE, Temenos, Guidewire, and Avaloq.
Conformiq user interaction libraries include typical mobile actions such as pinch and swipe to make modeling easier and faster. Further, users can create their own custom library elements to capture additional operations. Execution would be through interfaces with testing tools including Mobile Test Center from Micro Focus and Appium, among others.
Conformiq’s unique tool architecture includes open APIs surrounding the core test design engine. This access allows Conformiq and the end users to create and enhance their own custom interfaces for increased end to end testing flexibility. CITA simplifies the ability to integrate tightly with proprietary customer platforms and platforms from testing service provider companies resulting in improved efficiencies.
Conformiq generated tests are all data driven and Conformiq is unique in this capability due to its ability to automatically include the test data needed for generated test cases by deriving it directly from the model. This means testers don’t need to spend significant time waiting to be told what data to include in their test cases. Conformiq software will do that automatically. Users can still enter data values to be used or include variables as data placeholders. If desired, testers can use externalized data, i.e., data outside of the generated test cases, such as production data from TDM tools, for execution to replace the generated or previously used data. This gives full flexibility to both the Conformiq users and the test execution teams.
Conformiq supports test automation of mainframe applications including testing with desktop user interface operations. Models are created from mainframe operations as custom library actions. Scripts for test automation are generated in the proper code needed for execution which might include multiple types of code for testing both the mainframe as well as the devices attached to it. 3270 type screen operations would use an emulator. Conformiq’s new AI based technology has special support for alternative mappings of actions based on their target interface type (web versus mainframe, etc.). In addition to modeling of mainframe actions, Conformiq also supports integration testing with other types of interactions (API, mobile, etc.).
Conformiq generates test scripts in the language, format, and syntax needed for direct execution in the user’s framework(s) of choice. The ability to match the exact input needs of execution tools, especially those that have been customized by adding key words and libraries means that applications written with almost any language can be tested so long as the execution tools used can run against that language. Thus Conformiq can test the variety of technologies used across large enterprises.
Since its inception, Conformiq technology has been available on both Windows and common Linux distributions. All Windows operating releases from the past several years are supported.
For very large test generation problems that are computationally complex and can take a long time to process, Conformiq uses a client server architecture where extremely large models requiring more computation power than is available on a user’s local computer to be executed on a much more powerful computation engine such as a blade server or on multiple PC nodes, for example, in a private or public cloud environment.
Very large or complex models can take significant time to design the optimized test cases on a single core PC. Conformiq’s unique multi-core architecture for the mathematics of test generation allows users to split test generation tasks across all local computation cores in order to speed up the test generation process. The process is completely deterministic.
Confromiq models can have multiple asynchronous components and for communications between those components within the model. The predicted internal communications can be reviewed in our tool and can be verified during test execution. This allows testing together of multiple types of technology such as user interface operation on a desk top with APIs and execution or data on a mainframe.
Conformiq supports all major mobile and non-mobile browsers (Chrome, IE, Firefox, Safari, etc.) as a test target by supporting the capabilities of the test execution platform being used. Using Transformer with Selenium, multiple browsers can be executed in parallel.