RIA Quality Assurance
The architectural complexity of RIAs calls for special approaches in running tests and achieving quality assurance. For instance, an application may support different workflows based on the role of the user working with the application. A thorough testing of such an application requires walking through all possible navigational paths. Additionally, modern client/server communication mechanisms impose many complexities caused by the “richness” and sometimes verbosity of the data exchange protocols. For instance, Flex clients can communicate with the backend systems using binary protocols like AMF or RTMP. Testing tools must be able to interpret the data at the protocol level to enable new level of testing control and quality assurance.
Introducing RIA AppPuncher™
RIA AppPuncher™ is a software testing and debugging product. AppPuncher was designed from the ground up to faciliate testing of the Rich Internet Applications. The product provides the following functionality:
AppPuncher supports Flash, Flex, Silverlight, AJAX clients as well as traditional browser-based applications. From the client/server protocol perspective, the product enables testin of the client/server applications using any of the following protocols: HTTP, SOAP, REST/XML, AMF and RTMP. AppPuncher runs on Windows, MAC OS X and *nix platforms.
Client/server communication debugging
Communication debugging consists of the following features:
Visualization – AppPuncher displays client/server traffic to detect application faults, invalid requests and responses
Evaluation – User can evaluate application behavior (responses) using custom, user-defined values in requests. Any captured request can be replayed with custom data entered by the user in order to inspect application’s response. For example, a SOAP or AMF method invocation can be modified to contain user-defined value for a method argument.
- Pause Conditions – AppPuncher can be configured to pause application execution when a user-defined condition evaluates to true.
Functional Testing
AppPuncher supports chaining of the requests where output of an invocation can be used as input for another one. Using this approach it is possible to achieve a high level of functionality coverage. Additionally, AppPuncher can execute functional testing by applying high load scenarios.
Regression Testing
AppPuncher can capture messages between an RIA client and the server backend application. Captured messages are in the form of issued requests and the corresponding responses. Messages can be grouped to represent user activity in a particular role and then saved as a repeatable test-case. A test-case can be ran at any point of time to make sure the server returns the same responses as during the original execution. Additionally, user can create a schedule for automated test case execution. At the end of the test case execution, AppPuncher creates a summary report which can be delivered via email.
Load Testing
Before an RIA is released into production, it is very important to understand the limits of the solution when it serves multiple users. Creating a real-world simulation with a high number of concurrent clients using the RIA can be a challenging task. AppPuncher provides a solution for creating and running stress tests for RIAs. The product supports several scenarios for conducting stress tests. It can be configured to increase the load until there is a server failure, alternatively it allows completely custom execution stress test configurations - test administrator can create a chart representing the number of concurrent users at any point of time. Similar to the regression testing, AppPuncher generates a report for every stress test execution.
Debugger Edition |
Regression Test Edition |
Stress Test Edition |
Ultimate Edition |
|
|---|---|---|---|---|
| Client-server debugging |  |
|
|
|
| AMF0 protocol support | |
|
|
|
| AMF3 protocol support | |
|
|
|
| HTTP, SOAP, XML/REST support | |
|
|
|
| Payload content substitution | |
|
|
|
| RTMP, RTMPT protocol support | |
|
||
| Play and record | |
|
|
|
| Invocation chaining - functional testing | |
|
|
|
| Regression testing | |
|
|
|
| Stress test designer | |
|
||
| HTTP-based multi-profile testing | |
|
|
|
| RTMP-based multi-profile testing | |
|
|
|
| Video streaming stress testing | |
|
||
| RSO stress testing | |
|
||
| Data push stress testing | |
|
||
| Scheduled tests | |
|
|
|
| Test reporting | |
|
|
|
| Server monitoring during tests (Memory, CPU) | |
|||
| Service coverage testing | |
|||
| Exporting tests as scripts (junit, nunit, asunit) | |
|||
| Test repository, centralized test coordination | |
|||
| Availability | Nov'08 | Q1'09 | Q1'09 | Q1'09 |