If you do not test it, you do not know it works.

Use automated testing, or it does not happen.

We automate all our testing. We use several test tools to ensure good coverage of features, and use cases both from a code point of view, and from the user point of view. We also inlude automated performance testing to ensure responsiveness and throughput do not suffer during the course of a project.

UI Testing

Code level testing is not enough. Testing must include testing from the UI to ensure all the software is being exercised.

Usability Testing

Early in every project the usability of the interface needs to be validated. Just as software must be tested to know if it works, the design of the interface must be tested against humans to know if it works.

Functional Testing

Testing the functions of the software is best done using an inteface that is not too tightly tied to the UI. For web applications this is best done with HTTP or code level APIs. For code libraries or desktop apps this is done with code level APIs.

Performance Testing

Performance is an important aspect of any application and needs to be tested, just like the other aspects.

Complexity

For any large applicaation or web site, the combination of operations can become too numerous to test exhaustively. There are several approaches to this issue.

A common approach is to randomly select between equivalent operations. Another approach is to chain use cases together hoping for some interferrence from one use case to another. By combining both these approches, and performing tests concurrently the best results can be acheived in reasonable time.

The optimum solution would be to analyze each use case for data dependencies and then execute all combinations of operations that interact with the same data. This is not feasible with current tooling, but reamins a goal.

Selenium

We use the Selenium UI test tool with the seleniumrc_fu rails plugin to automate use case testing.

Test::Unit

Rails inludes the Test::Unit test framework and adapts it to both code level unit testing and funtional web testing.

Performance Testing

Both seleniumrc_fu and runit allow scripting in ruby which can be used to provide performance testing for both code level operations and end to end UI testing.

Concurrent Testing

TechnoMage is extending the test frameworks to allow tests to be run concurrently to make good use of multi-core computers, in addition to running tests on separate physial or virtual machines.