I recently visited a client to discuss many topics surrounding their software testing and deployment project process.  Though there were many outputs from these meetings, there was a very obvious bottleneck in their QA area of the processes.  Due to this we spent a good deal of time on this and I felt it would be worth sharing.

This client is a client of a particular software vendor, so they aren’t the developing party.  Please keep this in mind.  What this essentially means, is that much of the “unit” type testing surrounding Alpha or Beta releases (or even final releases) should be unnecessary, however this particular client is finding itself peforming much of the unit testing that it should have the confidence with being completed by the vendor.

Here is some info passed onto them:

When testing major software upgrades, the test approach should focus on business flow and not on application unit testing (except for client custom-developed code).  The basic approach (in this order of execution) to testing is as follows:

1. End to End Testing: Approx: 5% total testing effort
2. New Feature Testing: Approx 30% – 40% total testing effort
3. Regressions/Baseline Testing: Approx 60 – 70% total testing effort

The testing effort kicks off with end-to-end testing, which puts items completely through the standard business workflow.  It is designed to identify basic problems quickly and without requiring several iterations to get a first pass through the entire workflow.  It is not intended to test specific test conditions such as exception processing, etc.

Once the end-to-end test is complete, the QA team will drill down on specific new functional areas for detailed testing.  There may be one or more test cycles associated with a given functional area, and the QA team may test new functionality testing in parallel with regression tesing, if it makes most sense.  The regression test, however, cannot be considered complete (or in some cases started) until the new functionality testing is complete.  The QA team will identify each of the functional areas to be tested and the associated testing at the outset of the test in the form of a test plan – they will strive to bring closure to each functionality area as soon as possible and provide regular progress reports to the appropriate team members.

Each new release brings additional functionality, and new test cycles will need to be developed to validate that new functionality, particularly as it is applied to the clients operating environment.