Need for Something New?

There are a crop of available test management systems that exist, but the question that must be answered is “do they meet our needs?” While you could theoretically use many of these tools to some effect, many of them have serious disadvantages in the context of heavy test automation.

The basic features that exist in test management systems include:

·         Test case and test suite management

·         Test cycle and test release management

·         Planning, group tasking and scheduling

·         Defect tracking

·         Reporting and trends analysis (usually defects trends, execution trends)

·         Traceability

Let’s take an honest look at how these features are often used on real projects:

Test case and test suite management

Once created, test cases are rarely kept up to date. Some of them may be used for a subsequent release, some will be junk. Also, most of the tools I’ve seen tend to complicate test case writing – multiple tabs and double clicks to navigate back and forth, page response, etc. I recognize the advantage of fielded data but should it come at the expense of features such as auto formatting, formulas, very simple copying and cloning? Many testers think not, and desire this type of simplicity, evident in the fact that they often create cases in Excel spreadsheets and then import them into the test management system for managers. 

Test cycle and release management

Basically, this nice feature is often implemented as a folder that inherits test cases from the test plan. Since the test management tool is often disconnected from the Continuous Integration (CI) server, this feature is not automatically tied to the creation of a new release on the CI server, which creates a disconnect that can make it tedious to keep the CI server and test management tool in sync.

Planning

As a manager, you can assign any test or test suite for execution for a particular day. But who wants to repeatedly perform this boring activity? Typically, testing tasks are based on the functionality to which a tester has been assigned.

Defect tracking

Who knows, maybe some companies use this feature. Very often, however, this feature of a test management solution goes to waste, because the organization uses some separate defect tracking solution or has no defect tracking solution at all.

Reporting

I suppose this is the most important feature but only if other features are used accurately. There is no value in the reporting of incorrect or incomplete data.

Traceability

If you have granular requirements in a single Test or Application Lifecycle Management (ALM) tool, it is nice to be able to obtain automatic linkages between tests and requirements. If specifications live somewhere outside of the tool used for test management, there is minimal value from this feature because the linking process and the process of keeping the links up to date will be a manual task.

So, conclusion is that existing test management tools are useful when their whole set of functionality is used; particularly when used on mid-to large scaled projects, on distributed and/or remote teams and for cross project analysis. Otherwise, they may be more trouble than they are worth. So we must ask ourselves the following question: “If these tools are flawed for manual test case management, why we should we rely on them from a test automation perspective?”