The previous post on this SQA tools series delved in Application Life-Cycle Management tools. This post will review the next link in the SQA chain, the test management tools. As in the rest of these series, we are only looking at those options that are available as SaaS.
This time the analysis will mostly be based on two main characteristics, agile fit, and flexibility. By agile-fit we try to distinguish those test management tools that have a better layout for agile development. When we talk about flexibility, we refer to the facilities the tools give to export or import data from the tests, whether their offer an API to create custom extensions to the platforms.
The importance of these two features is acknowledged by most, if not all, of the vendors. As such, most of the tools tick the agile-fit and flexibility check boxes. Without further ado, we will briefly analyse the tools we have found. The table shows a quick outlook on how the tools fulfill the expectations in the two categories. The more ✔ the better, and one ✘ means that the feature is not a big selling point for that concrete solution.
Analysis of the Test Management Tools
The tools can be roughly classified into two big categories: those that belong to a bigger suite of software development tools; and those that are independent. In the first group we can find tools from IBM, Seapine Software, Smartbear, Tricentis. In the second groups we can find both robust and nimble solutions from Teststuff, TestCollab, TestLodge, TestRail, TestRun, and Zephyr.
This group distinction shapes how flexible are the test management tools. Those tools that depend on other vendors have a design so that it facilitates integrating the test management with project management, requirements management, source code repositories, etc. Worth mentioning in this category are tools like Catch Software’s Enterprise Tester, QMetry, TestRail, and Zephyr. On the other hand, tools that are already integrated do not have the same facilities, like Seapine’s TestTrack. However, we can still find products in this category that excel in flexibility: Tricentis’ Tosca Suite. The outlier when it comes to flexibility is Test Run, a simple solution that just does Test Management and does not mess with anything else.
The agile-fit evaluation is harder to make, as most of the times, when it comes to Test Management; agile ready solutions basically mean a renaming of the features from more classical software development methodologies. Also, actual agile implementations vary a lot from team to team and, sometimes, the right tool may be no tool at all. Even with this caveats, we believe that some tools have agile in mind and have managed to differentiate itself in that aspect.
Zephyr is a relative newcomer and has made it agile-fitness a big selling point, and we have to agree. Other tools come close and we can hand honorable mentions to QA Symphony’s qTest Platform, SmartBear’s QAComplete, and Tricentis’ Tosca Suite.
One important consequence of the flexibility of the tool is that the more flexible, the easiest it is to link your test management with the results of automated tests, continuous integrations and other agile staples. Thus, flexibility is also an important factor to consider when adopting a test management tool in an agile environment.
Pricing of SaaS solutions
The price of test management tools has settled around 25 dollars per user per month. Examples are QA-Symphony (29$), Teststuff (27$), Testcollab(25$) and TestRail (25$). This prices usually goes down as you add more users to your monthly subscription. As an exception Enterprise Tester of Catch Software starts with only 10$ per user per month. While user subscription based pricing is the most common solution, is not the only one.
TestLodge is pricing its SaaS app according to the number of test plans needed (test plans are composed of test cases). The recommended subscription gets you 15 test plans for 29$ per month, around the same pricetag than the user based subscriptions.
Smartbear has decided for a different model. They will host a SaaS instance of QAComplete for 359 euros per user per year, which results in a cost of 30 euros per user per month, slightly higher than the rest of the pack. While being higher, it is not highest one. Zephyr solutions start at 96$ per user per month.
Finally, bigger players (IBM, QMetry, Seapine Software and Tricentis), and probably more expensive as well, will not give you a price straight and require you to go through a demo session.
The SaaS market for Test Management is getting crowded, and choosing one solution mostly depends on trying a lot of them before setting for one. For that matter, having a lower price barrier can help to give the first step. However, more expensive tools usually come with more powerful integrated test automation features, which, once configured, can save you time and money. You can, as well, use different test automation tools and later integrate them in the test management tool (flexibility!).