• Test Management eats itself (Part III of SQA Tool Series)

    by  • February 26, 2015 • Market trends, Quality assurance market, Quality assurance trends, Series of SaaS Tools for Software Development, Software Testing & QA, Tools • 1 Comment

    The previous post on this SQA tools series delved in Application Life-Cycle Management tools. This post will review the next link in the SQA chain, the test management tools. As in the rest of these series, we are only looking at those options that are available as SaaS.

    This time the analysis will mostly be based on two main characteristics, agile fit, and flexibility. By agile-fit we try to distinguish those test management tools that have a better layout for agile development. When we talk about flexibility, we refer to the facilities the tools give to export or import data from the tests, whether their offer an API to create custom extensions to the platforms.

    The importance of these two features is acknowledged by most, if not all, of the vendors. As such, most of the tools tick the agile-fit and flexibility check boxes. Without further ado, we will briefly analyse the tools we have found. The table shows a quick outlook on how the tools fulfill the expectations in the two categories. The more ✔ the better, and one ✘ means that the feature is not a big selling point for that concrete solution.

    Test Management Tools evaluated by agile fitness and flexibility

    Test Management Tools evaluated by agile fitness and flexibility

     Analysis of the Test Management Tools

    The tools can be roughly classified into two big categories: those that belong to a bigger suite of software development tools; and those that are independent. In the first group we can find tools from IBM, Seapine Software, Smartbear, Tricentis. In the second groups we can find both robust and nimble solutions from Teststuff, TestCollab, TestLodge, TestRail, TestRun, and Zephyr.

    This group distinction shapes how flexible are the test management tools. Those tools that depend on other vendors have a design so that it facilitates integrating the test management with project management, requirements management, source code repositories, etc. Worth mentioning in this category are tools like Catch Software’s Enterprise Tester, QMetry, TestRail, and Zephyr. On the other hand, tools that are already integrated do not have the same facilities, like Seapine’s TestTrack. However, we can still find products in this category that excel in flexibility: Tricentis’ Tosca Suite. The outlier when it comes to flexibility is Test Run, a simple solution that just does Test Management and does not mess with anything else.

    The agile-fit evaluation is harder to make, as most of the times, when it comes to Test Management; agile ready solutions basically mean a renaming of the features from more classical software development methodologies. Also, actual agile implementations vary a lot from team to team and, sometimes, the right tool may be no tool at all. Even with this caveats, we believe that some tools have agile in mind and have managed to differentiate itself in that aspect.

    Zephyr is a relative newcomer and has made it agile-fitness a big selling point, and we have to agree. Other tools come close and we can hand honorable mentions to QA Symphony’s qTest Platform, SmartBear’s QAComplete, and Tricentis’ Tosca Suite.

    One important consequence of the flexibility of the tool is that the more flexible, the easiest it is to link your test management with the results of automated tests, continuous integrations and other agile staples. Thus, flexibility is also an important factor to consider when adopting a test management tool in an agile environment.

    Pricing of SaaS solutions

    The price of test management tools has settled around 25 dollars per user per month. Examples are QA-Symphony (29$), Teststuff (27$), Testcollab(25$) and TestRail (25$). This prices usually goes down as you add more users to your monthly subscription. As an exception Enterprise Tester of Catch Software starts with only 10$ per user per month. While user subscription based pricing is the most common solution, is not the only one.

    TestLodge is pricing its SaaS app according to the number of test plans needed (test plans are composed of test cases). The recommended subscription gets you 15 test plans for 29$ per month, around the same pricetag than the user based subscriptions.

    Smartbear has decided for a different model. They will host a SaaS instance of QAComplete for 359 euros per user per year, which results in a cost of 30 euros per user per month, slightly higher than the rest of the pack. While being higher, it is not highest one. Zephyr solutions start at 96$ per user per month.

    Finally, bigger players (IBM, QMetry, Seapine Software and Tricentis), and probably more expensive as well, will not give you a price straight and require you to go through a demo session.

    Summary

    The SaaS market for Test Management is getting crowded, and choosing one solution mostly depends on trying a lot of them before setting for one. For that matter, having a lower price barrier can help to give the first step. However, more expensive tools usually come with more powerful integrated test automation features, which, once configured, can save you time and money. You can, as well, use different test automation tools and later integrate them in the test management tool (flexibility!).


    Interested what U-Qasar can do for your Agile organization? join the mailing list for updates in the U-Qasar platform and methodology.

    First Name*

    email*

    At U-Qasar, we respect your privacy and will not share your email with anyone, period.

    About

    We are a strategy consultancy. We work with companies & institutions worldwide on strategy crafting, internationalisation & business modelling.

    http://www.vaibmu.com