This is the second post on the SQA tools series. Part I is already published. In this series we are focusing on SaaS tools, and this post is about Application Life-cycle Management (ALM) tools that can be used by monthly or yearly fees. In our frenzied Google searches we may have left some ALM’s behind, but we have quite a few to show you. From big ol’ guys (HP, MS) to smaller guys (Practitest, Countersoft) to newer entrants focused on Agile Development (VersionOne, Rally). We are going to present them alphabetically, and hopefully, after reading this post you can have a better idea which tool is better for you, as it is impossible for us to recommend you one
Gemini is the main product of the Countersoft company. Gemini is designed from the scratch to be SaaS, and it shows that in the clean interface. The main goal of Gemini is to combine the software development aspect with the customer management side of the software. To keep track of the software development it allows to link to existing software repositories and manage and run tests. The novel part is on the customer management side. Gemini includes a modules for all service-desk related aspects.
Its reporting features are not very advanced, but cover the most important cases. The metrics visualization widget lets the user select the metrics to be visualized in a simple interface.
Gemini has an open API that allows to extend and integrate the platform with other platforms. Countersoft has extensive integrations to Microsoft solutions (Visual Studio, Team Foundation Server,…) and other communication platforms.
HP’s ALM suite is the elephant in the room and it features a complete set of tools to manage the application management. It is already in version 12.0, which was released last year. The new version brought greater browser support compatibility, so now clients can connect with Safari and Firefox, rather than only just Internet Explorer-now you can use HP ALM on a Mac!
One of the main components of the ALM is the Quality Center, which also got a big polish in the version 12.0. One best features introduced in the version 11 were the Business Views. Business Views allow for creating visualizations of the metrics and other data via a GUI, a great improvement from previous versions when SQL was needed to achieve similar results. In any case, the resulting graphs are 2-D visualizations of 2 chosen variables.
HP claims that its SaaS solution can cut operational costs by 40%, though, as expected, you get to loose some of the customization benefits.
SpiraTeam is an example of a tool that started as Test Management tool (SpiraTest) and has evolved to a full-fledged ALM. Requirements, tests, defects, builds, and releases can be managed with SpiraTeam, in addition to project management. One lacking feature, at the moment, is the lack of connection to source code repository, which could help track the defects better.
The project management component contains a dashboard that can visualize the projects’ health, allowing to group projects by client, technology, and other. The dashboard visualizes metrics through bars and 2-D graphs. The dashboard collects all the visualizations in one place and creates a bit chaotic layou, with many places demanding the user attention.
The SaaS option starts with 10 dollars per user per month. However, the price goes up if you want support for 3 users to 80 dollars per month (about 27 dollars per user per month), and goes down from there.
Microsoft Visual Studio Online
Team Foundation Server (TFS) is the center of Microsoft’s efforts to support software developing companies. However, it has recently (2013) released Visual Studio Online. It integrates seamlessly with Microsoft Visual Studio. Visual Studio Online is the cloud version of TFS, and provides hosting, testing, and building of the software source.
Visual Studio Online is very much developer centric, as it lacks features for project management and reporting. Requirements, defects, and manual test results are visualized nicely with a great editor with plenty of options to define graph type, rows and columns. Automated tests are put together with automated builds, which happily run in MS’s cloud, but the reporting of such test is more limited.
To get the whole set of benefits of VSO ALM features, advanced users should pay 80 dollars per month. Interestingly, if the users are not interested in the test management, i.e. developers, 60 euros per month will give them the most important features and a licence to Visual Studio Professional.
Polarion produces three solutions, the Requirements Manager, the Test Management, and the ALM, that contains the previous two and adds support for better reporting, configuration, and build and release management. Polarion is trusted by very big companies that are happy to pay its price to achieve quality certifications, most of the time required by the industry (e.g., car manufacturing)
The Polarion tools ease the acceptance by users through its integration with Word and other Office products. Its visualization and reporting capabilities will be improved in the coming version (2015) allowing non-programmers to design reports by adding widgets to them. This widgets can accept parameters for the users to adapt them.
The Polarion ALM SaaS comes for 1788 dollars per user per month. Luckily, not everyone has to get the whole ALM package, meant for managers, and normal users could do with the PRO package, which costs 468 dollars per user per month.
Practitest sells a Test Management tool that almost passes as an ALM. It is not as complete as the other ones in the list but they have added Requirement and Defect Management components to it.
The Test Management component has added support for automated test. External tools can call the API when they have completed the test run and submit the results. A bot or daemon calls the tool when a new script is ready. The solution is missing a Build Management tool that could ease the script writing.
The dashboard follow the customization possibilities of the competence. Practitest allows for seven tabs in the dashboard. And each tab can get customized visualization of metrics as tables, pie-charts, 2-D graphs,…
The software comes in three SaaS versions. The Developer version is 15 dollars per user per month, and the most expensive one, the Enterprise version is 45 dollars per user per month. The main features of the Enterprise version is that it can manage test and requirements, while the Developer version is can only see them. Also the Enterprise version allows access to the API and the different integrations (Jira, automation, and version control).
Rally itself is not an ALM. Rally is created with Agile development in mind, and it makes for a great management tool for Agile companies. Rally deals mostly with projects and teams. Answering questions like is the project on time, or is the team performing, are Rally biggest advantages.
To act as the center of software development, Rally provides deep integration with the most used tools in the industry. The plug-ins are either produced by them or a third party. For example, Rally can integrate with other Test Management suites, like Tosca TestSuite in case the QA team is not using Rally.
Rally has several dashboards for different aspects of the project: iteration, build, release,… These dashboards are considered “apps” that can be added to different pages. The dashboards are pre-defined by Rally and do not have much flexibility.
When Rally released the Unlimited edition, the cost was 49 dollars per user per month. The Unlimited edition contains the most useful add-ons for SQA. For current prices of the Unlimited and their other editions you have to contact them.
VersionOne is a direct competitor to Rally. It is first and foremost an agile project management platform. As Rally it can deal comfortably with teams and projects. In contrast to Rally, VersionOne users can tweak the platform to different varieties of Agile methodologies (Scrum, XP, Kanban,…).
VersionOne has made an effort to create new and interesting visualizations for Agile projects. Specially the heat map, in which cells are color-coded and sized according to different agile metrics (e.g., business value, risk,…). Still, when it comes to health of the project, and the code quality, metrics are lacking and only a couple of visualizations are available to inspect the trends and overall status of the tests.
VersionOne is more open to their pricing than Rally. The Team edition is free but limited to one team and a single project. The price increases up to the Ultimate Edition, which goes for 39 dollars per user per month. The Ultimate Edition is the one that provides the nice visualizations as well as the Executive Dashboards
As you can see, there is a lot to choose from. However, this big solutions come only once you have already established your software development operations. Then, moving your workflow to one of the solutions can be costly in time and money. Gemini’s approach is quite interesting as it is lightweight and well thought of. Visual Studio Online can leverage faster with the developing team, arguably the first team you will hire.
When it comes to managing projects and communicating results good reporting and visualization are quite important. We haven’t seen yet the right amount of intelligence, as the dashboard gallery above shows, and we hope at U-Qasar will bring some sense to life-cycle management. Sign up for the newsletter to be informed.
You can find other ALM comparisons done by TechExcel (also an ALM vendor), and Contribyte (a software development and training company) that can complement the analysis done here. We will keep updating the list presented here as we review more tools, currently missing are Jira and Jama. Let us know if you know of another up and coming SaaS ALM tool to review.