How RevSpring Managed to Cut Regression Testing Time and Achieve Higher Quality Releases

RevSpring, a Michigan-based software company within Financial Services and Healthcare, recently reported having seen their regression testing time cut down by 75%. Finding time to adopt and achieve a high percentage of automated test cases can be challenging, regardless of how easy and powerful the automation tool in use could be. What was the key to RevSpring’s success?

Finding Time to Save Time

According to the 2020 State of Testing Report, 76% of respondents listed "coping with timeframes" as one of their testing team’s main challenges. A common reason why teams rely mainly on manual testing and overlook automation though, is lack of time. This brings us to quite a common conclusion: We don’t have time because we don’t automate.

Allocating time and resources to automation however has proven vital for the quality of the products we build. The report cited above continues by anonymously quoting QA engineers willing to share their experience.

“I officially implemented automated testing in early 2019 and have seen a 30% decrease in the number of post-release bugs reported when compared to the prior year.”

The shift in mentality, which Gartner outlines in their 2019 Magic Quadrant, going from “we need to automate” to “we need a lot more than just a test automation tool” is driving the whole industry into a direction of integrated suites that work best for the team instead of implementing single-vendor best-of-breed tools that cover isolated use cases.

Delivering an outstanding UI through a robust process that lets everyone on the team collaborate requires your test suite to include support for various technologies, testing practices, service-based architectures, machine-virtualization, remote scheduling, integration with DevOps toolchains, reporting and results visualization, along with solid capabilities to enable different team roles and cover hard-to-automate scenarios.

The Case of RevSpring

Similar goes the story of RevSpring. To accommodate an ever-growing diversity of payment options within an infinitely configurable healthcare payment services platform, RevSpring chose Test Studio to scale their test automation efforts. They achieved 80% faster time to market with significantly improved release quality and 100% coverage in payment methods, while protecting sensitive patient data in compliance with HIPPA standards.

The application under test is extremely configurable platform for medical billing supporting about 68 different ways of accepting payments. Critical to the ultimate success of this project was to ensure sensitive medical patients’ data is protected and in full compliance with HIPAA.

“To account for all different scenarios is a challenging task due to the nature of our platform. That's where Test Studio is very good as it helps us record one scenario and tweak it in the spot that needs to be adapted without recreating the entire test. It makes for rapid engineering progress allowing us to have 100% coverage in all the ways we provide payments, handle statements, set up payment plans etc. It's really allowed us to go to places that have been very difficult,” said Pete O'Grady, QA Manager at RevSpring.

The level of complexity such applications impose requires solid strategy on all levels, starting with the integration of automated testing into the delivery cycle to enable feedback earlier in the process and prevent bugs slipping into production. Previous experience with open source though, shows that the level of complexity when it comes to the automation suite is becoming unmanageable, especially in a fast-pace agile environment with frequent releases introducing constant changes to the UI.

Writing everything from scratch, relying on community help rather than on dedicated support and involving expensive development resources in the constant test suite maintenance accumulates long term cost that doesn’t account for the flexibility at scale they were looking to achieve.

“The automation that was provided that route versus what we have now—there's just no comparison. And we find things that would never have been caught before” noted O'Grady.

The Solution in Place

The easy integration with the Atlassian Suite and Jira and the flexibility provided by the Run-Time extensions and features allowed RevSpring to seamlessly incorporate testing into their continuous delivery. The ability to loop through data in a single test step using the data binding features, the intuitive test recording UI and the hybrid element find logic enabled the team to go deeper into their regression efforts, increasing code stability, improving release quality and freeing up time for focusing on critical work.

Crafting traditionally complicated but also expensive load and API tests for specific test cases accounted for the impressive 100% payment scenario coverage and enabled the rapid progress RevSpring was looking for.

"At first we did a week-long regression testing for each release cycle. There was a suite of around 40 stories which was hard to cover in the short time and would take four people five days to complete this. Thanks to Test Studio, we've been able to shorten that to a day and a half. And probably it'll get down to a day thanks to being able to automate almost anything with Test Studio," explained O'Grady.

Critical Success Factors

It won’t be wrong to say that one of the most common critical success factors in software delivery is cutting time to market. In end-user facing, sensitive data-processing applications, expensive errors and release delays caused by bugs are unacceptable.

While nice to have for some organizations, acquiring compliance capabilities turned critical for RevSpring. Thanks to Test Studio, they were able to secure an auditable QA process path for authorities allowing them to provide secure, compliant and future-proof payment services to their customers.

With one of the most intuitive visual test recording interfaces on the market and the industry's first patent-pending element find logic, Test Studio eliminates bottlenecks with regression testing, increases test coverage across UI, RESTful APIs, and load scenarios, and helps keeping the entire team productive.

Start your 30-day trial with free access to support and online training.

Try Now