Thoughtful Performance Testing for Retail

Predictability, velocity, validation and quality consistency at the speed and scale of today’s dynamic market environment: these are some of the impactful benefits of performance testing for Retail, and they have proved essential in the far-reaching shift in online consumer behaviour. Every Retail enterprise heavily dreams of the fast and reliable customer experience that will propel their business into the next level of profitability and long-term continuity.

Yet customer expectations are higher than ever – some say “slow is the new downtime” -, traffic spikes are a constant and annoying danger, customer databases are more geographically distributed than in any other industry, the TPS numbers are never satisfactory, and the list of issues can rapidly transform the above-mentioned dream into a nightmare.

Performance Testing can be considered the most important and last validation gate between feature and end-user. Our Performance Testing teams have seen just how important their work is for Retail clients who build the entire business on a mass-user-centred-perspective. The probable bottlenecks in scalability and stability are anywhere. Every online transaction depends on tens of different technologies, customized API services, plugins, 3rd-party integrations from internal processes to the cloud. Speaking of the Cloud, Retailers who decide to migrate to a cloud or hybrid cloud environment need to ensure that performance does not regress.  

So, yes, performance testing is important (spike testing, endurance testing, volume testing, scalability testing, to mention just a few of the subsets). As our colleague, Andrei Marinoiu, mentioned in his article about the future of Testing, “I dare to say that the future of testing will be like pit stops in Formula 1. The ones who will execute the fastest testing and will first reach the market will have the biggest revenue.”

Fast and thoughtful performance testing for a lead Retailer in consumer electronics

One of the most complex performance testing projects we delivered was in support of a European renowned Retailer, who had reached a point in which their e-commerce platform was experiencing critical stability issues. It goes without saying that a fast and thoughtful delivery approach to meeting their need was essential.

In short, our client was dealing with capacity limitation, poor performance test strategy and limited test platform coverage – there were serious issues with limited test scripts customizations, usage profiling and load-sourcing patterns for mobile, app and web. We immediately delved into rethinking their performance testing strategy for attaining the best possible shopping experience. What we intended was to create a system for early detection mechanisms, accurate predictions, scalable cross-brand and cross-country test coverage, load-model emulations.

A key component of the success of our project was our continuous commitment to open collaboration with our client’s teams in development, infrastructure and middleware. Together, we managed to ensure a reliable approach to testing and expanding coverage.

By working with a class-A pack of technologies such as JMeter, Java, Groovy, Jenkins, Docker, Kibana and Dynatrace, we were able to offer our client a set of indisputable benefits for their performance testing environment, as mentioned in our Case Studies:

• Capability Buffered Solution: during our collaboration we’ve helped visualize pain-points, component strain, scaling factors, balancing strategy issues and were able to deliver them, via comprehensive reports, one-by-one, to client teams in charge of fixing. Our contribution enabled the platform to go from 400 TPS to a whopping 9000 TPS measured during 2019 Black Friday preparation.  
• Full test coverage: Test scenarios and reporting mechanisms are in place, assessing 100% emulated user-behaviour, with integrated real user monitoring (RUM) and application performance monitoring (APM) solutions.  
• Extended customization: Multi-environment, multi-brand, multi-country with on-the-fly scaling to any new resource accommodated by the solution platform is now a reality, as is a custom utility framework that aggregates test-data from multiple sources, making it seamlessly available for all test scenarios.  
• Open Source: At the heart of our test scenario strategy lays the ubiquity of test-script deployment and that’s precisely why we’re focusing on Open-Source driven solutions. We delivered full compatibility with small-scale debugging approaches and large-scale live assessment via in-house or commercial SaaS solutions.