Client background & business context

  • Client: Retail company with 21 bill EUR revenue in 2020 and over 53000 employees 
  • Project: Performance testing: from strategy to continuous volume testing, tuning, scaling, migration and risk assessment 

Challenges

  • Capacity limitation: the client’s solution platform needed initial gauging in real-world scenario conditions, in order to assess risks under normal daily load or during peak sales periods. Based on client’s existing measurements, the application was able to handle roughly 400 transactions per second (TPS), pointing at potential business risks.  
  • Undefined performance test strategy: performance testing revolved around a rarely conclusive end-to-end (E2E) approach, preventing early issue detection, by overlooking component, feature and endpoint level isolated assessment.  
  • Limited test platform coverage: limited test scripts customizations at country or brand level, request distribution, usage profile and load-sourcing patterns (mobile, web, app), as well as deprecated test-data script sourcing.  

 

Our approach

The e-commerce solution was experiencing critical performance and stability issues that needed to be addressed as soon as possible. It was high time for a huge architectural redesign, involving heavy refactoring effort coupled with incorporating new/other technology stack, more capable of handling the expected capacity.  

Setting up the performance testing strategy was the prio 0 aspect. While keeping a close contact with the client’s development, infrastructure and middle-ware team, we’ve setup, implemented and documented a long-term, sustainable approach for performance testing and expanding coverage in a prioritized manner. In doing so, we’ve provided our client with early detection mechanisms, accurate prediction strategies, invaluable load-model emulation and scalable cross-brand and cross-country test coverage, all vital assets in ultimately fulfilling the end-customers’ desire for the best possible shopping experience 

Technologies

  • JMeter 
  • Java 
  • C++ 
  • Groovy 
  • Jenkins 
  • Docker 
  • Git 
  • Kibana 
  • dynatrace 

Results

One of the greatest achievements our client has obtained is the fact that we've rooted our strategic partner’s developers to shift left with performance testing, by thinking at performance testing as the most important and last validation gate between feature and end-user. It’s safe to say this approach reshaped and amplified their overall inherent skills to a mass-user-centered perspective, speed-avid society. Here are but a few of the reaped benefits achieved through our work and close collaboration with the development teams:  

  • Capability Buffered Solution: during our collaboration we’ve helped visualize pain-points, component strain, scaling factors, balancing strategy issues and were able to deliver them, via comprehensive reports, one-by-one, to client teams in charge of fixing. Our contribution enabled the platform to go from 400 TPS to a whopping 9000 TPS measured during 2019 Black Friday preparation 
  • Full test coverage: Test scenarios and reporting mechanisms are in place, assessing 100% emulated user-behavior, with integrated real user monitoring (RUM) and application performance monitoring (APM) solutions.  
  • Extended customization: Multi-environment, multi-brand, multi-country with on-the-fly scaling to any new resource accommodated by the solution platform is now a reality, as is a custom utility framework that aggregates test-data from multiple sources, making it seamlessly available for all test scenarios.  
  • Open Source: At the heart of our test scenario strategy lays the ubiquity of test-script deployment and that’s precisely why we’re focusing on Open-Source driven solutions. We delivered full compatibility with small-scale debugging approaches and large-scale live assessment via in-house or commercial SaaS solutions.  

Related case studies

CMS PaaS to Bluemix IaaS Cloud Service Migration

Together with IBM and the client, we worked on an infrastr ...

24/7 Monitoring services for increased stability and secur ...

End-to-end 24/7 Monitoring services for a German lighting ...

PEGA monitoring for optimized and improved platform manage ...

The PEGA platform includes several applications for the cr ...

München

Mies-van-der-Rohe-Strasse 6, c/o Design Offices
80807 München, Germany
E-mail: hello (at) accesa.eu

Cluj-Napoca

Constanta 12, Platinia
400158 Cluj-Napoca, Romania
E-mail: hello (at) accesa.eu

Oradea

Cetatii Square 1, Oradea Plaza
410520 Oradea, Romania
E-mail: hello (at) accesa.eu