Client background & business context

  • Client profile: largest European electronics retailer 
  • Project: performance testing implementation 

Challenges

  • Capacity limitation: the client’s solution platform needed initial gauging in real-world scenario conditions. This would allow to better assess risks under a normal daily load or during peak sales periods. Based on the client’s existing measurements, the application was able to handle 400 transactions per second (TPS). As the company grew and diversified its needs, this number of transactions represented a potential risk.  
  • Undefined performance test strategy: performance testing revolved around a rarely conclusive end-to-end (E2E) approach, preventing early issue detection, by overlooking components, features, and endpoint-level isolated assessment.  
  • Limited test platform coverage: limited test scripts customisations at country or brand level. Limitations were also present in terms of request distribution, usage profile and load-sourcing patterns (mobile, web, app). 
  • Deprecated test-data script sourcing that were put in place for previous testing sessions 

Our approach

Setting up the performance testing strategy was our prio 0 aspect. While keeping close contact with the client’s development, infrastructure, and middle-ware team, we’ve started implementing and documenting a long-term, sustainable approach for performance testing and expanding coverage, in a prioritised manner. In doing so, we’ve provided our client with an early detection mechanism, accurate prediction strategies, invaluable load-model emulation, and scalable cross-brand and cross-country test coverage. All these were vital assets in ultimately fulfilling the end-customers’ desire for a customer-experience oriented shopping experience

Technologies

  • Apache JMeter and HP LoadRunner to replicate user load on the SUT
  • BlazeMeter and Octoperf 3rd party cloud provider
  • Multi-zone load distribution using AWS, Scaleway, DigitalOcean
  • Grafana , DynaTrace, Munin, AppDynamics, NewRelic, Kibana as monitoring tools (APM)
  • Jenkins, Taurus for CI / CD integration of Performance Validation in the pipeline
  • JAVA, C, Groovy, BeanShell as scripting languages
  • Containerised load generating infrastructure deployed on client premise

Results

One of the greatest achievements for this project was that we collaborated with our strategic partner’s developers to make the shift towards performance testing. We did it by changing the mindset and by looking at performance testing as the most important and final validation gate between the features and the end-user. This approach reshaped and amplified their overall inherent skills to a mass-user-centred perspective. Some of the benefits achieved through our work and close collaboration with the development teams were: 

  • Capability Buffered Solution: during our collaboration we’ve helped visualise pain-points, component strain, scaling factors, and balancing strategy issues. After this endeavour, we were able to deliver to the company comprehensive reports of issues to be one-by-one addressed and fixed. Our contribution enabled the platform to go from 400 TPS (in 2014) to 9000 TPS (measured during 2018’s Black Friday preparation). 
  • Full test coverage: test scenarios and reporting mechanisms were put in place. These assess 100% emulated user-behaviour, with integrated real user monitoring (RUM) and application performance monitoring (APM) solutions, allowing the company to act accordingly. 
  • Extended customisation: multi-environment, multi-brand, multi-country with on-the-fly scaling to any new resource accommodated by the solution platform is now a reality. We also implemented a custom utility framework that aggregates test-data from multiple sources, making it seamlessly available for all test scenarios. 
  • OpenSource: at the heart of our test scenario strategy lays the ubiquity of test-script deployment. That’s why we’re focusing on OpenSource-driven solutions. We delivered full compatibility with small-scale debugging approaches and large-scale live assessment via in-house or commercial SaaS solutions. 

Related case studies

API first integration for enhanced availability and reserv ...

Receiving notifications about any possible breaks in the ...

OMS integration for increased speed, consistency, and qual ...

Undergoing the rollout of an integrated OMS, requires comp ...

TIBCO integration for enhanced custom OMS continuity

Our specialists managed to build new integration component ...

Say hello

München

Mies-van-der-Rohe-Strasse 6, c/o Design Offices
80807 München, Germany
Phone: +49 89 2154 851 15

Zürich

Wallisellen, Zwirnereistr. 22
8304 Zürich, Switzerland
Phone: +41 44 830 92 30

Cluj-Napoca

Constanta 12, Platinia
400158 Cluj-Napoca, Romania
Phone: +40 364 115 115

Oradea

Cetatii Square 1, Oradea Plaza
410520 Oradea, Romania
Phone: +40 364 115 115

Whether you’re interested in our services or you would like to learn more about our company, we are happy to provide you with the information you need.

I agree to the Privacy Policy.
I consent to the processing of my personal information.