Over the last year, the team was involved in every software development phase, starting with developing the Data Lake infrastructure, ETL jobs, data processing scripts and jobs as well as development of the business-facing reports, data analysis scripts and ending with developing the statistical and machine learning models for improving the way our client’s devices work.
To create a clear vision about our client’s business, every new member attends several knowledge sharing sessions and new members have a mentor/buddy assigned. The sessions include different types of subjects, starting with general ones (client’s industry, Data Lake Infrastructure, goals), and ending with subjects which are related to the new member’s area of expertise.
Throughout the year the team has developed a strong connection with the client’s main stakeholders which allowed the team the ability and freedom to express their ideas for further developing the infrastructure and the processes. Most of the suggestions have already been implemented and we were able to reduce the ETL and data processing time of all of our incoming data by more than 5 times.
We managed to provide a client-centric software solution by:
- Having a proactive attitude and taking care of our client’s Data Lake and ETL jobs in order to always provide him with the ability to analyze his data, monitoring jobs and coming up with proposals for newer solutions that could reduce the time it takes for the data to become available in the Data Lake
- Understanding the bigger picture and the actual need of the client, not only the technical problem that needs solving. We keep in mind the big picture and the actual business problem the client tries to solve and we come up with the best solution for his problem, sometimes with improvements in areas where he did not explicitly request.
- Implementing new techniques and upgrade the Data Lake infrastructure to reduce costs and increase the processing speed of the client’s data