Performance improvement & scalability of a people analytics platform

Background

A leading HR analytics platform with years of experience in the analytics offers customers analytical insights in their HR operations. This helped the end-customers make correct data backed HR decisions.

The platform was successful and used by many small and mid sized organizations across North America but had challenges when it came to adoption by large corporations. The platform could not scale up to operate on the large datasets needed by big corporations. Long data fetch and page load times were limiting the platform’s sales potential and the company’s growth.

The company’s chief technology officer (CTO) was tasked to address the issue to be able to serve larger customers which could bring in significantly more revenue.

This called for re-engineering of quite a few key components including data storage and retrieval ones. The internal team was not equipped in terms of knowledge, experience and ability to re-engineer the components to handle the necessary volumes. The CTO proposed to outsource the reengineering effort to a competent team in order to achieve the required result. After overcoming some initial resistance from the internal teams, the top management agreed to move forward with the CTO’s proposal.

The company approached Zogato for re-engineering the product for scalability and performance.

Zogato Solution

Zogato benchmarked the product’s throughput and response times on queries and workflows to identify bottlenecks. Zogato also analyzed the business logic layer and the UI for any possible performance bottlenecks there. The data access layer used a sophisticated object-relational mapping (ORM) tool to fetch the analytics data. The ORM had enabled fast development in the initial days of the platform but had now become a technical debt. The ORM was not suited to handle the complex analytics queries being fired on large data sets. Zogato replaced the layer with optimized database queries, making the query responses an order of magnitude faster than before.

Zogato also looked at the data being fetched by the front end. In many cases large amounts of redundant data was being pulled by the front end. Effective use of pagination, lazy loading and some restructuring of the front end ensured optimal data retrieval.

With the restructuring the overall number of lines of code was reduced by about 50-60%. Thus making the code more maintainable. The performance of some modules of the product improved by over 500 times.

Benefits

The significant improvements in the product’s performance opened up new and attractive market segments consisting of larger and more complex organizations. The company could demo, pilot, and sell the data- intensive product to the new prospect base, which had a scale of operations that was 10-30 times that of the existing customers.

  • The improvements in product performance decreased the hardware configuration requirement at the backend, reducing the hosting costs significantly.
  • The ease of demoing and piloting with large data sets led to shorter sales cycles, resulting in significant improvements in its top line and bottom line.
  • With performance issues out of the way, the company focused on the critical features during sales discussions, resulting in a more motivated sales team.
Tools and Technologies
  • MongoDB
  • NodeJS,
  • RESTful API
  • JSON, XML, CSV
  • ReactJS