Data Processing Optimisation for True Digital

Vodworks optimised True Digital's COVID-19 data processing system, reducing location data processing time from over 15 hours to under 10 minutes. The project processed 5 trillion data points across 30 million customers and cut infrastructure costs by 50%, helping Thailand's government improve COVID-19 policy decisions.

Challenge

True Digital needed to predict the spread of COVID-19 using location data from 30+ million customers, but their existing GCP and MemCache architecture took over 15 hours to process the data — far too slow to provide timely insights for government health policy. The sheer volume of data (5 trillion data points) posed a major technical challenge in terms of speed, accuracy, and reliability.

Solution

Vodworks data engineers and solution architects redesigned and re-implemented the entire data architecture. They replaced the GCP and MemCache setup with a new stack using BigQuery, Google Cloud Storage (GCS), and Kafka, enabling massively parallel processing of the location data. The new architecture reduced processing time from over 15 hours to under 10 minutes while maintaining accuracy and reliability across 5 trillion data points.

Results

Location data processing time reduced from 15+ hours to under 10 minutes. Infrastructure costs reduced by 50%. 5 trillion data points successfully optimised. Hiring process optimised to 2 weeks. The project directly improved the quality of COVID-19 policies and interventions for the Thai government.

Vodworks' work helped improve the quality of COVID policies and interventions. The team delivered everything on time and provided excellent project governance and updates. Very resilient team, motivated to tackle substantial technology challenges.

Bernd Vindevogel, Former Chief Analytics Officer, True Digital Group