Modern DevOps & Infrastructure As Code Modernization Achieving 86% Cost Optimization
We partnered with Hello Chef to modernize their machine learning infrastructure. We introduced MLflow for streamlined model tracking and comparison, enhancing code quality through rigorous business logic tests. By establishing an agile workflow, we improved team collaboration and efficiency. This transformation empowered Hello Chef to scale their ML operations effectively, leading to more accurate predictions and optimized meal-kit deliveries.
Challenge
Hello Chef, a popular meal-kit delivery service reaching thousands of households across the UAE, faced several obstacles in their machine learning infrastructure:
⚠️ No model visibility or performance tracking for critical ML decision-making models, with all ML processes running only in production
⚠️ Production codebase cluttered with drafts, duplicates, and commented logic with inefficient training and inference mechanisms
⚠️ Team with deep research and business experience but limited skills in developing and deploying production-ready ML systems
Solution
🧪 Model Tracking Framework: Implemented MLflow on AWS using Terraform to organize multiple models, compare different approaches, and version production-ready models with clear performance metrics
🔧 Code Refactoring: Systematically improved source code by removing duplicates, fixing bugs, optimizing SQL queries, and addressing inefficient training and inference mechanisms through short, iterative development cycles
🔄 Infrastructure Modernization: Replaced building Docker images per model with a web server application using one endpoint per model, and introduced Flyway for database migrations
🤝 Knowledge Transfer: Established a cross-functional ML Engineering team with clear goals, implementing sprint-based workflows with daily standups, backlog refinement, and retrospectives to ensure sustainable growth
Results
✅ Created scalable infrastructure to support ML models with MLflow deployed on AWS using Terraform
✅ Made 8 models production-ready, reliable, and scalable with proper performance metrics
✅ Covered ~30% of the code with business logic validation tests (up from 0%)
✅ Consolidated a monorepo with complicated CI/CD workflow into an easy-to-extend repository
✅ Replaced per-model Docker images with a unified web server application (one endpoint per model)
✅ Introduced Flyway for database migrations, enabling easy verification of database changes
✅ Established a cross-functional ML Engineering team with clear goals and sprint-based workflow
✅ Empowered stakeholders with access to complete model history and decision logs
The transformation resulted in a team that can confidently prepare and deploy models, compare outcomes against business objectives, and debug code to identify and fix issues more quickly—all while operating independently with industry-standard best practices.
"Our culture of learning and continuous improvement resonates deeply with the approach of the DLabsAI team. We truly appreciate the extensive learning opportunities they provided. Even when initial challenges seemed daunting, their support helped us overcome these obstacles and successfully achieve nearly all our significant goals."
Hitesh Pachpor, Engineering Manager at Hello Chef
Project Details
2024
