Modernizing ETL at Scale with AI Coworkers
OBZ deployed a GenAI-powered multi-agent framework to automate legacy ETL pipeline migration, delivering a 50% reduction in migration time with 60% of pipelines transitioned without human intervention.
Powered by Amazon Bedrock (Claude Sonnet 3.5)
50%
reduction in end-to-end migration timelines
60%
of ETL jobs migrated autonomously with minimal review
1,000+
ETL jobs per month, repeatable and scalable
Overview
About the Customer
A leading financial services enterprise was undergoing a major data modernization initiative, shifting its legacy ETL processes to a cloud-native stack. With thousands of pipelines to migrate and limited engineering bandwidth, the client sought a faster, more intelligent way to transition to modern data platforms without compromising logic integrity or compliance.
The Challenge
Business Challenges
- The client's legacy ETL environment lacked scalability and was becoming increasingly difficult to maintain
- Migrating to AWS-native tools like Glue and Spark required rewriting thousands of ETL scripts, an effort that was manual, slow, and error-prone
- Traditional approaches demanded large engineering teams, increasing cost, duration, and risk of logic drift
The Solution
Solution Approach
OBZ deployed a GenAI-powered migration crew, an agentic team of AI coworkers purpose-built for ETL modernization. Each agent had a defined role (e.g., developer, QA, lead agent) and collaborated to automate the conversion process end-to-end. This was powered by OBZ's GenAI Studio, hosted within the client's AWS environment, ensuring full data security and scalability. The studio enabled agents to parse job metadata, rewrite code using NLP, validate outputs, and deploy to target platforms like AWS Glue and Spark, all within an auditable, role-governed workflow.
AWS Services Used
Built on AWS
Amazon Bedrock (Claude Sonnet 3.5) – GenAI foundation model for intelligent code conversion and agent orchestration
Amazon EventBridge – Event-driven workflow orchestration for asynchronous task management
Amazon Aurora – Backend database for storing Langfuse prompts and system metadata
AWS Lambda – Custom authorizers, Jira event handlers, and supporting functions
AWS Cross-Account Access – Secure access to Bedrock and other resources across OBZ-managed AWS environments
Outcome
Business Outcomes
- Cut migration timelines by 50%
- 60% of pipelines migrated autonomously with minimal review
- Reduced engineering and QA workload significantly
- Enabled repeatable, scalable migration across 1,000+ ETL jobs/month
- Improved consistency, reliability, and logic integrity at scale
About OneByZero
OneByZero is a frontier Systems Integrator specialising in AI Coworker design, build, and deployment for regulated enterprises across financial services, telecommunications, and retail. We operate across ASEAN, India, ANZ, and Japan, with a presence in the United States, combining domain expertise in regulated industries with deep AI engineering capability.
HQ Singapore · Tech HQ USA · Australia · India · Indonesia · Malaysia · Philippines · Thailand · Vietnam
