Senior Data Engineer
Atmora Tech
- Home
- Careers
- Senior Data Engineer
- Jodhpur, India
Vantara Metals runs six plants with OPC-UA historians, two ERP instances and a decade of spreadsheets. We are consolidating that into a lakehouse on Delta Lake, with about 420 dbt models feeding plant yield, energy cost per tonne and inventory ageing. The current nightly batch takes 3 hours 10 minutes and needs to become a 15-minute micro-batch without doubling the compute bill.
- Full-time — on site
- 5-8 years building production data pipelines, including at least one platform you migrated while it was in use
- Jodhpur, India
The work
What you will do
- Build ingestion from OPC-UA historians and ERP change-data-capture streams, handling clock skew and out-of-order events from sites with intermittent connectivity.
- Restructure the dbt project into clearly layered staging, intermediate and mart models, with tests on every column a report depends on.
- Cut the critical path from 3 hours 10 minutes to a 15-minute micro-batch through incremental models and partition pruning.
- Define and enforce data contracts between the ingestion layer and the analytics team, including a deprecation process for breaking changes.
- Own cost: track spend per pipeline and keep the monthly warehouse bill within an agreed ceiling.
What we look for
- Strong SQL — window functions, incremental merge strategies, and knowing why your query scanned 400 GB when it should have scanned 4.
- Spark or an equivalent distributed engine in production, including how you diagnosed a skewed join.
- Python for orchestration and transformation, with tests that run in CI.
- Dimensional modelling that survives contact with a client changing their reporting hierarchy mid-project.
- Direct experience of a late-arriving or corrected record breaking a downstream figure, and what you built so it did not happen twice.
Other openings
- Bengaluru, India
- Full-time — hybrid, three days a week on site
You will work on the settlement and ledger services behind Meridian Exchange's B2B marketplace: roughly 40 million ledger rows a month, Java 21 on Spring Boot, Postgres 16, Kafka for event fan-out. The interesting part is not new endpoints, it is correctness under load — idempotent writes, exactly-once effects on top of at-least-once delivery, and holding p99 write latency under 250 ms as the ledger grows.
- Jodhpur, India
- Full-time — on site
Atmora runs more than 40 client workloads across EKS and AKS, from a nine-node cluster carrying Kestrel Industrial's shop-floor telemetry to a two-region active/passive setup for Corvus Logistics. You will own the internal platform those delivery teams deploy onto: the Terraform module library, the Argo CD app-of-apps layout, and the paved path that takes a service from merge to production in under 12 minutes.
- London, UK
- Full-time — London office, with four to six client-site days a month across the UK and EU
Most of our UK work is legacy modernisation for regulated businesses: Cordage Bank moving a 22-year-old core banking periphery off a shared Oracle schema, Aldridge Risk replacing an overnight batch that had grown to 9 hours and 40 minutes against a 6-hour window. You will run discovery, decide the migration shape, and stay accountable for it once delivery starts — not hand over a slide deck and leave.
- Home
- Careers
- Senior Data Engineer
- Jodhpur, India
Vantara Metals runs six plants with OPC-UA historians, two ERP instances and a decade of spreadsheets. We are consolidating that into a lakehouse on Delta Lake, with about 420 dbt models feeding plant yield, energy cost per tonne and inventory ageing. The current nightly batch takes 3 hours 10 minutes and needs to become a 15-minute micro-batch without doubling the compute bill.
- Full-time — on site
- 5-8 years building production data pipelines, including at least one platform you migrated while it was in use
- Jodhpur, India
The work
What you will do
- Build ingestion from OPC-UA historians and ERP change-data-capture streams, handling clock skew and out-of-order events from sites with intermittent connectivity.
- Restructure the dbt project into clearly layered staging, intermediate and mart models, with tests on every column a report depends on.
- Cut the critical path from 3 hours 10 minutes to a 15-minute micro-batch through incremental models and partition pruning.
- Define and enforce data contracts between the ingestion layer and the analytics team, including a deprecation process for breaking changes.
- Own cost: track spend per pipeline and keep the monthly warehouse bill within an agreed ceiling.
What we look for
- Strong SQL — window functions, incremental merge strategies, and knowing why your query scanned 400 GB when it should have scanned 4.
- Spark or an equivalent distributed engine in production, including how you diagnosed a skewed join.
- Python for orchestration and transformation, with tests that run in CI.
- Dimensional modelling that survives contact with a client changing their reporting hierarchy mid-project.
- Direct experience of a late-arriving or corrected record breaking a downstream figure, and what you built so it did not happen twice.
Other openings
- Bengaluru, India
- Full-time — hybrid, three days a week on site
You will work on the settlement and ledger services behind Meridian Exchange's B2B marketplace: roughly 40 million ledger rows a month, Java 21 on Spring Boot, Postgres 16, Kafka for event fan-out. The interesting part is not new endpoints, it is correctness under load — idempotent writes, exactly-once effects on top of at-least-once delivery, and holding p99 write latency under 250 ms as the ledger grows.
- Jodhpur, India
- Full-time — on site
Atmora runs more than 40 client workloads across EKS and AKS, from a nine-node cluster carrying Kestrel Industrial's shop-floor telemetry to a two-region active/passive setup for Corvus Logistics. You will own the internal platform those delivery teams deploy onto: the Terraform module library, the Argo CD app-of-apps layout, and the paved path that takes a service from merge to production in under 12 minutes.
- London, UK
- Full-time — London office, with four to six client-site days a month across the UK and EU
Most of our UK work is legacy modernisation for regulated businesses: Cordage Bank moving a 22-year-old core banking periphery off a shared Oracle schema, Aldridge Risk replacing an overnight batch that had grown to 9 hours and 40 minutes against a 6-hour window. You will run discovery, decide the migration shape, and stay accountable for it once delivery starts — not hand over a slide deck and leave.