Senior ML Infrastructure Engineer (DevOps)

About the role

We are seeking a Senior ML Infrastructure Engineer who loves running production systems at scale. You will partner closely with applied scientists, turning their models into reliable, real‑time services for millions of customers. The role offers full ownership of the end‑to‑end infrastructure, from deployment to operations, with a 24/7 on‑call rotation.

Key responsibilities

• Maintain real‑time inference services, handling alerts, live debugging, incident resolution, rollbacks and post‑mortems as part of a shared on‑call rotation.
• Design, build, and improve AWS infrastructure using IaC, CI/CD pipelines, Kubernetes, Docker, autoscaling, monitoring, load testing and cost‑control.
• Manage the GPU fleet: NVIDIA driver and CUDA upgrades, node provisioning, debugging and capacity planning.
• Automate user access, SSH credentials, service accounts and developer environments for the science team.
• Run real‑time data pipelines, stream ingestion into an online feature store, and serve low‑latency endpoints.
• Orchestrate batch jobs with Airflow or Databricks Workflows.
• Collaborate with scientists to productionise prototypes and keep them healthy.
• Write and review production‑grade Python code, including validation of AI‑generated code.

Required profile

• Proven 24/7 on‑call experience with high‑load online services and hands‑on incident resolution.
• Strong Linux systems expertise with practical Kubernetes, Docker and IaC experience.
• Solid Python development skills, version control with Git and CI/CD implementation.
• Experience with workflow orchestration tools such as Airflow or Databricks Workflows.
• Familiarity with Gen AI concepts and English proficiency at B2 level or higher.

Required skills

• AWS
• Infrastructure as Code (e.g., Terraform, CloudFormation)
• CI/CD pipelines
• Kubernetes
• Docker
• Python
• Git
• Apache Airflow
• Databricks Workflows
• NVIDIA drivers & CUDA
• GPU fleet management

What we offer

• Remote work flexibility with access to offices in Astana, Almaty or Karaganda.
• Opportunities to work abroad up to two months per year.
• Relocation possibilities across 55+ countries.
• Corporate and social events.
• Leadership development, career advising and well‑being programs.
• Certification support for GCP, Azure and AWS.
• Unlimited access to EPAM’s internal learning platform.
• Free English classes with certified teachers and discounts for local language courses.