Senior ML Infrastructure Engineer (DevOps)

About the role

We are seeking a Senior ML Infrastructure Engineer who loves running production systems at scale. You will partner closely with applied scientists, taking their models from prototype to reliable, real‑time services serving millions of customers.

Key responsibilities

• Maintain real‑time inference services, respond to alerts, debug live incidents, perform rollbacks and write post‑mortems as part of a 24x7 on‑call rotation.
• Design, build, and improve AWS infrastructure using IaC, CI/CD pipelines, Kubernetes, Docker, autoscaling, monitoring, load testing and cost‑control mechanisms.
• Manage the GPU fleet: driver and CUDA upgrades, node provisioning, debugging and capacity planning.
• Automate user access, SSH credentials, service accounts and developer environments for the science team.
• Run streaming data pipelines, feed an online feature store and serve low‑latency endpoints.
• Orchestrate batch workloads with Airflow or Databricks Workflows.
• Collaborate with scientists to productionise prototypes and ensure ongoing health.
• Write and review production‑grade Python code, including validation of AI‑generated code.

Required profile

• Proven 24x7 on‑call experience with high‑load online services and hands‑on incident resolution.
• Strong Linux systems expertise with practical Kubernetes, Docker and IaC experience.
• Solid Python development skills, comfortable with Git and CI/CD workflows.
• Experience with workflow orchestration tools such as Apache Airflow or Databricks Workflows.
• English proficiency at B2 level or higher.

Required skills

• Linux
• Kubernetes
• Docker
• Infrastructure as Code (e.g., Terraform, CloudFormation)
• AWS services
• Python
• Git
• CI/CD pipelines
• Apache Airflow
• Databricks Workflows
• NVIDIA drivers & CUDA
• GPU fleet management

What we offer

• Remote work flexibility with optional access to offices in Astana, Almaty or Karaganda.
• Opportunities to work abroad up to two months per year.
• Leadership development, career advising and well‑being programs.
• Certification support for GCP, Azure and AWS.
• Unlimited access to EPAM’s internal learning platform and free English classes.