Lead Data Quality Engineer

About the role

We are seeking a Lead Data Quality Engineer to drive automated testing across data pipelines, APIs, and platform components within our Azure‑based ecosystem. You will work closely with Data Engineers, Platform Engineers, and DevOps teams to ensure data reliability and performance.

Key responsibilities

• Design, build, and maintain automated testing solutions for data pipelines, APIs, and platform services.
• Validate data quality, transformations, and pipeline outputs using SQL and/or Python.
• Maintain and troubleshoot data pipelines to guarantee reliability and optimal performance.
• Monitor, log, and diagnose pipeline failures to enable rapid resolution.
• Collaborate with engineering and DevOps teams on testing strategies and CI/CD integration.
• Implement CI/CD pipelines to support automated testing processes.
• Manage version‑control workflows to facilitate collaborative development.

Required profile

• 5+ years of experience as a QA/Test Automation Engineer in data engineering or data platform environments.
• Proficiency with Azure cloud services and big‑data platforms such as Azure Databricks, Spark or PySpark.
• Strong knowledge of data pipelines, ETL/ELT processes, and pipeline maintenance.
• Hands‑on experience with SQL and/or Python for testing data quality.
• Experience implementing CI/CD pipelines (Azure DevOps, GitHub Actions, etc.).
• Familiarity with version‑control systems, especially GitHub.
• English proficiency at B2 level or higher.

Required skills

• Azure cloud services
• Azure Databricks / Spark / PySpark
• SQL
• Python
• CI/CD tools (Azure DevOps, GitHub Actions)
• GitHub version control
• Monitoring and logging of data pipelines
• Delta Lake (nice to have)
• Azure Data Factory, ADLS, Synapse, Event Hub (nice to have)
• Infrastructure as code (Terraform, ARM/Bicep)
• Docker and Kubernetes
• Agile/Scrum methodologies

What we offer

• Remote work flexibility across Kazakhstan with access to offices in Astana, Almaty, and Karaganda.
• Opportunities to work abroad for up to two months per year.
• Relocation possibilities within a network of 55+ countries.
• Leadership development, career advising, and well‑being programs.
• Certification support for GCP, Azure, and AWS.
• Unlimited access to EPAM’s internal learning platform and free English classes.