Senior Spark QA Engineer

Pune, Maharashtra, India | Full-time | Fully remote

Apply
Senior Spark QA Engineer – Functional & Performance Testing

Location: Remote
Experience: 5+ Years 

Job Description - 
We are looking for a Senior Spark QA Engineer with strong expertise in functional and performance testing of Apache Spark applications and distributed data platforms.

Key Responsibilities -
  • Perform manual and automated testing of Spark jobs, Spark SQL queries, and ETL pipelines.
  • Execute functional, scalability, and performance testing for Spark workloads.
  • Set up and manage Spark clusters on Standalone, YARN, Kubernetes, Mesos, Databricks, EMR, and Dataproc.
  • Conduct benchmarking, validation, and performance analysis of Spark applications.
  • Identify bottlenecks and troubleshoot distributed system issues.
  • Lead QA initiatives and mentor team members.
Required Skills - 
  • 5+ years of QA/testing experience with strong hands-on expertise in Apache Spark.
  • Experience in functional and performance testing of distributed systems.
  • Strong understanding of Spark architecture and optimization techniques.
  • Experience with Kubernetes and cloud-based Spark environments.
  • Proficiency in Python or Java.
  • Experience with automation frameworks and CI/CD pipelines.