Please share your resume at ruchita.parsekar@e-stonetech.com

Experience: 4+ Years
Location : Mumbai

Job Description:

1. Spark Cluster Deployment: Deploy, configure, and maintain Apache Spark clusters on Kubernetes, ensuring scalability, reliability, and performance.
2. Application Deployment: Collaborate with data engineers and data scientists to deploy Spark applications and workloads, ensuring they run efficiently.
3. Monitoring and Optimization: Implement monitoring solutions to track cluster performance, resource utilization, and application health. Proactively identify and resolve performance bottlenecks.
4. Resource Management: Manage cluster resources, including CPU, memory, and storage allocation, to ensure optimal utilization and cost efficiency.
5. Security: Implement and maintain security measures, including authentication, authorization, and encryption, to protect sensitive data and Spark clusters.
6. Backup and Recovery: Develop and maintain backup and recovery strategies to ensure data integrity and availability in case of failures.
7. Documentation: Maintain clear and comprehensive documentation of Spark cluster configurations, deployment procedures, and best practices.
8. Troubleshooting: Quickly diagnose and resolve issues related to Spark clusters, applications, and Kubernetes infrastructure.
9. Collaboration: Work closely with cross-functional teams, including data engineers, data scientists, and DevOps, to understand application requirements and optimize Spark clusters accordingly.

Qualifications:

• Proven experience deploying and managing Apache Spark on Kubernetes in a production environment.
• Proficiency in containerization technologies, particularly Docker and Kubernetes.
• Strong knowledge of Spark architecture, including cluster, driver, and worker nodes.
• Familiarity with Spark tuning, optimization, and performance monitoring.
• Experience with resource management tools like Kubernetes Resource Quotas and LimitRanges.
• Understanding of data processing and analytics workflows.
• Excellent problem-solving and troubleshooting skills.
• Strong communication and collaboration skills.

Preferred Qualifications:

• Experience with Spark cluster orchestration tools like Helm.
• Knowledge of Spark ecosystem components such as Spark SQL, Spark Streaming, and MLlib.
• Familiarity with cloud-based solutions (e.g., AWS, Azure, Google Cloud).
• Certification in Kubernetes (e.g., Certified Kubernetes Administrator – CKA).
• Knowledge of CI/CD pipelines and infrastructure as code (IaC) tools (e.g., Terraform).
• Scripting skills in languages like Python, Bash, or Shell.
• Understanding of DevOps practices and automation.

Kindly share your updated resume in word format.