Data Engineer with Scala and Spark/ Pyspark

About the Role

We are seeking a talented and motivated Data Engineer with Scala and Spark/ Pyspark to join the engineering team at Capco. In this role, you will help design, build, and maintain robust infrastructure solutions, ensuring high availability, performance, and security across cloud deployments.

This position is located in India - Bengaluru; India - Chennai ; India - Gurugram; India - Hyderabad; India - Mumbai; India - Pune and is aligned with the Data domain of our technology stack.

Key Responsibilities

• Design and deploy scalable, secure, and resilient cloud infrastructure.
• Build and optimize continuous integration and continuous deployment (CI/CD) pipelines.
• Collaborate with software developers, product managers, and security teams to align deployment processes.
• Monitor system performance, troubleshoot production incidents, and implement automated healing mechanisms.
• Promote best practices for infrastructure as code, configuration management, and system monitoring.

Required Skills & Experience

• Strong familiarity with cloud service providers (AWS, Azure, or GCP).
• Experience with container orchestration tools like Kubernetes or Docker.
• Proficiency in scripting and automation (Python, Bash, or Go).
• Knowledge of Infrastructure as Code tools (Terraform, Ansible, or CloudFormation).
• Understanding of modern CI/CD practices and version control tools (Git, GitLab, GitHub).
• Solid background in networking, operating systems, and security baselines.