Looking for 8 years of lead Big Data engineer candidate
5+ years of experience with Big Data pipelines with Spark in Java, Scala, Python
5+ Years of extensive AWS experience in building Enterprise scale applications and services
5+ Years of experience with AWS services like ECS Fargate , EKS , Docker containers , Lambdas , EMR , EC2 , SNS/SQS , MKS , S3 , RDS etc.
In-Depth understanding of Orchestration services and hands-on experience with Airflow
Proven track record of building scaled data platforms and enterprise products, working in large engineering teams
Experience with public Clouds such as AWS, Azure, GCP
Extensive Experience in building Enterprise Level AWS infrastructure using Terrafrom and/or Cloud Formation Templates
Excellent understanding of DevOps and CI/CD concepts and ability to create deployment Pipelines
Ensure industry best practices around data pipelines, metadata management, data quality, data governance and data privacy.
Build highly scalable AWS Infrastructure (from scratch or through 3rd party products) to enable Big Data Processing in the platform.
Ability to work with internal stakeholders like Networking/API Market Place/Infosec etc. and external ones like third party apps to build new services inside the platform.