Develop Architecture, high-level design for enhancing Cloud data platform capabilities in the areas like Machine Learning (ML), Data Lifecycle Management with focus on Data Archival, Data Governance utilities and best practices
Develop prototypes for new & complex use cases in data engineering using tools like DBT, Airflow, S3, Athena, Glue, RDS by leveraging engineering best practices like Infrastructure as Code, SQL as code
Work with data engineering teams to identify opportunities for improving developer efficiency, implementing data security & data quality measures
Work with data governance teams across various functions to define data security policies, data governance processes and utilities
Continuously refine data lake architecture on AWS for cost efficiency, enhanced security, user experience, resiliency, and alignment with industry best practices
Skills & Experience
Minimum 5 years of experience as Data Infrastructure architect with AWS Tech Stack
Minimum 10 years of experience in data infrastructure, AWS Infrastructure as Code (IaC), data integration with programming
Worked on Data Governance and Data Security related projects with hands-on experience
Worked on Infrastructure as code tools such as Terraform, Used GitHub tool as infra pipeline and for CI/CD
Experience in building custom applications, search portals, monitoring & observability solutions using AWS Cloud Native technology stack
Knowledge of Data Mesh / Data Fabric with exposure to various architecture frameworks