At least 8 to 10+ years’ experience in data engineering , AI and ML and Aws/Azure
Architect, Design and implement distributed data processing pipelines using Spark, Python, SQL and other tools and languages prevalent in the Big Data/Lakehouse ecosystem.
Experience in AI and ML on AWS/AZURE
Analyzes design and determines coding, programming, and integration activities required based on general objectives.
Play the technical lead role representing deliverables from vendor team resources at onsite and offshore locations.
Lead the technical co-ordination and Business Knowledge transition activities to offshore team.
Reviews and evaluates designs and project activities for compliance with architecture, security and quality guidelines and standards.
Writes and executes complete testing plans, protocols, and documentation for assigned portion of data system or component; identifies defects and creates solutions for issues with code and integration into data system architecture.
Collaborates and communicates with project team regarding project progress and issue resolution.
Works with the data engineering team for all phases of larger and more-complex development projects and engages with external users on business and technical requirements.
Collaborates with peers, engineers, data scientists and project team.