Must be strong in Java/Python/Scala and have worked with Hadoop, Spark3, Cloudera and ElasticSearch.
Overall 12+ years of experience in Data Engineering, Data Modelling.
Excellent (Hands-On) in Azure, ADF, Databricks (PySpark), Python, SQL, Unix Shell scripting. Good experience in Snowflake Datawarehouse.
Experience in ETL, tools like Matillion, QLIK, DataStage, and performance tuning / optimization.
Experience in building dimensional data models.
Experience in working with ML teams to coordinate and enable the promotion of ML models to a governed production environment to bring stability and robustness to be supported by AMS (Data Ops and ML Ops team) teams.
Coordinating with onshore and offshore cross-functional teams to deliver concurrent projects.
Strong ETL experience in handling large volumes of data in the complex heterogeneous data warehouses and processing high volume jobs.
Experience in building data ingestion pipeline and data replication into cloud environments hosted on Azure platform.
Ability to architect scalable data pipelines, following the best practices.