- 8+ years of experience in creating Data Platform Solutions
- 5+ years of experience in implementing Data bricks, Data bricks Note book, Data Lakes, Data warehouse and Data Lake Architectures best practices for data storage, loading, retrieving data from data lakes.
- 3+ years of Azure Cloud Technologies building pipelines using Pyspark/Python
- Must have good understanding of Azure Data Lakes and hands-on experience in implementing best practices in data storage, loading, retrieving data from data lakes.
- Deep experience in Azure Platform and Azure Data Services
- Data Processing & Transformation using DataBricks, HDInsight, Azure Functions
- Data Staging & Storage using Data Lake Store, Synapse, Cosmos DB
- Data Ingestion using Event Hub
- Data Orchestration using Azure Data Factory
- Working on Azure Databricks
- Spark scripts to process the data loads to different layers
- Understanding of AI/ML Services, Data Modelling & Analytics
- Engineer and maintain a modern Cloud data pipeline for ETL/ELT
- Implement modern data solutions with Azure Data Factory, Data Lake, Data Bricks, SQL data warehouse, and Cosmos DB.
- Azure Data Engineer Certification is preferable
Good to have
- Machine learning experience
- Big Data knowledge
- Snowflake or Big Query knowledge