Big data ,ETL, Design Approach, SQL ,Cloud ,Spark
- Expert in writing SQL Query including understanding of leveraging index and partitions.
- Experience in delivering Data Engineering work using Databricks or any other Apache Spark platform.
- Experience with streaming data processing.
- Experience working with APIs as both source and target.
- Experience working in Azure environment (focusing on Azure SQL Database, Azure Data Lake Gen2, Azure Data Factory, Azure Analysis Service, CosmosDB, Azure Event Hub).
- Worked with GIT and CI/CD pipeline (preferable Azure DevOps / VSTS).
- Data analysis, data modelling, data integration, data warehousing and database design.
- Responsible for quality and reliability of deployed ETL pipelines and Data warehouse