Strong verbal and written communication skills to effectively articulate messages to internal and external teams.
Hands on experience with Azure data platform stack (Azure Databricks, Azure Data Factory, Azure Data Lake storage, Azure DevOps, Azure Functions) – open to other public cloud exp
Experience with dimensional modeling and data warehousing concepts
Experience with designing, building, optimizing, troubleshooting end-to-end big data pipelines using structured (relational and files) and semi-structured data
Experience with building metadata driven data processing frameworks
Strong experience in Spark, SQL, python, Pyspark, shell scripting.
Apache Airflow
Ability to work in an agile team.
Ability to take ownership of a request from initial requirements, design, development and production deployment