Build Data Pipelines from scratch with Python and SQL: leverage standard toolset and develop ETL/ELT code to move data from various internal and external sources into the analytics platform
Create optimized pipelines using Composer to ingest data from many sources
Assemble large, complex data sets that meet functional / non-functional business requirements.
Qualifications
3+ years of Airflow/Composer experience
Strong proficiency in Python with an emphasis in building data pipelines
Ability to write complex SQL to perform common types of analysis and aggregations
Strong ELT/ ETL designer/developer with 5+ years experience
Strong SQL & Python Expert level with 5+ years experience
Performance tuning with SQL
Structured & unstructured data expertise
Cloud environment development & operations experience (GCP, AWS)