Tagcor

Data Engineer with AirFlow

Experience with Apache Airflow, including installing, configuring and monitoring Airflow cluster.
Develop guidelines for Airflow cluster and DAG (Direct Acyclic Graph).
Build DAG to orchestrate augmented pipeline implementing models in pipeline, running reconciliation, DQ, etc.
Develop data pipeline to on-board and change management of datasets
Implement CI/CD for Airflow deployment and test automation frameworks.
Understanding of airflow rest services and integration of airflow of platform eco-system
Orchestrating the Airflow / workflow in hybrid cloud environment is big plus
Experience working in an Agile or iterative approach to delivery preferred.
Should be hands on with: Python & Pandas, with demonstrated experience in ETL.
Ability to maintain clean and secure data environments.
Working in a hybrid cloud environment.
Develop and contribute across the stack (Javascript/React, Java backend)
Experience in setting and Schedulers: Developing hooks, templates, Alerts and scheduling for pipelines in Airflow