US Citizen
Green Card
EAD (OPT/CPT/GC/H4)
H1B Work Permit
Corp-Corp
W2-Permanent
W2-Contract
Contract to Hire
Consulting/Contract
UG :- - Not Required
PG :- - Not Required
No of position :- ( 1 )
Post :- 28th Nov 2023
We are seeking a talented and experienced individual to join our team as an Airflow Specialist with expertise in migrating jobs from Control-M to Astronomer managed Airflow. In this role, you will play a crucial part in transitioning our workload orchestration from Control-M to Airflow, ensuring seamless and efficient execution of data workflows.
Responsibilities:
Migration Planning:
Collaborate with stakeholders to understand existing Control-M job dependencies, schedules, and requirements.
Develop a comprehensive migration plan outlining the step-by-step process for transitioning jobs to Airflow.
Airflow Implementation:
Configure and set up Astronomer Airflow environments for optimal performance.
Design and implement Airflow DAGs (Directed Acyclic Graphs) to represent and schedule workflows efficiently.
Utilize Airflow operators and sensors to integrate with various systems and technologies.
Job Migration:
Analyze existing Control-M jobs and translate their logic into Airflow workflows.
Ensure data integrity and consistency during the migration process.
Debug and troubleshoot issues that may arise during the migration.
Documentation:
Create comprehensive documentation for migrated workflows, including DAG structures, dependencies, and scheduling details.
Provide training and knowledge transfer sessions to the operations team for managing and maintaining Airflow workflows.
Optimization and Performance Tuning:
Fine-tune Airflow configurations for optimal performance and resource utilization.
Identify and implement improvements to enhance overall system efficiency.
Collaboration:
Work closely with cross-functional teams, including data engineers, developers, and system administrators, to ensure a smooth transition and ongoing support.
Qualifications:
Education and Experience: