US Citizen
Green Card
EAD (OPT/CPT/GC/H4)
H1B Work Permit
Corp-Corp
W2-Permanent
W2-Contract
Contract to Hire
Consulting/Contract
UG :- - Not Required
PG :- - Not Required
No of position :- ( 1 )
Post :- 27th Nov 2024
· Data Pipeline Development:
Design, build, and implement robust data pipelines using Python libraries like Pandas, NumPy, and PySpark to ingest, cleanse, transform, and load data from diverse sources (databases, APIs, flat files) into data warehouses or data lakes.
· Architecture Design:
Architect scalable data processing systems considering data volume, velocity, and variety, including distributed computing frameworks.
· Code Optimization:
Write efficient and maintainable Python code, optimizing data processing logic for performance and minimizing latency in data pipelines.
· Azure Cloud Integration:
• 10+ years of experience building data platforms & application architectures.
• Proven experience building & deploying data warehouses with Azure platform
• A strong business and technical proficiency in SQL, Azure Data Factory.
• Build the infrastructure required for optimal extraction, transformation, and data loading from various sources.
• Comprehensive understanding of Sales opportunities, pipeline, etc.
• Development of Pipelines using Azure Data Factory.
• Integrate ADF with target db like Snowflake.
• Utilize various activities like Copy Activity, Execute Activity, Variables etc in ADF.
• Usage of Data Flow in the ADF for the transformations.
• Load data from Sources to Azure SQL using Azure Data Factory.
Work closely with data scientists, analysts, and other stakeholders to understand business requirements and translate them into effective data solutions.
· Monitoring and Maintenance:
Monitor data pipelines for performance issues, troubleshoot errors, and implement necessary maintenance updates.