You’ll be part of a team delivering data driven revolution to the workplace experience and empowering each employee with augmented decision making. It is an exciting and multifaceted data driven project that needs a talented and driven data engineer to make it all work.
- Collecting and formulating business requirements and design solution architecture and Data pipeline.
- Design Data Pipeline solutions for the enterprise using the Microsoft Azure data platform including ETL and EL processes for complex datasets.
- Experience in development of data pipelines for both real time and batch data on Azure platform covering structured and unstructured data.
- Automation of ETL auditing and ETL email notification.
- Advanced Experience with ADF (Azure Data Factory) is a mandatory skill.
- Experience with using REST and GRAPH API to retrieve or post data is also a must have skill for this role.
- Strong experience in programming languages like Java or Python.
- Guidance to junior Azure ETL developers.
- Experience in Data Lake and Data warehouse design.
- Collaboration with business analysts, data analyst and AI/ML team.
- Feature Engineering, Data Validation, Data Cleansing.
- Design data migration strategies, including working with software architects to ensure continuity of enterprise services.
- Experience in SQL Development, specifically with complex queries, tuning indexes
- Able to work with Scaled Agile Framework (SAFE)
- Drive Quality Assurance to ensure data quality is adhered as is required per requirement of the project (less than 2% of defects).
- Production Deployment of data pipeline and orchestration.
- Being able to work independently.
- Working in an environment where requirements are not provided to you by an analyst. Person needs to be comfortable working where a problem arises or a business case is outlined to the resource AND they are able to solution and come up with options for solution. Then put into action what they think is best while keeping the client team abreast of the latest developments.
- Excellent communication skills with client facing expertise is required.
Qualifications:
Education and Years of Experience (Must Have) :
- 5 - 9 years of overall Data Engineering experience with 5+ years of experience working with Azure services.
- 2+ years demonstrable experience of software development lifecycle, Lean/Agile development methodologies.
- 2+ years in guidance/supervisions to junior Azure ETL developers
- 1-2 projects consuming and leveraging REST, Graph, Groq, or SOAP APIs for receiving and providing data to downstream consumers. (Must Have)
Required and Desired Skills/Certifications (All the Skills which are mention below must be shown in the Resume):
- Azure Data Lake Services
- Azure Data Factory
- Synapse/AZ SQL DB
- SQL
- Azure Event Hub
- Azure Stream Analytics
- Function Apps
- C#
- Logic Apps
- Spark/Data Bricks