Client is seeking a Data Engineer (contract) to join their Digital team.
The Data Engineer will be developing data integration solutions to support our Analytics platform. The candidate will need to understand business and technical requirements and translate this into data pipeline designs and solutions. The solutions must align with EDW, Data mart and ETL standards and guidelines. The data integration solutions will be based on Microsoft Azure Data Factory with Snowflake and MS SQL Server data warehouses/databases & PowerBI for reporting platform. The position also requires supporting the SDLC process in terms of testing, deployment, and postproduction support.
ESSENTIAL DUTIES/RESPONSIBILITIES
- Understand business requirements and convert them into detail technical specifications
- Design and Develop ETL programs primarily using MS Azure Data Factory, TSQL, Power Shell Scripts
- Extract, transform & load into dimension model structures of EDW & data marts
- Performance improvement & tuning of ETL mappings, SQL & TSQL scripts
- Technical reviews, Data validation & end to end testing of ETL Objects, Source data analysis and data profiling.
- Develop new data integrations and support existing integrations
- Create required documentation as per SDLC standards
- Update the project documentation (Change Requests and JIRA’s) on the confluence
- Upload and maintain version control of developed code by using version control systems (GIT)
- Perform Unit Testing of ETL Programs and support QE on End-to-End testing
- Work with deployment team in the code migration across the SDLC environments
- Triage the Production issues and work on the code fixes
- Build reports using PowerBI
- Create and maintain optimal data pipeline architecture
- Assemble large, complex data sets that meet both functional and non-functional business requirements
- Identify, design, and implement internal process improvements: automating manual processes,
- optimizing data delivery, re-designing infrastructure for greater scalability, etc.
EDUCATION:
- A Bachelor’s degree in Computer Science Engineering, Data Analytics, or a related technical degree.
- TRAININGS/CERTIFICATIONS REQUIREMENTS (licenses, programs, or certificates):
- Snowflake /MS Azure Data Factory Certification is preferred
EXPERIENCE:
- Minimum 6-7 years of experience in implementation of BI projects in a large-scale enterprise data warehouse environment
- Must have working experience in eCommerce or Salesforce and in areas of Sales, Marketing, Finance, or Support domains
- Knowledge of end-to-end SDLC process in EDW & BI projects
- Dimensional Modeling and Data warehouse, ODS concepts, such as star schemas, snowflakes and normalized data models
- Minimum 5 years of experience in ETL Architecture and ETL products
- Converting functional specifications into ETL technical specifications and source to target mappings
- Hands on experience in building the ETL jobs using Microsoft Azure Data Factory, Snowflake and TSQL.
- Hands on experience with database Performance tuning like Snowflake, MSSQL Server, MySQL, and
- strong database languages like SQL, TSQL, Shell scripting and Python
- Exposure to Web services, Cloud based data integrations with Salesforce.com (SFDC) and other SAAS
- applications using Rest or Soap API’s.
- Experience working in agile methodology and Knowledge of JIRA, Confluence Tools and other
- collaboration tools a plus
- Working as Technical lead and leading the small technical teams
Other Knowledge, Skills and Abilities:
- Ability to coordinate effectively with On-site and Off-shore resources through Managed Service
- Providers & IT teams
- Excellent verbal and written communication skills
-