US Citizen
Security Clearance
Green Card
Need New H1B (OPT/CPT Fresh Grads)
Corp-Corp
Consulting/Contract
UG :- - Not Required
PG :- - Not Required
No of position :- ( 1 )
Post :- 11th Apr 2022
A Sr. Data Engineer is proficient in the development of all aspects of data processing including data warehouse architecture/modeling and ETL processing. The position focuses research on development and delivery of analytical solutions using various tools including Confluent Kafka, Kinesis, Glue, Lambda, Snowflake and SQL Server. A Sr. Data Engineer must be able to work autonomously with little guidance or instruction to deliver business value.
Responsibilities
• Partner with business stakeholders to gather requirements and translate them into technical specifications and process documentation for IT counterparts (on-prem and offshore)
• Highly proficient in the architecture and development of an event driven data warehouse; streaming, batch, data modeling, and storage
• Advanced database knowledge; creating/optimizing SQL queries, stored procedures, functions, partitioning data, indexing, and reading execution plans
• Skilled experience in writing and troubleshooting Python/PySpark scripts to generate extracts, cleanse, conform and deliver data for consumption
• Expert level of understanding and implementing ETL architecture; data profiling, process flow, metric logging and error handling
• Support continuous improvement by investigating and presenting alternatives to processes and technologies to an architectural review board
• Develop and ensure adherence to published system architectural decisions and development standards
• Multi-task across several ongoing projects and daily duties of varying priorities as required
• Interact with global technical teams to communicate business requirements and collaboratively build data solutions
Requirements
• 8 years of development experience
• Expert level in data warehouse design/architecture, dimensional data modeling and ETL process development
• Advanced level development in SQL/NoSQL scripting and complex stored procedures (Snowflake, SQL Server, DynomoDB, NEO4J a plus)
• Extremely proficient in Python, PySpark, and Java
• AWS Expertise – Kinesis, Glue (Spark), EMR, S3, Lambda, and Athena
• Streaming Services – Confluent Kafka and Kinesis (or equivalent)
• Hands on experience in designing and developing applications using Java Spring Framework (Spring Boot, Spring Cloud, Spring Data etc)