• Us Citizen

• Green Card

• EAD (OPT/CPT/GC/H4)

• H1B Work Permit

• Corp-Corp

• UG :- - Not Required

• PG :- - Not Required

• No of position :- ( 1 )

• Post :- 25th Nov 2020

•        Evangelist for data engineering function leveraging big data processing framework.

•        Creation and optimization of data engineering pipelines for analytics projects.

•        Support data and cloud transformation initiatives

•        Support our software engineers and data scientists

•        Contribute to our cloud strategy based on prior experience

•        Understand the latest technologies in a rapidly innovative marketplace

•        Independently work with all stakeholders across the organization to deliver enhanced functionality

Skills - Experience and Requirements

A successful Data Engineer will have the following:

•        Must be from Data warehouse/Big data background.

•        Experience in advanced Apache Spark processing framework, spark programming languages such as Scala/Python/Advanced Java with sound knowledge in shell scripting.

•        Experience in working with Core Spark, Spark Streaming, Dataframe API, Data set API, RDD APIs & Spark SQL programming dealing with processing terabytes of data. Specifically, this experience must be in writing "Big Data" data engineering jobs for large scale data integration in AWS.

•        Advanced SQL experience using Hive/Impala framework including SQL performance tuning

•        Experience in writing spark streaming jobs integrating with streaming frameworks such as Apache Kafka or AWS Kinesis.

•        Create and maintain automated ETL processes with special focus on data flow, error recovery, and exception handling and reporting

•        Gather and understand data requirements, work in the team to achieve high quality data ingestion and build systems that can process the data, transform the data

•        Knowledge of using, setting up and tuning resource management framework such as standalone spark, Yarn or Mesos.

•        Experience in physical table design in Big Data environment

•        Experience working with external job schedulers such as autosys, aws data pipeline, airflow etc.

•        Experience working in Key/Value data store such as Hbase

•        Experience in AWS services such as EMR, Glue (server less architecture), S3, Athena, IAM, Lambda and Cloud watch is required.

SAM!                                                                                                                                        

Technical Recruiter

IDC Technologies.

Ph: 408-903-4154

Email: sami.uddin@idctechnologies.com