• Us Citizen

• Green Card

• EAD (OPT/CPT/GC/H4)

• H1B Work Permit

• Corp-Corp

• Consulting/Contract

• UG :- - Not Required

• PG :- - Not Required

• No of position :- ( 1 )

• Post :- 4th Nov 2020

Title – Data Engineer

Location – Englewood, CO

Duration – Contract

All the below requirements are mandatory.

1. Person should be having Datawarehousing experience. 
2. Person should have hands on experience on python or spark(using scala or python)
3. Person should have hands on experience on AWS services (S3,EC2, Athena, Redshift,Redshift Spectrum,Glue,Lambda and EMR)

Job Duties and Responsibilities
•        Evangelist for data engineering function leveraging big data processing framework.
•        Creation and optimization of data engineering pipelines for analytics projects.
•        Support data and cloud transformation initiatives
•        Support our software engineers and data scientists
•        Contribute to our cloud strategy based on prior experience
•        Understand the latest technologies in a rapidly innovative marketplace
•        Independently work with all stakeholders across the organization to deliver enhanced functionality

Skills - Experience and Requirements
A successful Data Engineer will have the following:
•        Must be from Data warehouse/Big data background.
•        Experience in advanced Apache Spark processing framework, spark programming languages such as Scala/Python/Advanced Java with sound knowledge in shell scripting.
•        Experience in working with Core Spark, Spark Streaming, Dataframe API, Data set API, RDD APIs & Spark SQL programming dealing with processing terabytes of data. Specifically, this experience must be in writing "Big Data" data engineering jobs for large scale data integration in AWS.
•        Advanced SQL experience using Hive/Impala framework including SQL performance tuning
•        Experience in writing spark streaming jobs integrating with streaming frameworks such as Apache Kafka or AWS Kinesis.
•        Create and maintain automated ETL processes with special focus on data flow, error recovery, and exception handling and reporting
•        Gather and understand data requirements, work in the team to achieve high quality data ingestion and build systems that can process the data, transform the data
•        Knowledge of using, setting up and tuning resource management framework such as standalone spark, Yarn or Mesos.
•        Experience in physical table design in Big Data environment
•        Experience working with external job schedulers such as autosys, aws data pipeline, airflow etc.
•        Experience working in Key/Value data store such as Hbase
•        Experience in AWS services such as EMR, Glue (server less architecture), S3, Athena, IAM, Lambda and Cloud watch is required.

Thanks and Regards,

Sonia | Recruiter

a: IDC Technologies Inc. | 920 Hillview Court, Suite 250 | 95035, Milpitas, CA
e: sonia@idctechnologies.com | w: www.idctechnologies.com 
D: +1 408-819-7586 T: +1 408-819-7586