Big Data Developer with Spark and Scala
Long term contract
Chicago,IL or Dallas,TX(Remote Till Covid)
-
Key responsibilities:
- Perform Development & Support activites for Data warehousing domain using Big Data Technologies
- Understand High Level Design, Application Interface Design & build Low Level Design. Perform application analysis & propose technical solution for application enhancement or resolve production issues
- Perform Development & Deployment. Should be able to Code, Unit Test & Deploy
- Creation necessary documentation for all project deliverable phases
- Handle Production Issues (Tier 2 Support, weekend on-call rotation) to resolve production issues & ensure SLAs are met
- Technical Skills:
-
Mandatory
- Should have very clear understanding of hadoop architecture
- 3+ Hands on in Hadoop(Hbase,HDFS,Pig,Hive,Map-reduce)
- Experience in SVN,Build Tools like Ant,Maven etc.
- 3+ Hands on in SQL, Unix & advanced Unix Shell Scripting
- Hands on file transfer mechanism (NDM, sFTP etc)
- Knowledge of Schedulers
-
Good to have
- working knowledge of Kafka, Storm, Spark.
- should have working experience of working on datalake environments,.
- should have handled xml, json, structured, fixed-width, un-structured files using custom MR/Pig/Hive, strong analytical thinking.
- Willingness to learn all data warehousing technologies & work out of the comfort zone in other ETL technologies (Datastage, Oracle, Mainframe etc). Hands on working experience is a plus
Kamal Sukhija
kamal@idctechnologies.com
Direct: 408 457 9399 Ext-3170/234-217-2599