Required Skills

Scala Spark Hadoop Kafka

Work Authorization

  • US Citizen

  • Green Card

  • EAD (OPT/CPT/GC/H4)

  • H1B Work Permit

Preferred Employment

  • Corp-Corp

  • W2-Permanent

  • W2-Contract

  • Contract to Hire

Employment Type

  • Consulting/Contract

education qualification

  • UG :- - Not Required

  • PG :- - Not Required

Other Information

  • No of position :- ( 1 )

  • Post :- 20th Oct 2023

JOB DETAIL

  • Monitor production jobs for the P1 system (Data Management and Reporting – DMR). 

  • Debug and restart in the event of production failures. 

  • Acknowledge, respond to, and resolve production issues. 

  • Understand change requests and enhancement requests. 

  • Design and develop data loading strategies. 

  • Build, develop, and test shared components that will be used across modules. 

  • Extract source data files, stage, transform, and load into the EERA system (PostgreSQL). 

  • Ingest data from the Mainframe system using Kafka and load it into EERA. 

  • Extract source data files, stage, transform, and load into the BDS system (Hive-based). 

  • Use Apache Spark for large data processing integrated with the functional programming language Scala. 

  • Create programs to continuously listen to requests from the consumption layer, generate data extracts, and publish them via email. 

  • Provide on-call support during the weekend. 

  • Collaborate with the offshore team on a daily or as-needed basis. 

Mandatory Skills: 

  • Proficiency in Scala, Spark, Hadoop, Kafka for Big Data processing. 

  • Experience with Postgres SQL and Hive for data storage and retrieval. 

  • Knowledge of workflow management tools like Airflow, Control M, or Oozie. 

  • Familiarity with containerization tools like Kubernetes for deployment and management. 

  • Agile project management methodologies. 

  • Strong architectural, design, and coding skills. 

  • Previous experience in production operations and support. 

  • Experience with distributed databases. 

 

Company Information