Required Skills

Big Data Engineer text files Sequence files JSON Parquet ORC

Work Authorization

  • US Citizen

  • Green Card

  • EAD (OPT/CPT/GC/H4)

  • H1B Work Permit

Preferred Employment

  • Corp-Corp

  • W2-Permanent

  • W2-Contract

  • Contract to Hire

Employment Type

  • Consulting/Contract

education qualification

  • UG :- - Not Required

  • PG :- - Not Required

Other Information

  • No of position :- ( 1 )

  • Post :- 20th Sep 2023

JOB DETAIL

•             Having 10 years of professional experience fields of software Analysis, Design, Development, Deployment and Maintenance of software and Big Data applications.

•             Experience in Big data Implementation with strong experience on major components of Iceberg, Tableau, Kafka, Superset, Druid, Hive metastore, apache, ranger, security, AWS

•             Experience in creating iceberg tables and loading the data from different file formats.

•             Good Experience in Data importing and exporting to Hive and HDFS with Sqoop.

•             Experience in using Producer and Consumer API’s of Apache Kafka.

•             Skilled in integrating Kafka with Spark streaming for faster data processing.

•             Experience in using Spark Streaming programming model for Real-time data processing.

•             Experience dealing with the file formats like text files, Sequence files, JSON, Parquet, ORC.

•             Extensively used Apache Kafka to collect the logs and error messages across the cluster.

•             Excellent knowledge and understanding of Distributed Computing and Parallel processing frameworks.

•             Experienced with Analytics with Hive Megastore.

•             Experience with Superset, Druid.

•             Experience working with EC2 (Elastic Compute Cloud) cluster instances, setup data buckets on S3 (Simple Storage Service), setting up EMR (Elastic MapReduce).

•             Good experience working on Tableau and enabled the JDBC/ODBC data connectivity from those to Hive Metastore.

•             Good with version control systems like GIT.

•             Strong knowledge on UNIX/LINUX commands.

•             Adequate Knowledge on Python scripting Language.

•             Adequate knowledge of Scrum, Agile and Waterfall methodologies.

•             Highly motivated and committed to the highest levels of professionalism.

•             Exhibited strong written and oral communication skills. Rapidly learn and adapt quickly to emerging new technologies and paradigms.

Company Information