Required Skills

Big Data Engineer

Work Authorization

  • US Citizen

  • Green Card

Preferred Employment

  • Corp-Corp

Employment Type

  • Consulting/Contract

education qualification

  • UG :- - Not Required

  • PG :- - Not Required

Other Information

  • No of position :- ( 1 )

  • Post :- 25th May 2022

JOB DETAIL

  • Create and maintain large, complex data sets that meet functional / non-functional business requirements.
  • Identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability, etc.
  • Build the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources using SQL in MySql DB, Snowflakes and AWS 'big data' technologies.
  • Keep our data separated and secure across national boundaries through multiple data centers and AWS regions.
  • Should monitor and maintain system uptime.
  • Perform maintenance and upgrades from time to time without breaking running systems or applications.

Qualifications Required :

Familiarity with a variety of databases like MySQL, Heatwave, Presto, Pinot, Hive, MongoDB or Cassandra.
 Knowledge in Shell Scripting and Terraform - Mandatory.
Manage HDFS and Zookeeper clusters.
Experience building and optimizing 'big data' data pipelines, architectures and data sets in Snowflakes, NiFi, AWS Redshift etc.
Strong analytic skills related to working with unstructured datasets.
Build processes supporting data transformation, data structures, metadata, dependency and workload management.
Working knowledge of message queuing, stream processing, and highly scalable 'big data' data stores.
Experience in installing and configuring big data tools: Spark, Kafka, etc.
Experience with AWS cloud/Azure cloud/GCP.
Experience with object-oriented/object function scripting languages: Python/Java/Scala, etc

 

Company Information