• US Citizen

• Green Card

• Corp-Corp

• Consulting/Contract

• UG :- - Not Required

• PG :- - Not Required

• No of position :- ( 1 )

• Post :- 21st Apr 2022

Below is a generic job description- This person will be pulling data out of a DB2 database and loading it into Snowflake to do modeling and reporting- HVR for the ingestion and DBT for the transformation to Snowflake - Develop solutions related to Big Data, and Data Sciences from end-to-end (data ingestion to consumption)

• Develop and maintain scalable data pipelines that will ingest, transform, and distribute data streams and/or batches within the Hadoop and Microsoft Platforms

• Identify, design, and implement process improvements for automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability

• Self-motivated, self-directed with strong problem-solving skills

• Support and decommission legacy platforms

• Enable business strategy, lean processes, increased data velocity, & insights

• Embody a culture of continuous innovation and learning

• Adhere to programming/development standards and governance framework

• Collaborate with business, analytical teams, and data scientist to improve efficiency, increase the applicability of predictive models, and help translate ad-hoc analyses into scalable data delivery solutions

• Consulting on data ingestion, data modeling, security and capabilities

• Manage the innovation cycle of conducting analyses, generating insights

• Assist with the selection and management of consultants and vendors

• Assist with the recruitment and development of talent

• Collaborate with DevOps team to integrate innovations and algorithms into a production system

• Support business decisions with ad hoc analysis as needed

• Work with the DevOps team to create and manage deployment workflows for all scripts and code using Microsoft Azure

 

Basic Qualifications (required):

• Bachelor's degree in Computer Science, MIS, or related field.

• Experience in Big Data Engineering or Business Intelligence

o Data Integration

o Data Modeling

o ETL/ELT and SQL Development

• 3 or more years of experience in Software Engineering

o Development Lifecycle

o Test Driven Development

• 1 one or more Experience with Cloud Big Data Technologies

o CDC tools (HVR, Qlik Replicate)

o AWS Native Tools (Glue, DMS, S3, Athena), Snowflake, Cloudera CDP or Databricks

o NoSQL Databases such as Hive, Spark

• 3 or more years of experience with programming languages like Java, C++ or C# o Object Oriented Design

• 3 or more years of experience with Scripting languages

o Python 2.7 and/or 3

 

Preferred Qualifications (Desired)

• In-depth experience with Apache Kafka

• In-depth experience with Change Data Capture Tools

• In-depth experience with cloud data movement techniques.

• Experience with Cloudera Data Platform including Impala

• Masters degree in Information Systems, Computer Science or related field

• 5 or more years of experience with C/C++

• NoSQL background such as MongoDB, HBase

• Experience with Search Engine tools such as Lucene and ElasticSearch

• Experience with IDEs such as Microsoft Visual Code, Eclipse and/or PyCharm

• Experience with version control using Git

• Experience with Agile Methodology using Scrum

• Knowledge of Data Science using python and R