• US Citizen

• Green Card

• EAD (OPT/CPT/GC/H4)

• H1B Work Permit

• Corp-Corp

• W2-Permanent

• W2-Contract

• Contract to Hire

• Consulting/Contract

• UG :- - Not Required

• PG :- - Not Required

• No of position :- ( 1 )

• Post :- 23rd Feb 2024

We are seeking an experienced and dynamic Lead Big Data Developer/Architect to join our team. The ideal candidate will have a strong background in Big Data technologies, Data Warehousing, and related tools. As a Lead Big Data Developer/Architect, you will play a crucial role in designing, developing, and implementing innovative solutions on a Hadoop-based platform.

Key Responsibilities:

- Design, develop, and implement Big Data analytic solutions on a Hadoop-based platform.
- Refine data processing pipelines focused on unstructured and semi-structured data.
- Create custom analytic and data mining algorithms to extract knowledge and meaning from vast data stores.
- Configure data flows from different sources (relational databases, XML, JSON) and orchestrate them using Nifi.
- Develop Spark Frameworks using PySpark and Java for Raw/Analytical Layers in Big Data.
- Utilize Jenkins for Continuous Integration and Git for Version Control.
- Write shell scripts and job management scripts to invoke and manage Data Ingestion steps.
- Design HIVE tables for better performance and apply partitions where needed.
- Review HDFS data organization and provide mechanisms to support Multi-Tenant features.
- Work on AWS Services like S3, EMR, Lambda, Glue Jobs, Athena as part of the Open data initiative.
- Build and maintain QlikView dashboards using data from various sources.
- Collaborate with cross-functional teams and stakeholders to gather requirements and provide technical expertise.
- Mentor and guide junior team members.

Qualifications:

- Bachelor’s or Master’s degree in Computer Science, Information Technology, or related field.
- 15+ years of total IT experience with a focus on Big Data and Data Warehousing technologies.
- Strong proficiency in Python, Java, SQL, PL/SQL, Unix Shell Scripting.
- Extensive hands-on development experience with Hadoop technologies including Spark using Python and Java, Hive, Impala, Sqoop, AWS EMR.
- In-depth knowledge of Spark distributed Framework involving RDD and Data Frames using Python, Java8, Scala.
- Experience with AWS services such as S3, EMR, Lambda, Glue, Athena.
- Proficiency in building and maintaining QlikView dashboards.
- Excellent understanding of Data Warehousing concepts, Dimensional data modeling, and ETL processes.
- Certification in relevant technologies, such as Oracle Certified Associate.

Preferred Skills:

- Familiarity with Apache SOLR, Elasticsearch for data indexing and search.
- Experience with Kafka for data streaming.
- Knowledge of Data Governance and Metadata Management.
- Previous exposure to Informatica PowerCenter or similar ETL tools.
- Strong analytical and problem-solving skills.
- Excellent communication and interpersonal skills.