• US Citizen

• Green Card

• EAD (OPT/CPT/GC/H4)

• H1B Work Permit

• Corp-Corp

• W2-Permanent

• W2-Contract

• Contract to Hire

• Consulting/Contract

• UG :- - Not Required

• PG :- - Not Required

• No of position :- ( 1 )

• Post :- 22nd Feb 2024

We are seeking a passionate and skilled ML Data Engineer (Band 4B) to join our team in USA. You will play a pivotal role in building and maintaining the data infrastructure and pipelines for our cutting-edge Generative AI applications. You will collaborate closely with the Generative AI Full Stack Architect and MLOps Engineer to ensure the quality, security, and accessibility of data for our Generative AI models.
Responsibilities:
Design, develop, and implement data pipelines for ingesting, pre-processing, and transforming unstructured data (Image, .pdf, Audio, video) for Generative AI model training and inference.
Need to have some level of understanding or working experience with Vector DB’s ( Like Pinecone , Redis , Chroma)
Understanding on Large Language Model’s ( Llama , GPT-4 , Claude 2.0 ) to do text summarization , entity extraction and classification.
Build and maintain efficient data storage solutions, including data lakes, warehouses, and databases, appropriate for large-scale generative AI datasets.
Implement data security and governance policies to ensure the privacy and integrity of sensitive data used in Generative AI projects.
Collaborate with data scientists and engineers to understand data requirements for Generative AI models and translate them into efficient data pipelines.
Monitor and optimize data pipelines for performance, scalability, and cost-effectiveness.
Stay up-to-date on the latest advancements in data engineering tools and technologies (e.g., Apache Spark, Airflow, Snowflake, Data Bricks ) and apply them to our Generative AI platform.
Document data pipelines and processes for clarity and transparency.
Communicate effectively with technical and non-technical stakeholders about data quality and availability for Generative AI projects.

Qualifications:
Bachelor’s degree in computer science, Data Science, Statistics, or a related field, or equivalent experience.
6+ years of experience in data engineering or related roles, such as data pipeline development, data storage, or ETL/ELT processes.
Proven experience building and maintaining data pipelines for machine learning projects.
Strong understanding of data modeling principles, data quality measures, and data security best practices.
Proficient in programming languages like Python, SQL, and scripting languages (e.g., Bash, Shell).
Familiarity with cloud platforms (e.g., AWS, GCP, Azure) for data storage and processing along with GenAI services like (AWS BedRock)
Excellent communication, collaboration, and problem-solving skills.
Ability to work independently and as part of a team.
Passion for Generative AI and its potential to solve real-world challenges.

Must have
Senior individual contributor with substantial data engineering expertise and leadership experience.
Manages complex data projects and initiatives with independent decision-making authority.
Provides technical guidance and mentorship to junior team members.
Has a demonstrated track record of success in delivering impactful data solutions.