• US Citizen

• Green Card

• EAD (OPT/CPT/GC/H4)

• H1B Work Permit

• Corp-Corp

• W2-Permanent

• W2-Contract

• Contract to Hire

• Consulting/Contract

• UG :- - Not Required

• PG :- - Not Required

• No of position :- ( 1 )

• Post :- 31st Jan 2024

Job Description:

The Data Engineer/Developer role is a crucial position that requires a highly skilled individual to handle complex data projects. This role involves developing and maintaining data pipelines using specialized scripting languages like PySpark. Proficiency in Databricks is a must as this person will be responsible for optimizing data processes. They will also need to work with various libraries to manage, transform, and analyze large datasets efficiently. The ideal candidate for this role should have exceptional problem-solving skills and be able to drive strategic data initiatives.

Qualifications:

As a Data Engineer / Developer, the ideal candidate should possess a strong background in scripting and hands-on experience with PySpark, Databricks, and developing data pipelines. They should have a deep understanding of how to optimize data processing and have familiarity with a variety of libraries and tools commonly used in data engineering. Additionally, a strong understanding of data architecture and data modeling is necessary for this role. Strong problem-solving skills, attention to detail, and the ability to work collaboratively in a fast-paced environment are essential for success in this role. The ideal candidate should also have a passion for continuously learning and staying updated with the latest technologies in the field of data engineering.
Responsibilities

Responsibilities:

• Design and develop scripts for data integration and processing using PySpark.
• Utilize Databricks platform to build and maintain data pipelines for data ingestion and transformation.
• Collaborate with data scientists and analysts to optimize data pipelines for efficient data processing.
• Identify bottlenecks and troubleshoot issues in data pipelines to ensure accuracy and timeliness of data.
• Develop and maintain libraries to support data processing and analysis.
• Implement and maintain data quality checks to ensure data accuracy and consistency.
• Work closely with cross-functional teams to understand data requirements and provide technical solutions.
• Follow best practices and standards in data engineering to maintain a high level of quality and performance in pipelines.
• Continuously monitor and improve data engineering processes for increased efficiency and scalability.
• Stay updated on new technologies and techniques in data engineering and incorporate them into current processes as applicable.