• US Citizen

• Green Card

• EAD (OPT/CPT/GC/H4)

• H1B Work Permit

• Corp-Corp

• W2-Permanent

• W2-Contract

• Contract to Hire

• Consulting/Contract

• UG :- - Not Required

• PG :- - Not Required

• No of position :- ( 1 )

• Post :- 13th Oct 2023

Responsibilities:

• Databricks Development:
  • Design, develop, and implement scalable and efficient data processing solutions using Azure Databricks.
  • Utilize SCALA and PySpark to build robust ETL processes for large-scale data sets.

• Cluster Management:
  • Manage and optimize Azure Databricks clusters for optimal performance and resource utilization.
  • Implement best practices for cluster configuration, monitoring, and maintenance.

• Big Data Processing:
  • Work with large volumes of data, ensuring effective processing, transformation, and integration.
  • Implement and optimize data processing workflows to meet performance and scalability requirements.

• Performance Tuning:
  • Fine-tune and optimize existing ETL processes for maximum efficiency.
  • Identify and address bottlenecks in data processing pipelines.

• Data Integration:
  • Integrate data from diverse sources, ensuring data consistency and accuracy.
  • Collaborate with data architects and other teams to understand and fulfill data integration requirements.

Qualifications:

• Minimum of 13 years of professional experience in data engineering and development.
• Expertise in SCALA and PySpark programming languages, with a focus on big data processing.
• Extensive hands-on experience with Azure Databricks, including cluster management and notebook development.
• In-depth knowledge of distributed computing and big data processing frameworks.
• Proven experience in designing and implementing complex ETL processes.
• Strong proficiency in SQL and data modeling.
• Excellent problem-solving and troubleshooting skills.
• Effective communication and collaboration skills.