Required Skills

Lead Data Engineering

Work Authorization

  • US Citizen

  • Green Card

  • EAD (OPT/CPT/GC/H4)

  • H1B Work Permit

Preferred Employment

  • Corp-Corp

  • W2-Permanent

  • W2-Contract

  • Contract to Hire

Employment Type

  • Consulting/Contract

education qualification

  • UG :- - Not Required

  • PG :- - Not Required

Other Information

  • No of position :- ( 1 )

  • Post :- 21st Jun 2025

JOB DETAIL

  • Design, develop, and maintain Databricks Lakehouse solutions sourcing from Cloud platforms such as Azure Synapse and GCP
  • Implement and manage DevOps and CICD workflows using tools like GitHub
  • Apply best practices in test-driven development, code review, branching strategies, and deployment processes
  • Build, manage, and optimize Python packages using tools like setup, poetry, wheels, and artifact registries
  • Develop and optimize data pipelines and workflows in Databricks, utilizing PySpark and Databricks Asset Bundles
  • Manage and query SQL databases (Unity Catalog, SQL Server, Hive, Postgres)
  • Implement orchestration solutions using Databricks Workflows, Airflow, and Dagster
  • Work with event-driven architectures using Kafka, Azure Event Hub, and Google+C4 Cloud Pub/Sub
  • Develop and maintain Change Data Capture (CDC) solutions using tools like Debezium
  • Extensive experience in design and implementation of data migration projects specifically involving Azure Synapse and Databricks Lakehouse
  • Manage cloud storage solutions, including Azure Data Lake Storage and Google Cloud Storage
  • Configure and manage identity and access solutions using Azure Active Directory, including AD Groups, Service Principals, and Managed Identities
  • Effective interactions in Customer for understanding requirements, participating in design discussions and translating requirements into deliverables by working with the development team at Offshore; Effective in collaborating with cross-functional teams across development, operations, and business units; Strong interpersonal skills to build and maintain productive relationships with team members
  • Problem-Solving and Analytical Thinking – Capability to troubleshoot and resolve issues efficiently; Analytical mindset for optimizing workflows and improving system performance
  • Ability to convey complex technical concepts in a clear and concise manner to both technical and non-technical stakeholders; Strong documentation skills for creating process guidelines, technical workflows, and reports

Company Information