- Collaborate with stakeholders to understand business requirements and translate them into scalable and efficient data architecture solutions using Databricks.
- Lead the design, development, and implementation of end-to-end data architectures leveraging Databricks Unified Data Analytics Platform to support healthcare analytics use cases.
- Architect data lakes, data warehouses, and data marts using Databricks Delta Lake and other relevant technologies to ensure data integrity, accessibility, and reliability.
- Design and optimize data pipelines, ETL processes, and data integration workflows to facilitate seamless data movement and transformation within the organization.
- Define and enforce data architecture best practices, standards, and guidelines to ensure consistency, reliability, and security of data assets.
- Collaborate with cross-functional teams including data engineers, data scientists, business analysts, and IT professionals to drive the adoption of Databricks and promote a culture of data-driven decision-making.
- Provide technical leadership and mentorship to junior team members, sharing expertise in Databricks and data architecture principles.
- Conduct performance tuning, optimization, and troubleshooting of Databricks workloads to maximize performance and efficiency.
- Stay abreast of emerging technologies, trends, and best practices in data architecture, Databricks, and healthcare analytics, and incorporate them into the organization's data strategy.
Qualifications:
Bachelor's or Master's degree in Computer Science, Information Systems, or related field.
Minimum of 5 years of experience in data architecture, data engineering, or related roles.
Strong proficiency in Databricks, including hands-on experience with Databricks Unified Data Analytics Platform, Delta Lake, and Spark-based data processing.
Experience designing and implementing data architectures for large-scale healthcare organizations or similar domains.
Proficiency in SQL, Python, or other programming languages commonly used in data engineering and analytics.
Solid understanding of data modeling, data warehousing, and ETL processes.
Excellent communication and collaboration skills with the ability to work effectively in cross-functional teams.
Proven track record of leading complex data architecture projects from conception to implementation.
Relevant certifications in Databricks or related technologies is a plus.