Define and implement the data architecture and data strategy aligned with business goals.
Design scalable, cost-effective, and high-performance data solutions using Databricks on AWS, Azure, or GCP.
Establish best practices for Lakehouse Architecture and Delta Lake for optimized data storage, processing, and analytics.
Data Engineering & Integration Architect ETL/ELT pipelines leveraging Databricks Spark, Delta Live Tables, and Databricks Workflows.
Optimize data ingestion from sources like Oracle Fusion Middleware, Web Methods, MuleSoft, and Informatica into Databricks.
Ensure real-time and batch data processing with Apache Spark and Delta Lake.
Work on data integration strategies, ensuring seamless connectivity with enterprise systems (e.g., Salesforce, SAP, ERP, CRM)
Data Governance, Security & Compliance Implement data governance frameworks leveraging Unity Catalog for data lineage, metadata management, and access control.
Ensure compliance with HIPAA, GDPR, and other regulatory standards in life sciences.
Define RBAC (Role-Based Access Control) and enforce data security best practices using Databricks SQL and access policies.
Enable data stewardship and ensure data cataloging for self-service data democratization.
Performance Optimization & Cost Management Optimize Databricks compute clusters (DBU usage) for cost efficiency and performance tuning.
Define and implement query optimization techniques using Photon Engine, Adaptive Query Execution (AQE), and caching strategies.
Monitor Databricks workspace health, job performance, and cost analytics.
AI/ML Enablement & Advanced Analytics Design and support ML pipelines leveraging Databricks ML flow for model tracking and deployment.
Enable AI-driven analytics in genomics, drug discovery, and clinical data processing.
Collaborate with data scientists to operationalize AI/ML models in Databricks.
Collaboration & Stakeholder Alignment Work with business teams, data engineers, AI/ML teams, and IT leadership to align data strategy with enterprise goals.
Collaborate with platform vendors (Databricks, AWS, Azure, GCP, Informatica, Oracle, MuleSoft) for solution architecture and support.
Provide technical leadership, conduct PoCs, and drive Databricks adoption across the organization.
Data Democratization & Self-Service Enablement Implement data sharing frameworks for self-service analytics using Databricks SQL and BI integrations (Power BI, Tableau).
Promote data literacy and empower business users with self-service analytics.
Establish data lineage and cataloging to improve data discoverability and governance.
Migration & Modernization Lead the migration of legacy data platforms (Informatica, Oracle, Hadoop, etc.) to Databricks Lakehouse.
Design a roadmap for cloud modernization, ensuring seamless data transition with minimal disruption.