- Data Modeling: Design and develop data models that accurately represent the structure and relationships of data within the organization. This includes conceptual, logical, and physical data models.
- Data Integration: Collaborate with data engineers and ETL developers to ensure that data models align with data integration and transformation processes.
- Data Quality and Governance: Establish and enforce data quality standards and data governance practices to ensure data consistency and reliability. This may involve defining data validation rules and data cleansing processes.
- Schema Design: Create and maintain data schemas for databases and data warehouses within the Databricks environment, ensuring optimal performance and accessibility.
- Optimization: Continuously review and optimize data models to improve query performance, data storage efficiency, and scalability.
- Documentation: Maintain documentation for data models, schemas, and related processes to ensure that stakeholders have a clear understanding of the data structure.
- Data Security and Compliance: Implement and adhere to data security and compliance standards, ensuring that sensitive data is handled and stored securely and in accordance with relevant regulations.
- Collaboration: Work closely with data scientists, analysts, and other stakeholders to understand their data requirements and provide them with data models that facilitate their work.
- Automation: Automate routine data modeling tasks and ensure consistency and repeatability in the data modeling process.
Qualifications:
- A bachelor's or master's degree in computer science, data modeling, data management, or a related field.
- Proficiency in data modeling tools and methodologies, such as ER modeling, dimensional modeling, and data modeling standards (e.g., SQL, UML).
- Experience with database systems, data warehousing, and ETL processes.
- Familiarity with Databricks' platform and its related technologies is often preferred.
- Knowledge of data governance and compliance standards.
- Strong analytical and problem-solving skills.
- Excellent communication and collaboration skills to work effectively with cross-functional teams.
Understanding of data security best practices.