Develop and maintain data pipelines using Python and Databricks to ingest, transform, and load data from various sources into data warehouses and data lakes.
Design and implement efficient data processing and analysis solutions using Apache Spark and other big data technologies.
Strong proficiency in Python programming language, including libraries like Pandas, NumPy, and Scikit-learn.
Solid understanding of data engineering principles, including data ingestion, transformation, and loading.
Hands-on experience with Databricks platform and Apache Spark.
Experience with SQL and data modeling.
Knowledge of cloud platforms (AWS, Azure, or GCP) is a plus.
Familiarity with version control systems (Git) and CI/CD pipelines.