Tagcor

Databricks Engineer

Lead the design and implementation of scalable data architectures using Spark in Scala and Databricks CLI.
Experience in supporting applications and handling Support engagement.
Ensuring supporting of applications under Databricks + Pyspark Python AWS as per schedule.
Providing on call support over weekend and holidays. Ensuring handshake with offshore support team
Be responsible for the integration of Amazon S3 for efficient data storage and retrieval.
Guide in Python programming to develop and optimize data processing workflows.
Use Databricks SQL to build and manage sophisticated queries for data analysis.
Implement and lead Databricks Delta Lake for reliable and efficient data storage solutions.
Develop and maintain Databricks Workflows to automate data processing tasks.
Apply PySpark to build and optimize large-scale data processing applications.
Collaborate with multi-functional teams to ensure data architecture aligns with business requirements.
Ensure data security and compliance with industry standards and regulations.
Supervise and solve data pipelines to ensure smooth operation and minimal downtime.
Provide technical guidance and mentorship to junior team members.
Stay updated with the latest industry trends and technologies to continuously improve data architecture.
Contribute to the overall success of the company by delivering high-quality data solutions that drive business growth.