Tagcor

Data Engineer

Design and Develop data ingestion pipelines and processes based on requirements in Python and PySpark.
Create error handing, exception management and data quality routines to expose the anomalies in the data.
Profile and analyze data to identify gaps and potential data quality issues.
Identifies relationships between disparate data sources.
Uses Python, Databricks and Spark to code the data Engineering routines.
Perform unit and integration testing.
Works with the group of data scientists and business SMEs to get the requirements and present the details in data.
Designs and jointly develops the data architecture with data architect and ensures security and maintenance.
Explores suitable options, designs, and creates data pipeline (data lake / data warehouses) for specific analytical solutions.
Identifies gaps and implements solutions for data security, quality and automation of processes.
Builds data tools and products for effort automation and easy data accessibility.
Supports maintenance, bug fixing and performance analysis along data pipeline.
Diagnoses existing architecture and data maturity and identifies gaps.