As a Data Engineer, you will be responsible for validating the data between two systems, ensuring accuracy and consistency.
Implement data quality checks and monitoring processes.
Design and Develop data entity validation scripts using python.
Validate and reconcile data between two systems, ensuring accuracy and consistency.
Collaborate with cross-functional teams to identify and resolve data discrepancies and issues.
Develop and implement data quality checks and monitoring processes to ensure data integrity.
Design and maintain data pipelines and ETL processes to extract, transform, and load data from various sources.
Optimize data storage and retrieval processes to improve performance and efficiency.
Perform data analysis and profiling to identify data quality issues and recommend solutions.
Develop and maintain documentation for data validation processes, data mappings, and data lineage.
Stay up to date with industry trends and best practices in data engineering and data validation.
Support migration of North American OPCOs, enabling data validation on the new platform.
Gather business requirements from key client stakeholders and translate these into technical tracking specifications based on standards and industry best practice.
Required Skills
Strong knowledge of IPAAS and data validation for migration of applications.
Demonstrable experience in delivering end to end data validation.
Confident understanding of web development and its languages and possess Python proficiency.
Proactive and highly organized with strong time management and planning skills.
Able to change direction and work on multiple projects across a wide range of topics.
[Desired] Knowledge of Python/Google BigQuery architecture.