Responsibilities:
- Collaborate with cross-functional teams to understand data requirements and establish seamless connections with external vendors
- Develop and implement robust data acquisition strategies, ensuring data quality, integrity, and security.
- Lead the design and implementation of data cleansing and transformation process to align vendor data with internal standards and structures
- Utilize internal tools and SDKs for efficient data integration and processing, focusing on scalability and performance optimization
- Implement and manage agents for monitoring, logging, and automation within AWS environments
- Work with Amazon SNS for efficient notification and communication workflows
- Work with other AWS services, including S3, Athena, and Glue, to optimize data storage, querying, and processing.
- Lead the integration efforts with the internal Archival Service Platform for seamless data purging and lifecycle management
- Collaborate with the data engineering team to continuously improve data integration pipelines, ensuring adaptability to evolving business needs.
Skills and Qualifications:
- Bachelor’s degree in Computer Science, Information Technology, or related filed.
- In-depth knowledge and hands-on experience with AWS services, including SNS, S3, Athena, Glue, and Lambda
- Expertise in data cleansing, transformation, and enrichment techniques
- Experience in deploying and managing Apache Kafka
- Proficient in programing language Java
- Excellent communication and collaboration skills
- Strong problem-solving skills and meticulous attention to detail.
Preferred Qualifications:
- AWS Certification.
- Advance expertise in Athena queries and optimization.
- Knowledge of data governance and compliance standards.