Roles and Responsibilities:
- Develop and maintain data pipelines using BigData processes.
- Focus on ingesting, storing, processing, and analyzing large datasets
- Work closely with data science team to implement data analytics pipelines.
- Help define data governance policies and support data-versioning processes.
- Maintain security and data privacy, working closely with data protection officer.
- Analyze vast number of data stores to uncover insights
- Enhancing data collection procedures to include all relevant information for developing analytic systems.
- Processing, cleansing, and validating the integrity of data to be used for analysis.
- Analyzing large amounts of information to find patterns and solutions.
- Developing prediction systems and machine learning algorithms
- Presenting results in a clear manner.
- Propose solutions and strategies to tackle business challenges
- Collaborate with Business and IT teams.
Required skills and qualifications:
- Experience with Python, Spark, and Hive
- Understanding of data-warehousing and data-modeling techniques
- Knowledge of industry-wide visualization and analytics tools (ex: Tableau, R)
- Strong data engineering skills with Azure cloud platform
- Experience with streaming frameworks such as Kafka
- Knowledge of Core Java, Linux, SQL, and any scripting language
- Good interpersonal skills and positive attitude.
- Strong Software Engineering Background
- Hands-on experience with data science tools
- Problem-solving aptitude
- Analytical mind and great business sense
- Degree in Computer Science, Engineering or relevant field is preferred.