Roles and Responsibilities
Responsibilities :
- Work on the ETL pipeline using open-source tools.
- Extraction of data from various sources using Web and PDF Scrapping.
- Write APIs for extraction using Python, Numpy and Pandas.
- Transformation of data using Python into the desired data format.
- Load the data to AWS using Postman and existing APIs related to the service. (AWS S3, VPC)
- Handle relational database using SQL queries and keeping track of data integrity and ingested data.
- Create new and improve existing infrastructure for ETL workflows.
- Completed Bachelor's in Computer Science or Engineering in IT.
- Candidate can have 4+ years of experience working in data engineering or python development.
- Proficient in Python and modules like Pandas, Numpy.
- Experience in RDBMS and SQL.
- Proficient in writing APIs and web scrapping is good to have.
- Candidate should sound communication skills and should be a team player with curious mindset.