Role
Senior Data Engineer
Responsibilities
Writing clean, fully-tested and well-documented code in Python 3.5+ with pandas, NumPy, Dask, TensorFlow, scikit-learn and Django.
- Creating complex data processing pipelines including optimization and user experience.
- Design, develop, test, deploy, support, enhance data integration solutions seamlessly to connect and integrate enterprise systems in an Enterprise Data Platform.
- Working directly with clients to identify pain points and opportunities in pre-existing data pipelines and build or improve clients analytics processes
- Developing and testing models using appropriate tools and technologies and deploying the same in the production environment using continuous delivery practices.
- Working directly with the stakeholders on analytics framework model building, database design and deployment strategies.
- Advising clients on the usage of different distributed storage and computing technologies from the plethora of options available in the ecosystem.
Candidate Profile
3+ years of overall industry experience specifically in data engineering
- 2+ years of experience building and deploying large scale data processing pipelines in a production environment
- Strong experience in building data pipelines and analysis tools using Python and PySpark
- Leverage experience in usage of tools and technologies like, SQL, Python, PySpark, etc. to Extract transform and prepare large scale datasets from various data source systems.