We are currently recruiting for a Data Engineer with solid experience in data engineering and a passion for Big Data.You will have the chance to join a team of Data Engineers and Data Analysts and be part of the growth and development of it, participate in a large variety of exciting and challenging projects and work with our exceptional team of Technology professionals.
Key Responsibilities:
- Development, customize and manage integration tools, databases, warehouses, and analytical systems with the use of data related instruments/instances
- Create and run complex queries and automation scripts for operational data processing.
- Test the reliability and performance of each part of a system and cooperate with the testing team
- Deploying data models into production environments. This entails providing the model with data stored in a warehouse or coming directly from sources, configuring data attributes, managing computing resources, setting up monitoring tools, etc.
- Responsible for setting up tools to view data, generate reports, and create visuals
- Monitoring the overall performance and stability of the system. Adjust and adapt the automated pipeline as data/models/requirements change.
- Mentor and train colleagues where necessary by helping them learn and improve their skills, as well as innovate and iterate on best practices.
- Solve complex issues with minimal supervision.
- Make improvement and process recommendations that have an impact on the business.
Qualification:
- BE/ME/MCA in Computer Engineering.
Technology Stack Used & Required Knowledge:
- Excellent understanding of ETL cycle
- Experience in using Python/ PySpark and/or Scala for data engineering.
- Understanding of data types/ handling of different data models.
- Desirable to have experience with Spark, Kafka, Flask
- Experience with the Microsoft Azure or AWS data management tools such as Azure Datafactory, Datalake, and Databricks, or AWS Snowflake is a plus
- Knowledge of data visualization tools is a plus (PowerBI, Tableau).
- Good scripting and programming skills.