As a data engineer at Saras Analytics, you will be responsible for building and maintaining large-scale data pipelines as well as create and data pipelines that deal with large volumes of data.
Primary Responsibilities:
- Experience in Database programming using multiple flavor of SQL and Python.
- Understand and translate data, analytic requirements and functional needs into technical requirements.
- Build and maintain data pipelines to support large scale data management projects.
- Ensure alignment with data strategy and standards of data processing.
- Deploy scalable data pipelines for analytical needs.
- Experience in Big Data ecosystem - on-prem (Hortonworks/MapR) or Cloud (Dataproc/EMR/HDInsight).
- Experience in Hadoop, Pig, SQL, Hive, Sqoop and SparkSQL.
- Experience in any orchestration/workflow tool such as Airflow/Oozie for scheduling pipelines.
- Exposure to latest cloud ETL tools such as Glue/ADF/Dataflow.
- Understand and execute IN memory distributed computing frameworks like Spark (and/or DataBricks) and its parameter tuning, writing optimized queries in Spark.
- Hands-on experience in using Spark Streaming, Kafka and Hbase.
- BE/BS/MTech/MS in computer science or equivalent work experience.
Requirements
- 4 to 6 years of experience in building data processing applications using Hadoop, Spark and NoSQL DB and Hadoop streaming. Exposure to latest cloud ETL tools such as Glue/ADF/Dataflow is a plus.
- Expertise in data structures, distributed computing, manipulating and analyzing complex high-volume data from variety of internal and external sources.
- Experience in building structured and unstructured data pipelines.
- Proficient in programming language such as Python/Scala.
- Good understanding of data analysis techniques.
- Solid hands-on working knowledge of SQL and scripting.
- Good understanding of in relational/dimensional modelling and ETL concepts.
- Understanding of any reporting tools such as Looker, Tableau, Qlikview or PowerBI.
Educational qualifications preferred
- Category: Bachelors Degree, Masters Degree.
- Field specialization: Computer Science.
- Degree: Bachelor of Engineering - BE, Bachelor of Science - BS, Master of Engineering - MEng, Master of Science - MS