Data Engineer
Buffalo, NY (Remote to Start)
6-12 months
Skillsets:
- Familiarity with spark programming paradigms (batch and stream-processing)
- Understanding of different data abstraction objects used in spark for different use cases, use of optimal data format and other optimization techniques.
- Strong programming skills in at least one of the following languages: Java, Scala. Familiarity with a scripting language like Python as well as Unix/Linux shells.
- Strong knowledge of writing optimized Spark & Hive sql and experience to tune poor performing queries.
- Outstanding programming and debugging skills. Strong knowledge of common algorithms and data structures.
- Good understanding of job scheduling and workflow orchestration through enterprise scheduling tools preferably CA-Automic or Control-M.
- Strong experience with SQL and relational databases like PostgreSQL, MySQL, Teradata, SQL Server and Oracle.
- Should be experienced in Data wrangling
- Should have analytical background to find hidden patterns using data
- Have a good understanding on Data modelling concepts.
- Familiarity with one or more stream processing / queuing technologies like Spark Streaming, Kafka, Kinesis, Flink, etc. preferred.
- Familiarity and prior experience with Agile / Scrum development methodologies.
- Prior Experience deploying to cloud platforms, preferably Azure or AWS Cloud
- Familiarity with any Object-Oriented Programming language
- Prior experience in Continuous Integration/Continuous Delivery tools and pipelines such as Jenkins, Maven, Gradle, etc.
- Experience working in aRegulated industry is preferred.
Vandan Richard