Data Engineer
Position Type :Contract
Location: Remote/Buffalo, NY
Job description:
Some of the most common responsibilities include:
- Participate in detailed technical design, development, implementation and support of Data applications.
- Develop, construct, test, automate and maintain Data Pipeline for enterprise and non-enterprise platforms
- Maintain standards compliance and ensure development artifacts are in alignment with patterns/ frameworks designed by software engineering teams.
- Experience in solving the business problem with the right Data architecture
- Identify ways to improve data reliability, efficiency and quality
- Use large data sets to address business issues
- Prepare data for predictive and prescriptive modeling
- Leverage data to discover tasks that can be automated
- Deliver updates to stakeholders based on analytics
Skillsets:
- Familiarity with spark programming paradigms (batch and stream-processing)
- Understanding of different data abstraction objects used in spark for different use cases, use of optimal data format and other optimization techniques.
- Strong programming skills in at least one of the following languages: Java, Scala. Familiarity with a scripting language like Python as well as Unix/Linux shells.
- Strong knowledge of writing optimized Spark & Hive sql and experience to tune poor performing queries.
- Outstanding programming and debugging skills. Strong knowledge of common algorithms and data structures.
- Good understanding of job scheduling and workflow orchestration through enterprise scheduling tools preferably CA-Automic or Control-M.
- Strong experience with SQL and relational databases like PostgreSQL, MySQL, Teradata, SQL Server and Oracle.
- Should be experienced in Data wrangling
- Should have analytical background to find hidden patterns using data
- Have a good understanding on Data modelling concepts.
- Familiarity with one or more stream processing / queuing technologies like Spark Streaming, Kafka, Kinesis, Flink, etc. preferred.
- Familiarity and prior experience with Agile / Scrum development methodologies.
- Prior Experience deploying to cloud platforms, preferably Azure or AWS Cloud
- Familiarity with any Object-Oriented Programming language
- Prior experience in Continuous Integration/Continuous Delivery tools and pipelines such as Jenkins, Maven, Gradle, etc.
- Experience working in a Regulated industry is preferred.
Please share your available and relevant consultants at the earliest.
Looking forward to hearing from you.
Thanks & Regards,
Virendra Singh
5555 Glenridge Connector Ste 200
Atlanta
GA 30342
(404) 999-5802