Design, develop, and launch extremely efficient and reliable data pipelines to move & transform data, and to provide intuitive analytics to our partner teams.
Make data more discoverable and easy to use for Data Scientists and Analysts across the company.
Collaborate with other engineers and Data Scientists to discover the best solutions
Support your colleagues by reviewing code and designs.
Diagnose and solve issues in our existing data pipelines and envision and build their successors.
Candidate must have hands-on experience in SQL, HDFS, Big data , Spark/Scala & Data analysis and profiling skills
Qualifications:
5+ years’ experience with highly scalable, high performance and high availability server development
2 years of work or educational experience in big data.
Experience with distributed processing and messaging systems, including Spark, Akka, Kafka (deploying and running in production), Pub/sub, Hive/pig, Mapreduce, etc.
Demonstrate clear and concise communication and data-driven decision-making capability
Expertise in some or all of the following:
Data Pipelines
Data Warehousing
Statistics
Strong understanding of SQL
Broad knowledge of the data infrastructure ecosystem
Solid background in algorithms, data structures, and object-oriented programming concepts
B.S. and/or M.S. in Computer Science or a related technical field, or equivalent experience