5 years of experience delivering complex enterprise-wide information technology solutions.
5 years of experience delivering ETL, data warehouse and data analytics capabilities on big-data architecture such as Hadoop.
5 years of Apache Spark design and development experience using Scala, Java, Python or Data Frames with Resilient Distributed Datasets (RDDs), Parquet or ORC file formats.
6 years of ETL (Extract, Transform, Load) Programming experience.
2 years of Kafka or equivalent experience.
2 years of NoSQL DB like Couchbase/MongoDB experience.
5 experience working with complex SQLs and performance tuning.
Responsibilities:
Design and implement automated spark-based framework to facilitate data ingestion, transformation, and consumption.
Collaborate with application partners, Architects, Data Analysts and Modelers to build scalable and performant data solutions.
Effectively work in a hybrid environment where legacy ETL and Data Warehouse applications and new big-data applications co-exist.