Proven expert knowledge and hands-on experience in designing and implementing Data pipelines using Hadoop, pySpark, UNIX shellscript
Good understanding of ETL process and integration patterns using Python on Spark.
Experience in translating business requirements to software components
Experience with SQL and relational databases (Teradata, MSSQL, Hive, Snowflake) preferred.
Experience working in an Agile software delivery environment, with exposure to continuous integration and continuous delivery (CI/CD) tools for deployment.
Expertise in performance tuning of HQLs, pySpark jobs.
Passionate about technology and delivering solutions to solve business problems using event driven and/or scale-out architectures
Great collaboration and interpersonal skills
Creative problem solving and a curiosity fueled by keeping up with advanced technology and industry trends
Ability to work with team members and lead by example in code, feature development, and knowledge sharing
Bachelor's degree or Master's degree in Computer science, Mathematics, or any other quantitative discipline. Nice to have skills
Experience in cloud technologies and practices, building applications using services such as Snowflake, AWS, GCP etc.
Experience in scheduling tools like Control - M, ESP