Hands on experience in Java, Spark, Spark SQL, Scala, CDH/Hadoop, HBase, Linux, Shell Script, Python
Good knowledge of big data querying tools, such as Pig, Hive & Impala
Good experience in debugging issue using the Hadoop & Spark Log files
Experience in Kafka & rest services.
Experience in AWS cloud
Strong background in Java-based Software Design & Development
Strong experience Designing, creating and maintaining Scala-based applications
Develop and implement data pipelines that extracts, transforms, and loads data into an information product that helps to inform the organization in reaching strategic goals
Working experience on ingesting, storing, processing, and analyzing large data sets
Create scalable and high-performance web services for tracking data
Ability to learn quickly, possessing analytical mindset
Strong analytical skills and problem-solving aptitude
Strong customer communication and stakeholder engagement
Experience leading & coordinating team in onshore -offshore development model for 3-5 team members
Experience working in Agile based project development
The evaluation must happen on the below mandatory skills
a. Spark( RDD, Transformation, UDAF, Partition, DAG- Usage)
b. SparkSQL
c. Spark Streaming using kafka and Kafka for Batch process
d. Spark performance tuning
e. Scala Oops
f. Hive.
g. Connect to NoSQL Database / Columnar database/ Sql server using Spark Framework