• Us Citizen

• Green Card

• EAD (OPT/CPT/GC/H4)

• H1B Work Permit

• Corp-Corp

• Consulting/Contract

• UG :- - Not Required

• PG :- - Not Required

• No of position :- ( 1 )

• Post :- 25th Nov 2020

1.    Check for handling Real-time High volume,  high input data ingestion in the capacity of Architect – validate against the Use Case they have implemented
2.    To validate the relevance on Tech Stack - check what is the 
•    Streaming platform used – Kenis or Kafka [either 1 should be ok]
•    Processing Framework usage – check for Spark or any relevant ones
•    Target data store used for Data Ingestion
•    Any Cloud experience

Skills - Experience and Requirements
A successful Solution Lead will have the following:
•        Should have prior experience in working as a Data warehouse/Big Data architect.
•        Experience in advanced Apache Spark processing framework, spark programming languages such as Scala/Python/Advanced Java with sound knowledge in shell scripting.
•        Should have experience in both functional programming and Spark SQL programming dealing with processing terabytes of data
•        Specifically, this experience must be in writing Big Data data engineering jobs for large scale data integration in AWS. Prior experience in writing Machine Learning data pipelines using Spark programming language is an added advantage.
•        Advanced SQL experience including SQL performance tuning is a must.
•        Should have worked on other big data frameworks such as MapReduce, HDFS, Hive/Impala, AWS Athena.
•        Experience in logical & physical table design in Big Data environment to suite processing frameworks
•        Knowledge of using, setting up and tuning resource management framework such as Yarn, Mesos or standalone spark.
•        Experience in writing spark streaming jobs (producers/consumers) using Apache Kafka or AWS Kinesis is required
•        Should have knowledge in variety of data platforms such as Redshift, S3, Teradata, Hbase, MySQL/Postgres, MongoDB
•        Experience in AWS services such as EMR, Glue, S3, Athena, DynamoDB, IAM, Lambda, Cloud watch and Data pipeline
•        Must have used the technologies for deploying specific solutions in the area of Big Data and Machine learning.
•        Experience in AWS cloud transformation projects are required.
•        Telecommunication experience is an added advantage

Krishna Kanth

IDC Technologies

Ph: 408-385-2661

Email: krishna.k@idctechnologies.com