It seems they lack the level of skillsets and although exposed to certain technologies their practical hands-on experience is lacking. Below is a firm set of requirements that the vendor should screen for:
What you Need:
- Must have Experience working in a Data Engineering / Big Data Environment – preferably in a large complex organization
- Must have Ab Initio Experience (at least some, but the more the better)
- Must have solid experience working with Spark
- Must have solid experience with Data Pipelines
- Must have solid ETL experience
- Strong SQL knowledge with strong complex query writing skills
- Cloud experience, preferably AWS
Preferred
- Experience in the distributed event streaming platforms such as Kafka, Kinesis, etc.
- Experience with Cloud/Virtual warehouse systems such as Snowflake/Redshift.
- Distributed computational framework experience
- AWS solutions and data experience preferred – Glue
- Experience with RDBMS and/or NoSQL Database
What You'll Do:
- Provide hands-on practical experience delivering system design, application development, testing, and operational stability
- Become a member of the team responsible for modernizing Data Engineering at the Firm including migrating their legacy data from on-prem into the cloud/data lake