US Citizen
Green Card
EAD (OPT/CPT/GC/H4)
H1B Work Permit
Corp-Corp
W2-Permanent
W2-Contract
Contract to Hire
Consulting/Contract
UG :- - Not Required
PG :- - Not Required
No of position :- ( 1 )
Post :- 14th Dec 2023
Bigdata with Spark in Python, SQL, Airflow with experience from 4-9 years (Individual contributor role) with Client's Vancouver Print R&D Center for Data Engineering. Client is keen on establishing a platform of capabilities which enables cost effective.
Years of experience needed –
4-9 years’ experience
Technical Skills:
· Database: RDBMS-SQL, Redshift
· Cloud: Data Bricks Cloud, Unity catalog, AWS – CLI, S3, IAM role, secrets, Lambda
· Programming languages: Python, Spark, SQL
· Scheduler: Airflow
· Containers: Dockers with Kubernetes
· CI/CD: Azure CI/CD, Terraform, Github
· Version control: Git
· Logging: Splunk
· Strong knowledge & hands-on experience in Spark with Python (2+ years)
· Extensive experience designing, developing and deploying Bigdata pipelines.
· Strong hands on in SQL development (2+ years) and in-depth understanding of optimization and tuning techniques in SQL with Redshift
· Must have good hands experience in Python programming (2+ years) along with unit testing framework.
· Develop and schedule jobs in Airflow (1+ years)
· Troubleshooting/handling and monitoring airflow operators is advantage.
· Exposure to Kubernetes or other container orchestration is an advantage.
· Good knowledge of industry wide technology data strategies and best practices working with very large data sets and unstructured data, including data, cleansing /transformation.
Certifications Needed:
· Bachelor’s or master’s degree in computer science, Information Systems, Engineering or equivalent.
· Develop and schedule jobs in Airflow (1+ years)
· Experience in developing & integrating in Databricks notebooks using Spark (2+ years) and build and deployed pipelines on Data Bricks Cloud – Delta, Unity catalog.
· Good Experience in AWS stack (1+ years) AWS – CLI, S3, IAM role, secrets, Lambda
· Experience in working with databases such as Redshift, DBT