Required Skills
- Half architecture, half hands on coding
- Must have extensive python
- Must have extensive DBT
- Must have extensive Snowflake
- Must be a ‘driver’ and assertive
Project Description
They are doing a massive cloud migration project. On prem to cloud. They will be doing data visualizations and working with Greenfield.
As a key member of the Data engineering team, this Data Engineer will work on diverse data technologies such as Azure Cloud, Snowflake, Streamsets, dbt, Starburst, data ops, data observability and others to build insightful, scalable, and robust data pipelines that feed our various analytics platforms.
Duties/Day to Day Overview
- Data Modeling (50%): facts and dimensions, looking at tables, breaking them down.
- Creating one big table. White table.
- Denormalize.
- All the modeling concepts in the resume.
- Have they done snowflake schema.
- Star schema.
- Need a Data Modeler who also knows how to data architect.
- Data architect – how is my data going to flow. What are my different layers.
- Real time streaming? Kafkfa. If it is batch streaming, will be using a different product.
- Design thinking experience.
- 50% architect. 50% modeler. (maybe 20% coding).
- Star schema. Data products. Data strategy. Cloud data warehouse architecture
Top Requirements
(Must haves)
- Data Strategy and Data Architecture is Biggest NEED.
- Need someone to look at holistic approach at building modern data platform.
- Experience modernizing the Data Ingestion. Data Transformation. Making data Available for insights.
- 10+ years in-depth data engineering experience and execution of data pipelines, data ops, data observability, scripting and SQL queries
- 5+ years proven data architecture experience - must have demonstrable experience with data architecture, accountable for data standards, designing data models for data warehousing and modern analytics use-cases (e.g., from operational data store to semantic models)
- DBA is must have, they want someone with a architect background.
- At least 3 years’ experience in modern data architecture that support advanced analytics including Snowflake, Azure, etc. Experience with Snowflake and other Cloud Data Warehousing / Data Lake preferred.
- Expert in engineering data pipelines using various data technologies – ETL/ELT, big data technologies (Hive, Spark) on large-scale data sets demonstrated through years of experience.
- 5+ years hands on data warehouse design, development, and data modeling best practices for modern data architectures
- Highly proficient in at least one of these programming languages: Java, Python
- Experience with modern data modelling tools, data preparation tools
- Experience with adding data lineage, technical glossary from data pipelines to data catalog tools.
- Highly proficient in Data analysis – analyzing SQL, Python scripts, ETL/ELT transformation scripts.
- Highly skilled in data orchestration with experience in tools like Ctrl-M, Apache Airflow. Hands on DevOps/Data Ops experience required.
- Knowledge/working experience in reporting tools such as MicroStrategy, Power BI
- Experience with Streamsets, DBT preferred.
- Must have good soft skills.
Additional Qualifications
(Nice to Haves)
- Experience working in an Agile Environment preferred and experience with Waterfall
- Bachelor’s degree in computer science, Information Systems, Engineering, Business Analytics, Business Management
- Self-driven individual with the ability to work independently or as part of a project team.
- SIGMA EXPERIENCE HUGE BONUS
- Retail experience
- Streaming/Kafka
Additional Notes
Does not come from a Finance or Healthcare background!!!