Job Title
Data Engineer
Department
Information Technology
Location
India
Area
Data
Summary
Reporting to the Manager of Data Engineering, the Data Engineer is someone who enjoys challenging work in a fast-paced environment and solving complex business problems with data and analytics, enabling our customers in their pursuit of better healthcare outcomes for their members.
Essential Functions
- Design, develop and implement data pipelines to ingest, store, and publish customer data
- Orchestrate and schedule data pipeline jobs to optimize run times and reduce compute utilization thresholds
- Significant participation in analysis, interpretation, and translation of complex health plan data, issues, trends, and relationships to provide our customers detailed insights into their data
- Analyze and interpret complex data on source and target systems, identify the gaps, and provide solutions
- Ability to development full end-to-end data pipelines that consist of complex SQL queries, stored procedures, triggers, and detailed logging
- Optimizes SSIS package execution and SQL execution to minimize load times.
Ensures data integrity throughout the ETL process and appropriately handle errors
- Work closely with Scrum team members using Agile processes to iteratively develop and improve ETL pipelines.
- Proactively provide feedback and process improvement recommendations to the team and management
- Participate in cloud migration development initiatives to move data and technologies to Azure
- Integrate with and implement detailed logging and status alerts to improve troubleshooting and investigation of data pipelines
- Build, implement and design data intelligence layouts for reports, dashboards, and other data visualizations
- Create, modify and integrate reports with SSRS
- Create, modify and integrate reports with Power BI
- Leverage DAX expressions to optimize performance and interactions with the data
- Participate in the migration from legacy reports and dashboards (SSRS) to Power BI
- Work with data engineers to develop and define data pipelines, data models and data sets
- Apply data warehousing best practices to define, design, and develop data transformation rules.
- Perform ETL/ELT design, development, and support for moving large data volumes from various sources into the various destinations after significant cleansing, transformation, and processing.
- Setup and manage Power BI Gateways for onprem data interactions
- Design and implement real-time and near real-time dashboards and reports, the associated data pipelines and the necessary DataMart
-
-
Skills
- Full mastery of entire MSBI platform stack (SSIS, SSRS, SSAS, T-SQL, MSSQL Server)
- Knowledge of ETL/ELT technologies (Python, Scala, dbt)
- Knowledge of cloud orchestration tools (Airflow, ADF)
- Knowledge of cloud data visualization and report technologies (PowerBI, Tableau, Looker)
- Knowledge of cloud compute technologies (Spark, Databricks, Snowpark)
- Knowledge of federated MPP sql query engine like Trino or Starburst
- Knowledge of US healthcare
- Knowledge of Building/Maintaining Reporting Dashboard tools.
Experience
- 3+ years of experience with data engineering pipelines in the cloud or with MSBI
- 3+ years of experience designing and developing end-to-end data pipelines with SSIS, SSRS and T-SQL
- 2+ years of experience in data migration or platform transformation.
- Azure or AWS Cloud certifications
Education
- Bachelors degree in Computer Science, Engineering, Mathematics or related discipline