- Heavy Scala exp. is Must to have
- Hands-on coding in Scala for ETL pipelines and building APIs. Must have experience in an Azure environment.
- Current project must healthcare
- One LinkedIn Manager or Lead Reference is Must with submission
- LinkedIn with Photo
Context:
- We have our data lake / platform setup
- NH has tons of raw / unaltered data we receive on a regular basis that makes it's way into the data lake
- Many of the data sources are about the same content (e.g. Centrum might receive medical claim information from over a dozen different insurance companies), but each source sends us the data in their own bespoke format
- We're working on an initiative to have all of this data end up in a standard set of database tables (i.e. data normalization) (e.g. take 12x medical claim spreadsheets/csvs that come on a regular basis and produce a single list of ALL claims
Role Specifications
- Write traditional code and server-less functions using the language best suited for the task, which is primarily Scala. May include development with C# and T-SQL.
- Build APIs, data microservices and ETL pipelines, to share data with internal and external partners and write interfaces to public data sets to enrich our analytics data stores.
- Participate in building and owning a culture of DevOps and Quality Assurance.
- Continuously document your code, framework standards, and team processes.
- Build and support Data Ingestion frameworks deployed in Azure.
The work / skills:
- Overall we're looking for contracting support that would be somewhere between a skilled analyst / junior data engineer.
- The job would be:
- input:
- a document that specifies how the source file format should be manipulated to match our standard format
- a sample source file
- a template for how the normalization code/configuration should be setup
- output:
a normalization configuration for a given source file format