US Citizen
Green Card
Corp-Corp
Consulting/Contract
UG :- - Not Required
PG :- - Not Required
No of position :- ( 1 )
Post :- 30th May 2022
It is the role of the Data engineer to build data in systems, data products and pipelines throughout the enterprise
· Collaborates with subject matter experts, source admins and consumers of data
· Understands usage patterns, toolsets and processes leveraging the data products
· Leverages knowledge of data access methods to relational and NoSql data sources and targets
· Leverages knowledge of Data definition and data manipulation methods and optimization techniques for relational and NoSql data
· Understands metadata available in source and target systems
· Understands data security and privacy rules
· Collaborates with API development team to creating a variety of data services to be exposed to a wider range of consumers
· Performs data transformations and cleansing
· Implements the appropriate integration patterns and methods throughout the data pipeline
· Maintains data quality levels and alerts/triggering, alternate flows and reporting
· Organizes and analyzes raw data for ad-hoc or scheduled requests
· Understands 3rd Party or Industry Standard File Formats (EDI, HL7, NCPDP, etc.)
· Uses tools, techniques and services to migrate data between on-prem and cloud.
· Designs orchestration of data workloads on-prem and cloud instances
· Performs Data prep for ML workflows. May be responsible for assisting data scientists with building efficient ML workflows
· Builds algorithms to leverage in pipelines and analytics
· Develops or manages interface to analytical tools and programs
· Understands the concept ""data temperature"" and movement of data through well-defined heat zones
· Considers volume, velocity, variety, veracity, performance, exception handling and other factors in the data pipeline"