• Us Citizen

• Green Card

• Corp-Corp

• Consulting/Contract

• UG :- - Not Required

• PG :- - Not Required

• No of position :- ( 1 )

• Post :- 15th Jul 2021

Azure Databricks is unified data analytics’ platform for large volume data processing and engineering.
 

Scope of work:

Data Services (Data Analytics organization) is looking for a senior Data Engineer with experience on ingestion of structured and unstructured data from multitude of sources into the Enterprise Data Lake infrastructure.
The Engineer performs cleansing, transformation, and standardization of inbound data into Databricks’ Delta tables preferably using Spark and/or Python.
The Engineer performs ‘feature engineering’ of variables in preparation for data exploration and input to Machine Learning’s models built by Data Scientists, using Spark-SQL and DataFrames.
Experience in Microsoft Azure’s Databricks, Data Factory, Data Lake Store (ADLS).
The requirement is for a hands-on lead developer with enough seniority to perform his/her coding and soft skills to coach other Data Engineers about development patterns, best practices, and design methodologies.
Data services’ expectation is for someone who can participate and contribute to overall Data Engineering solution design based on his/her in-depth technical expertise.
 

Out of Scope:

This developer is not expected to build Machine Learning’s models or participating in Statistical programming work.
 

Required technical skillsets to make the engagement a success:

Databricks experience on Azure or any virtualized environment.
Excellent in writing code using PySpark, Spark-SQL or Python.
Scripting language, such as Linux/Unix Shell, Windows PowerShell, and Azure CLI.
Knowledge of SQL Server, ETL tools like SSIS, Azure Data Factory is a plus.
Other preferred skillsets:
Agile methodology to product delivery.
Programming skills on .NET or Java.
Any NoSQL database like CosmosDB, MongoDB, Cassandra, Redis etc.
Experience building Microservices and Web-APIs