- Develop set processes for data mining, data modeling, and data production
- Able to Install/update disaster recovery procedures
- Recommend different ways to constantly improve data reliability and quality.
- Contribute to the implementation of complex programs
- Recommend different ways to constantly improve data quality.
- Develop and maintain documentation related to all assigned systems and projects
- Experience in writing queries and functions running over massive rows of data running in a distributed query engine
- Perform root cause analysis to identify permanent resolutions
- Ability to learn new concepts and or technologies that are required to perform work
- Research new uses for existing data.
Job Qualifications
We are seeking for Data Engineer for a long term contract position.
- Must have Azure Analytics platform knowledge
- Deep experience in developing data processing tasks using PySpark such as reading data from external sources, merge data, perform data enrichment and load in to target data destinations.
- Experience in relational databases (MS SQL, Teradata, Oracle, DB2) and good knowledge of SQL
- Experience in Azure Cloud technology and Big Data – Azure Databricks & ADLS
- Strong background in data architecture, data mining, and data cleansing, and using AI & ML techniques to analyze data.
- Strong analytical skills and statistical modeling to collect, analyze, and interpret data sets, Agile and DevOps Team Experience a plus