DISH is looking for a Big Data Engineering Lead that will work on the collecting, storing, processing, and analyzing of huge sets of data. The primary focus will be on choosing optimal solutions to use for these purposes, then maintaining, implementing, and monitoring them. You will also be responsible for integrating them with the architecture used across the company. This role is critical for the timely development of machine learning models.
Essential Duties and Responsibilities
- Design and maintain data pipelines and databases.
- Lead data curation efforts and ensure the quality of datasets.
- Manage data-related software systems.
- Maintain libraries and repositories for data processing and management.
- Manage documentation for datasets and data manipulation tools.
- Selecting and integrating any Big Data tools and frameworks required to provide requested capabilities
- Monitoring performance and advising any necessary infrastructure changes
- Defining data retention policies
Minimum Qualifications, Education, and Experience
- Bachelor’s degree in related field.
- At least five (5) years of data management experience.
- Expertise in scripting languages, such as Python.
- Experience with Big Data ML toolkits, such as Apache Spark.
- Hands-on knowledge of AWS services
- Good knowledge of Big Data querying tools, such as Pig, Hive, and Impala
- Familiarity with database languages and other big data tools.
- Experience with telecom/network data
- Experience with integration of data from multiple data sources
- Experience with NoSQL databases, such as HBase, Cassandra, MongoDB
- Strong individual team contributor and collaborator.
- Excellent written and oral communication skills.