Data Ops Lead is responsible for the support of Big Data, Machine Learning, Data Warehouse and ETLsolutions across the company supporting Senior Data & ML Engineering Manager. The ideal candidate will have at least 8 years of relevant experience, plus experience managing multiple. Excellent knowledge of Big Data tools, database design, SQL, data warehousing, and data science solutions is necessary.
Job Scope:
Responsible for the understanding and supporting data platform deliverables
Manage a team of Data Ops Engineers
Supporte the Big Data architecture
Support and enhance AI (Artificial Intelligence) data science applications
Deep understanding of ETL methodologies, Big Data, API and Data Warehousing principles, approaches, technologies, and architectures including the concepts, designs and usage of data warehouses and data marts
Works closely with business and technicalteams to understand, support, troubleshoot, and test ETL processes
Work with Agile methodologies, CI/CD and DevOps strategy
Review and enhance the data flow strategy ensuring data governance policies are followed
Support and troubleshoot Databricks jobs including coding in python and SQL
Troubleshooting any bug\\issue in the system and identifying the appropriate fix
Responsible for the Integrity, Maintenance, and changes to the databases.
Maximizing applications performance by monitoring performance, troubleshooting database
MINIMUM QUALIFICATIONS, EXPERIENCE AND SKILLS REQUIREMENTS
Qualifications
Essential:
Bachelor\'s degree in Business Administration/Informatics or any relevant field or equivalent work experience accepted
Microsoft certification in Big Data - Azure or similar
Desirable:
Master\'s degree in Business Administration/Informatics or any relevant field or equivalent work experience is desirable
Certification in Business Intelligence (Tableau or similar)
Years of Experience
Essential:
6+ years of experience within particular area of expertise
5+ years of experience in Big Data architecture and tools
5+ years of experience in ETL tools (Talend, Azure DF or similar)
3+ years of experience in Azure Databricks
3+ years of experience of development using Python, SQL and/or Java
3+ years of experience of data science delivery (Azure ML, Databricks and/or python)
Desirable:
8+ years of experience within particular area of expertise
3+ years of experience in Data Visualization tools (Tableau or similar)
3+ years of experience leading development team as a manager or team lead
Skills
Essential:
Strong knowledge and expertise in Big Data tools (Databricks, Hadoop, Spark, Azure Data Lake, Hive, Pig, etc)
Experience with Talend
Demonstrable, strong analysis skills are necessary
Programming skills utilizing Python, SQL
Experience with data science solutions (Azure ML, Databricks and/or python)
Experience with Microsoft Azure Data Factory
Demonstrable, strong expertise in data modeling
An understanding of utilizing Visual Studio and Team Foundation Server\\GitHub
Experience with MS Azure Cloud
Experience with data modeling tools such as Erwin
Experience with customer profiling and MDM solutions