Technical Lead Data
Abu Dhabi, United Arab Emirates
Job Description
Overview:
Group 42 is an Abu Dhabi based artificial intelligence (AI) and cloud computing company, uniquely positioned in the national ecosystem to develop and deploy holistic and scalable AI solutions.
G42 Healthcare is committed to developing a world-class, sustainable healthcare sector in the UAE and wider region. At the forefront in the battle against the pandemic, G42 Healthcare partnered with Abu Dhabi authorities to develop a massive throughput laboratory in 14 days and spearheaded the world\xe2\x80\x99s first Phase 3 clinical trial of COVID-19 inactivated vaccine. Beyond Covid-19, G42 Healthcare is also developing a program of activities to support the health of future generations \xe2\x80\x93 ranging from genomics, imaging and diagnostics to digitization programs, manufacturing and cutting-edge research.
About the role:
About the role:
We are seeking an experienced Technical Lead in Data Engineer to join our team to design, help architect and build a scalable and secure health data platform. In this role, you will architect, design, build and optimize data pipelines (batch & streaming) for big data systems. Extracting, Analyzing and Modeling of rich & diverse health data sets. As a tech lead you would take end to end ownership, guide senior data engineers and data engineers, work with architects and stakeholders to build scalable, reliable, resilient data platform components that will help our stakeholders like product management, project delivery management, data scientists and bioinformaticians solve problems. Responsibilities:
- Take ownership and is completely responsible for delivery of data solution.
- Communicate clearly to all stakeholders to help narrow down the requirements and split problems into executable chunks of work.
- Lead a team of small group software engineers, providing technical guidance and fostering a collaborative and innovative work environment.
- Working closely with the Product Manager and Architect to ensure technical solutions are appropriately designed, implemented with support in mind.
- Supporting your team & wider engineering community, instilling technical excellence and confidence, (i.e. growing our technology capabilities).
- Architecture & Design: support architects by contributing to options analysis, presenting solutions and owning the implementation.
- Ability to present technical solutions to the team, tailoring the messaging based on the audience.
- Proven technical skills and experience of the technology stack (including tooling) that is used by the team.
- Comfortable rolling up their sleeves and actively participate in coding activities such as feature development and bugs resolution, approximately 50% of their time.
- A trusted partner within the engineering community, able to provide support, guidance and recommendations as and when challenges arise.
- Understands how automation supports successful delivery, always thinking about how to be reduce time-to-market (being lazy).
- Provides some support and guidance to the PM & EM with estimations for the more complex areas of work.
- Takes an active role in supporting (the team and EM) hiring (especially technical/team rounds).
- Support new team members onboard (development practices, architecture, setup, tooling etc).
- Regularly spends time with other engineers to support their career goals. The Engineer and EM will create OKRs/goals and the Technical Lead will help support, mentor engineers to meet them (e.g. pair programming, discussions, technical workshops etc).
- Partners collaboratively with Product and the Engineering Manager to manage the scope (e.g. is it clear?) and deliverables aligned to the product roadmap. Is able to clearly understand the why (can challenge when needed) and can articulate the how & when (current and future work).
- Motivates engineers and the team, bringing positive energy. Uses risks, challenges and issues as an opportunity to learn and also teach.
- Represents the team in tech community meetings and forums.
- Supports the Engineering Manager\'s culture and environment design. Ensuring that they are good representatives of good culture.
- Fosters an environment of collaboration between all team members, open discussions and team decisions (even when conflict/differing opinions).
- Actively supports the Engineering Manager and the team to find ways to improve the delivery process, especially around automation (e.g. finding ways to reduce the test time).
- Owns the technical implementation for the area the team is responsible for. Mentors & supports engineers within the team.
- Has a deep commitment to data quality, quality of reports and analytics.
- Capacity Planning: Assess the workload and capacity of team members regularly, ensuring a balanced distribution of tasks and responsibilities.
- Work closely with project managers and stakeholders to prioritize and plan project deliverables, taking into account team capacity.
- Proactively identify and address resource constraints, making adjustments to team assignments as needed.
Past experience includes
Nice to have
- Design and implement data pipelines, ETL processes, schemas, and data models to ingest, process, and prepare multi-petabyte scale datasets for downstream analytics and machine learning.
- Build and optimize data processing systems on modern platforms like Spark, Delta Lake, Kafka, etc.
- Implement data quality, validation, and monitoring measures leveraging tools such as Great Expectations.
- Ensure compliance with security, access control, and regulatory requirements related to PHI and other sensitive data types.
- Support adoption of emerging standards like FHIR for healthcare data exchange or the OMOP schema.
- Collaborate with data scientists, analysts, and engineers to understand data needs and deliver performant, reliable data products
- Keep track of emerging technologies & trends in the Data Engineering world, incorporating modern tooling and best practices at Craft.
- 10+ years experience building and operating production big data platforms and pipelines
- Good understanding of cloud computing and cloud managed databases be it on Azure or Google Cloud.
- Very good understanding of data warehousing, data lake
- Good knowledge of PySpark, Scala and java.
- Good knowledge ofopen source solutions for storage like apache iceberg or delta lake
- Spark and Hadoop
- Trino
- Presto
- RDBMS - Any one of AWS Redshift or PostgreSQL or mysql
- OLAP and OLTP
- Elastic Search or Solr
- Strong experience with SQL, Spark, workflow orchestrators, distributed message bus, Python, Presto, Deltalake, apache big data tool suites, Docker, Kubernetes, MPP
- Hands on with the design and implementation of cloud-based data solutions using platforms like AWS, Azure, or GCP, optimizing for scalability, cost-efficiency, and performance.
- Implement and maintain data lakes and warehouses, lakehouses including data modeling, ETL processes, and data quality assurance to empower data-driven decision-making.
- Develop real-time data pipelines using streaming technologies like Apache Kafka or AWS Event hub, enabling timely insights and actions from incoming data streams.
- Manage and enhance distributed data systems (e.g., Hadoop, Spark) to efficiently process large-scale datasets, ensuring data availability and reliability.
- Previous experience of working on health data and Azure cloud is a strong plus
- Strong track record of designing and implementing scalable data models, schemas, ETL logic
- Experience with data governance, master data management, data pseudonimization and anonymization, and data catalog solutions .
- A strong interest in learning new things and team player ethics.
- Strong analytical skills and good understanding of data structures and algorithms.
Nice to have
- Experience building data pipelines for machine learning.
- Knowledge of genomics, medical imaging, and/or EHR data domains
- Knowledge of HIPAA, HL7 and other healthcare data privacy requirements
- Hands on experience with fully managed data warehousing solutions Azure Synapse, AWS Redshift ,Big query, Snowflake etc:
Beware of fraud agents! do not pay money to get a job
MNCJobsGulf.com will not be responsible for any payment made to a third-party. All Terms of Use are applicable.
Job Detail
-
Job IdJD1608428
-
IndustryNot mentioned
-
Total Positions1
-
Job Type:Full Time
-
Salary:Not mentioned
-
Employment StatusPermanent
-
Job LocationAbu Dhabi, United Arab Emirates
-
EducationNot mentioned