Data Lake Engineer
Abu Dhabi, United Arab Emirates
Job Description
:
Help AG is looking to hire a Data Lake Engineer.
Responsibilities:
Qualification:
Benefits
About Us:
Help AG is the cybersecurity arm of e& enterprise (formerly Etisalat Digital) and provides leading enterprise businesses and governments across the Middle East with strategic consultancy combined with tailored information security services and solutions that address their diverse requirements, enabling them to evolve securely with a competitive edge.
Present in the Middle East since 2004, Help AG was strategically acquired by e& (formerly Etisalat Group) in Feb 2020, hence creating a cybersecurity and digital transformation powerhouse in the region.
Help AG has firmly established itself as the region's trusted IT security advisor by remaining vendor-agnostic, trustworthy, independent, and cybersecurity focused. With best-of-breed technologies from industry-leading vendor partners, expertly qualified service delivery teams and a state-of-the-art consulting practice, Help AG delivers unmatched value to its customers by strengthening their cyber defences and safeguarding their business.
Help AG is looking to hire a Data Lake Engineer.
Responsibilities:
- Responsible for designing a large Data Lake, managing data flows that integrate information from various sources into a centralized repository implementing data pipelines based on the ETL model to create a security data lake that can be used for detection, analysis and enhancing the security posture of the organization.
- 3+ years experience and hands-on knowledge across a wide range of technologies such as Linux as OS layer, Jenkins, Ansible, Git as automation engine, Mapr (HPE) Hadoop cluster as storage with Hive, Spark, Drill, Ranger, Kafka, Hue ecosystem services.
- Develop or enhance Hadoop workflows including data ingestion, extraction and processing of structured and unstructured data that can be utilized best in a security data lake scope.
- Provide guidance on suitable options, designing and creating data pipeline for analytical solutions.
- Knowledge and experience with best practices and standards around big data.
- Independently handle data ingestion issues, create root cause analysis, and adhere to data availability SLA.
- Knowledge in common CI/CD best practices and hands on experience with Continuous Integration Tools for the big data use cases.
- Produce data through complex SQL and other programming tools and develop reports, dashboards to feed into weekly, monthly, quarterly, and annual reports.
- Automate manual work and minimize manual intervention
- Monitor data ingestion finish time - handle surge in data volume
- Maintain project metrics and identify new metrics based on the Security Operations requirements
Qualification:
- Experience implementing a data lake architecture based on Hadoop, enterprise data solutions
- 3+ years using ETL / data movement tools
- 2+ years programming in one or more languages (Python, Java, Scala, C/C++)
- Experience with Agile process methodology, CI/CD automation, Test Driven Development.
- Experience with NO-SQL database like HBase and Apache Phoenix.
- Experience with Hive Query Language and optimization.
- Sound Knowledge in performance tuning and optimization (Spark, Hbase & Hive).
- Sound Working knowledge in Kafka and Spark.
Benefits
- Health insurance with one of the leading global providers for medical insurance.
- Career progression and growth through challenging projects and work.
- Employee engagement and wellness campaigns activities throughout the year.
- Excellent learning and development opportunities.
- Annual Flights Tickets.
- Inclusive and diverse working environment.
- Flexible/Hybrid working environment.
- Open Door Policy.
About Us:
Help AG is the cybersecurity arm of e& enterprise (formerly Etisalat Digital) and provides leading enterprise businesses and governments across the Middle East with strategic consultancy combined with tailored information security services and solutions that address their diverse requirements, enabling them to evolve securely with a competitive edge.
Present in the Middle East since 2004, Help AG was strategically acquired by e& (formerly Etisalat Group) in Feb 2020, hence creating a cybersecurity and digital transformation powerhouse in the region.
Help AG has firmly established itself as the region's trusted IT security advisor by remaining vendor-agnostic, trustworthy, independent, and cybersecurity focused. With best-of-breed technologies from industry-leading vendor partners, expertly qualified service delivery teams and a state-of-the-art consulting practice, Help AG delivers unmatched value to its customers by strengthening their cyber defences and safeguarding their business.
Beware of fraud agents! do not pay money to get a job
MNCJobsGulf.com will not be responsible for any payment made to a third-party. All Terms of Use are applicable.
Job Detail
-
Job IdJD1414219
-
IndustryNot mentioned
-
Total Positions1
-
Job Type:Full Time
-
Salary:Not mentioned
-
Employment StatusPermanent
-
Job LocationAbu Dhabi, United Arab Emirates
-
EducationNot mentioned