Data Engineer Caliberly
Dubai, United Arab Emirates
Job Description
Job Details
Employment Type
full_time
Industry
Salary
18000.00 - 20000.00
Location
Dubai, United Arab Emirates
Date Posted
June 15, 2023
About the Job
Key Responsibilities:
1 Data Acquisition
Skill Set:
Expertise in Big Data querying tools, such as Hive, Hbase and Impala.
Expertise in SQL, writing complex queries/views, partitions, bucketing
Strong Experience in Spark and Storm.
Expertise in Monitoring Tools like Grafana,Prometheous,Zabbix.
Hands on experience in Management of Hadoop Pentaho.
Expertise in storage layer/DB like HDFS,Cassandra,Opentsd Expertise in Event-driven Architecture.
More jobs on https://www.qureos.com/
Employment Type
full_time
Industry
Salary
18000.00 - 20000.00
Location
Dubai, United Arab Emirates
Date Posted
June 15, 2023
About the Job
Key Responsibilities:
- Data Engineering
1 Data Acquisition
- Vendor should manage the existing Data pipelines built for data ingestion.
- Create and manage new data pipelines following the best practises for the new ingestion of data.
- Continuously monitor the data ingestion through Change Data Capture for the incremental load
- Any failed batch job schedule to be analysed and fixed to capture the data
- Maintaining and continuously updating on the technical documentation of the ingested data and maintaining the centralized data dictionary, with necessary data classifications.
- Extraction of data from the data sources to be cleaned and ingested into big data platform
- Automation of data cleaning has to be defined before ingestions
- Data cleaning to handle the missing data and remove any outliers and resolve any inconsistencies
- Data quality check has to be performed in terms of accuracy, completeness, consistency, timeliness, believability, and interpretability
- Exposing of Data views or Data models to Reporting and source systems using Hive or Impala, or similar tools provided by us
- Exposing of cleansed data to Artificial Intelligence team for building data science models
Skill Set:
Expertise in Big Data querying tools, such as Hive, Hbase and Impala.
Expertise in SQL, writing complex queries/views, partitions, bucketing
Strong Experience in Spark and Storm.
Expertise in Monitoring Tools like Grafana,Prometheous,Zabbix.
Hands on experience in Management of Hadoop Pentaho.
Expertise in storage layer/DB like HDFS,Cassandra,Opentsd Expertise in Event-driven Architecture.
More jobs on https://www.qureos.com/
Beware of fraud agents! do not pay money to get a job
MNCJobsGulf.com will not be responsible for any payment made to a third-party. All Terms of Use are applicable.
Related Jobs
Job Detail
-
Job IdJD1591439
-
IndustryNot mentioned
-
Total Positions1
-
Job Type:Full Time
-
Salary:18000 - 20000 per month
-
Employment StatusPermanent
-
Job LocationDubai, United Arab Emirates
-
EducationNot mentioned