to design, develop, and optimize data pipelines, ETL processes, and data integration solutions. The ideal candidate should have expertise in AWS cloud services, data engineering best practices, open-source tools, and data schema design. The role requires hands-on experience with large-scale data processing, real-time data streaming, and cloud-based data architectures.
Key Responsibilities:
Develop and Maintain Data Pipelines
to process structured and unstructured data efficiently.
Implement ETL/ELT Workflows
for batch and real-time data processing.
Optimize Data Processing Workflows
using distributed computing frameworks.
Ensure Data Integrity and Quality
through data validation, cleaning, and transformation techniques.
Work with AWS Cloud Services
, including S3, Redshift, Glue, Lambda, DynamoDB, and Kinesis.
Leverage Open-Source Tools
like Apache Spark, Airflow, Kafka, and Flink for data processing.
Manage and Optimize Database Performance
for both SQL and NoSQL environments.
Collaborate with Data Scientists and Analysts
to enable AI/ML model deployment and data accessibility.
Support Data Migration Initiatives
from on-premise to cloud-based data platforms.
Ensure Compliance and Security Standards
in handling sensitive and regulated data.
Develop Data Models and Schemas
for efficient storage and retrieval.
Required Skills & Qualifications:
8+ years of experience
in data engineering, data architecture, and cloud computing.
Strong knowledge of AWS Services
such as Glue, Redshift, Athena, Lambda, and S3.
Expertise in ETL Tools
, including Talend, Apache NiFi, Informatica, dbt, and AWS Glue.
Proficiency in Open-Source Tools
such as Apache Spark, Hadoop, Airflow, Kafka, and Flink.
Strong Programming Skills
in Python, SQL, and Scala.
Experience in Data Schema Design
, normalization, and performance optimization.
Knowledge of Real-time Data Streaming
using Kafka, Kinesis, or Apache Flink.
Experience in Data Warehouse and Data Lake Solutions
.
Hands-on experience with DevOps and CI/CD Pipelines
for data engineering workflows.
Understanding of AI and Machine Learning Data Pipelines
.
Strong analytical and problem-solving skills
.
Preferred Qualifications:
AWS Certified Data Analytics - Specialty or AWS Solutions Architect certification.
Experience with Kubernetes, Docker, and serverless data processing.
Exposure to MLOps and data engineering practices for AI/ML solutions.
Experience with distributed computing and big data frameworks.
Job Category:
Information Technology
Job Type:
Contract
Job Location:
Dubai
Languages:
English
Experience:
8 + Years
Beware of fraud agents! do not pay money to get a job
MNCJobsGulf.com will not be responsible for any payment made to a third-party. All Terms of Use are applicable.