Job Title: Data Engineer
Company Overview: Our client are a leading retail organization in Dubai
Position Overview: The Data Engineer is responsible for designing, implementing, and maintaining data platform, data marts, data architecture and data pipelines through ETL, APIs/Interfaces. Collaborate with data scientists, analysts, and other teams to ensure data is always available with accuracy. This role involves optimizing data systems for performance and scalability, identifying and resolving data-related issues, and staying up-to-date with emerging technologies in the data engineering field.

Key Responsibilities:

• Data Pipeline Design and Implementation: Design, build, and maintain efficient and scalable data pipelines to extract, transform, and load (ETL) data from various sources into data warehouses or other storage systems.

• Data Architecture: Design and optimize data storage solutions that support the organization\'s data needs, ensuring data quality, availability, and security.

• ETL Process Management: Develop, implement, and manage ETL processes to transform raw data into usable formats, while considering data integrity and performance.

• Data Modeling: Design and implement data models that align with business requirements and support analytical and reporting needs.

• Performance Optimization: Monitor and optimize data pipelines and databases for performance, reliability, and efficiency, addressing bottlenecks and improving query performance.

• Data Quality Assurance: Implement processes to ensure data accuracy, consistency, and integrity across various data sources.

• Collaboration: Work closely with data scientists, analysts, and other stakeholders to understand data requirements and deliver data solutions that meet their needs.

• Technology Selection: Evaluate and recommend appropriate technologies, tools, and frameworks for data processing, storage, and analysis.

• Cloud Platforms: Deploy and manage data solutions on cloud platforms like AWS, Azure, or GCP, leveraging their services for scalability and cost efficiency.

• Version Control and Documentation: Maintain version control for code and configurations, and document data processes, pipelines, and architecture for knowledge sharing.

• Data Security and Compliance: Implement data security measures and ensure compliance with data protection regulations and best practices.

• Troubleshooting and Issue Resolution: Identify and resolve data-related issues, troubleshoot errors, and perform root cause analysis.

• Continuous Learning: Stay updated with the latest trends, tools, and techniques in data engineering to drive innovation and improvements within the organization.

• Project Management: Lead or contribute to data engineering projects, collaborating with cross-functional teams to deliver on time and within scope.

Job requirements:

• Education: A master\'s degree in computer science, engineering, data science, or a related

• Experience: 10+ years of experience in data analytics, data engineering, data architect or related roles, demonstrating a strong understanding of data pipelines, ETL processes, and data architecture.

Technical Skills:


Proficiency in programming languages such as Python, Java, Scala, or SQL.
Experience with data warehousing solutions like Amazon Redshift
Expertise in designing and building ETL pipelines using tools like Glue, Talend, Data Factory/custom scripts.
Knowledge of database systems like PostgreSQL, MySQL, or NoSQL databases.
Hands-on experience with cloud platforms such as AWS.
Understanding of data modeling, schema design, and data normalization/denormalization.

• Data Technologies:

Knowledge of data integration techniques and data processing methodologies.
Understanding of data streaming technologies like Apache Kafka or AWS Kinesis.
Experience working with APIs, data formats (JSON, XML, etc.), and data serialization.

• Data Quality and Governance:

Ability to implement data quality checks, validation, and data cleansing processes.
Understanding of data governance practices and compliance with data privacy regulations (e.g., GDPR).

• Performance Optimization:

Skill in optimizing data pipelines and databases for performance, scalability, and reliability.
Knowledge of query optimization techniques and indexing strategies.

• Problem-Solving and Analytical Skills:

Strong analytical thinking and problem-solving skills to troubleshoot data issues and optimize processes.

• Communication and Collaboration:

Excellent communication skills to work effectively with cross-functional teams and translate business requirements into technical solutions.

• Adaptability and Learning:

Willingness to learn and adapt to new technologies and tools in the evolving data engineering landscape.

• Project Management:

Capability to manage projects, timelines, and priorities while collaborating with multiple stakeholders.