We are looking for an experienced? Data Engineer?to join our team full-time. This role requires a strong background in building and optimizing data pipelines, with a focus on supporting AI-driven solutions. The candidate will work closely with software engineers to ensure reliable data flows for AI models and contribute to cloud-based data integration using Microsoft Azure.
Key Responsibilities:
Design, develop, and maintain scalable data pipelines and ETL processes to support AI-driven projects
Build and optimize data infrastructure to ensure efficient data management, processing, and storage in cloud environments (Azure)
Collaborate with AI engineers to facilitate data preparation for machine learning models and AI solutions
Ensure data pipelines are secure, reliable, and meet performance standards
Monitor and troubleshoot data pipeline performance to ensure timely and accurate data delivery
Implement data governance policies and ensure compliance with data security standards
Required Qualifications:
Proven experience as a Data Engineer, with a focus on building and maintaining complex data pipelines for AI projects
Proficiency in SQL and experience with relational databases (e.g., PostgreSQL, SQL Server) and NoSQL databases (e.g., MongoDB)
Expertise in cloud-based data solutions, specifically Microsoft Azure (e.g., Azure Data Factory, Azure Synapse, Azure Databricks)
Experience with data integration tools and ETL frameworks (e.g., Apache Airflow)
Proficiency in at least one programming language for data manipulation (e.g., Python)
Knowledge of data warehousing solutions and best practices
Experience in working with data pipelines for AI or machine learning workflows
Strong problem-solving and analytical skills
Bachelor's degree in Computer Science, Engineering, or a related field
Proficiency in English (written and spoken)
Desirable Qualifications:
Familiarity with machine learning models and data preprocessing for AI solutions
Knowledge of containerization tools (Docker, Kubernetes) for data pipeline deployments
Job Type: Full-time
MNCJobsGulf.com will not be responsible for any payment made to a third-party. All Terms of Use are applicable.