Job Description

As a Senior Azure Data Engineer, you will be a key member of our data engineering team responsible for designing, implementing, and optimizing data pipelines and solutions on the Azure cloud platform. You will play a crucial role in building and maintaining data infrastructure, ensuring data availability, reliability, and performance. This position requires a strong background in Azure data services, ETL processes, and a commitment to best practices in data engineering.

Main Duties & Responsibilities:

The role must execute the following core responsibilities:

Design, develop, and maintain end-to-end data pipelines, leveraging Azure Data Factory, Azure Databricks, and other Azure data services. Implement efficient data extraction, transformation, and loading (ETL) processes. Experience in building end to end architecture for Data Lakes, Data Warehouses and Data Marts Collaborate with data architects and other stakeholders to design scalable and robust data solutions. Ensure data integration across various source systems and data stores. Perform data transformation, cleansing, and enrichment to support analytics and reporting needs. Optimize data processing for performance and scalability. Leverage Azure cloud services, such as Azure SQL Data Warehouse, Azure Synapse Analytics, and Azure Data Lake Storage, to build and manage data solutions. Implement best practices for Azure resource management and cost optimization. Implement data governance policies and ensure data security and compliance with industry standards. Manage access controls, encryption, and auditing. Monitor data pipelines and infrastructure performance, identifying and resolving bottlenecks. Continuously optimize data processes for efficiency. Maintain comprehensive documentation of data pipelines, architecture, and configurations. Share knowledge and mentor junior team members.

Qualification/ Experience:

Bachelor's or master's degree in computer science, Information Technology, or a related field. 5 to 10 years of experience in data engineering with a focus on Azure cloud technologies. Expert in Azure Data Factory, Azure Databricks, Azure SQL Data Warehouse, and Azure Data Lake Storage. Strong SQL and scripting skills for data manipulation and automation. Knowledge of data warehousing concepts, data modeling, and data integration patterns. Expertise in Python & PySpark development, DWH, Data Integration, Cloud, Architecture, Design, Data Modelling, Cloud data warehouse like Azure Synapse, Snowflake analytical warehouse Familiarity with data governance and compliance standards. Excellent problem-solving and troubleshooting skills. Strong communication and collaboration skills. Microsoft Certified: Azure Data Engineer Associate * Microsoft Certified: Azure Solutions Architect Expert