Job Summary

The Careem Big Data Platform team\'s mission is to provide a platform to abstract big data complexities and enable fast, reliable and secure access to data. As the leader of this team, you will be at the forefront of fulfilling this mission. You will be working with and leading the top talent of the region, leveraging modern big data tools and techniques to solve the region\'s day to day problems, on top of our own in-house data platform, serving users in real-time.

Main activities and Responsibilities

• Define the architecture, scope and deliver various Big Data solutions.
• Build and maintain large scale deployment of Cassandra cluster.
• Support other teams by providing guidance on data modeling, data usage, processing and how they can best leverage the platform.
• Build scalable data pipelines to ingest data from a variety of data sources, identify critical data elements and define data quality rules.
• Leverage Spark/Hadoop ecosystem knowledge to design and develop capabilities to deliver innovative and improved data solutions.
• Provide insights on area of improvements including Data Governance, best practices, large scale processing.
• Support the bug fixing and performance analysis along the data pipeline.
• Collaborate, coach, and mentor colleagues in an energetic, growing team.

Requirements

• 4+ years of experience as software engineer, with strong skills in at least one programming language is mandatory, preferably Scala or Java or Python.
• 2+ years of experience with Cassandra Deployment/Maintenance and Cassandra Data Modelling.
• Expertise with distributed systems and design/implementation for reliability, availability, scalability and performance.
• Proven experience with AWS technologies like S3, EMR, Cloudformation.
• Creative and innovative approach to problem-solving.
• 3+ year of experience as a Big Data Engineer or similar role.
• 1+ year of experience with Hadoop and/or Spark.
• Experience with CICD using Jenkins, Terraform or other related technologies.
• Familiarity with containerized platform like Docker and Kubernetes.
• Experience working with real time data processing using Kafka, Spark Streaming or similar technology.
• Experience working with Hive, Presto or other querying frameworks.

Edarabia