Principal Data Engineer

OverviewThe opportunityCore42 is the UAE’s national-scale enabler for cloud and generative AI, combining G42 Group’s expertise across multiple technology disciplines into a single platform for public sector and large enterprise transformations. Building on our capabilities as sovereign cloud and HPC specialist, we bring generative AI, cybersecurity, professional and managed services expertise to enable national-scale program deployments across industries.ResponsibilitiesYour key responsibilitiesTechnical Leadership and Strategy Develop and implement comprehensive data strategies, guiding the technical direction of projects while aligning with the organizations long-term objectives.Mentorship and Team Development Mentor junior data engineers, fostering a culture of continuous learning and development within the team.Data Pipeline Architecture and Development Design, construct, install, test, and maintain highly scalable data pipelines with a focus on machine learning models and analytics.Data Integration Work closely with data scientists, ML engineers, and stakeholders to ensure that data is accessible, consistent, and reliable for ongoing projects.API and Data Services Develop and maintain APIs for data access and manipulation, and integrate with external data services as needed.Data Storage Manage and optimize data storage solutions, including relational databases, Search Engines like Elasticsearch and NoSQL databases, to support the requirements of machine learning models. Understand data engines and structure to effectively design solutions for transactional, analytics, and search purposes.Data Quality and Governance Implement processes to monitor data quality and ensure production data is always accurate and available for key stakeholders.Performance Optimization and Scalability Ensure the high performance and scalability of data systems to accommodate increasing data demands.Collaboration and Support Collaborate with ML engineers to assist in data-related technical issues and provide architectural guidance and solutions.Security and Compliance Ensure compliance with data security and privacy policies.Documentation Maintain clear and up-to-date documentation including data dictionaries, metadata, and architectural diagrams.Research and Innovation Stay updated with emerging technologies and trends in data engineering and integrate innovative solutions to enhance data processing and analysis capabilities.Project Management and Delivery Lead data engineering projects from inception to completion, ensuring timely delivery, budget adherence, and quality standards.Stakeholder Management and Reporting Engage with key stakeholders to understand their data requirements, delivering insightful reports and analyses to support decision-making.QualificationsSkills and attributes for success Bachelor’s degree in Computer Science, Engineering, Mathematics, or a related field; or equivalent work experience. 7 years of experience in a Data Engineering role. Proficiency in SQL and programming languages like Python, Java, and Scala. Hands-on experience with big data technologies like Hadoop, Spark and Flink. Familiarity with machine learning frameworks such as TensorFlow, PyTorch, or similar. Strong understanding of data warehousing concepts, ETL processes, and data modeling. Experience with API development and integration with data services. Experience with cloud platforms like AWS, GCP. Knowledge in DevOps, CI/CD methods, and containerization technologies like Docker or Kubernetes. Experience with real-time data processing.Technical stack Programming Languages: Python, Java, Scala, SQL, Bash Big Data Technologies: Hadoop, Spark, Flink Databases: MySQL, PostgreSQL, MongoDB, Cassandra, HBase, Redis Cloud Platforms: Azure, AWS API Development: RESTful APIs, GraphQL, OpenAPI Data Services: Kafka, RabbitMQ Containers: Docker, Kubernetes Monitoring: Prometheus, Grafana ELK, Cloud Native monitoring tools.What we look forIf you are a performance-driven, inquisitive mind with the agility to adapt to ambiguity, you will fit right in. You should be eager to explore opportunities to build meaningful collaborations with stakeholders and aspire to create unique customer-centric solutions. Bias for action and a passion to conquer new frontiers in the AI space is at the heart of the Core42 community.What working at Core42 offersCulture: An open, diverse and inclusive environment with a global vision that encourages personal growth and focuses on ground-breaking, industry-first innovations.Career: Outstanding learning, development & growth opportunities via structured training programs and innovative, high-tech projects.Work-Life: A hybrid work policy to strike the perfect balance between office and home.Rewards: A competitive remuneration package with a host of perks including healthcare, education support, leave benefits and more.

Back to blog

Other Jobs To Apply

No other job posts for this day.

Common Interview Questions And Answers

1. HOW DO YOU PLAN YOUR DAY?

This is what this question poses: When do you focus and start working seriously? What are the hours you work optimally? Are you a night owl? A morning bird? Remote teams can be made up of people working on different shifts and around the world, so you won't necessarily be stuck in the 9-5 schedule if it's not for you...

2. HOW DO YOU USE THE DIFFERENT COMMUNICATION TOOLS IN DIFFERENT SITUATIONS?

When you're working on a remote team, there's no way to chat in the hallway between meetings or catch up on the latest project during an office carpool. Therefore, virtual communication will be absolutely essential to get your work done...

3. WHAT IS "WORKING REMOTE" REALLY FOR YOU?

Many people want to work remotely because of the flexibility it allows. You can work anywhere and at any time of the day...

4. WHAT DO YOU NEED IN YOUR PHYSICAL WORKSPACE TO SUCCEED IN YOUR WORK?

With this question, companies are looking to see what equipment they may need to provide you with and to verify how aware you are of what remote working could mean for you physically and logistically...

5. HOW DO YOU PROCESS INFORMATION?

Several years ago, I was working in a team to plan a big event. My supervisor made us all work as a team before the big day. One of our activities has been to find out how each of us processes information...

6. HOW DO YOU MANAGE THE CALENDAR AND THE PROGRAM? WHICH APPLICATIONS / SYSTEM DO YOU USE?

Or you may receive even more specific questions, such as: What's on your calendar? Do you plan blocks of time to do certain types of work? Do you have an open calendar that everyone can see?...

7. HOW DO YOU ORGANIZE FILES, LINKS, AND TABS ON YOUR COMPUTER?

Just like your schedule, how you track files and other information is very important. After all, everything is digital!...

8. HOW TO PRIORITIZE WORK?

The day I watched Marie Forleo's film separating the important from the urgent, my life changed. Not all remote jobs start fast, but most of them are...

9. HOW DO YOU PREPARE FOR A MEETING AND PREPARE A MEETING? WHAT DO YOU SEE HAPPENING DURING THE MEETING?

Just as communication is essential when working remotely, so is organization. Because you won't have those opportunities in the elevator or a casual conversation in the lunchroom, you should take advantage of the little time you have in a video or phone conference...

10. HOW DO YOU USE TECHNOLOGY ON A DAILY BASIS, IN YOUR WORK AND FOR YOUR PLEASURE?

This is a great question because it shows your comfort level with technology, which is very important for a remote worker because you will be working with technology over time...