Data Platform System Engineer

You will be responsible for operating and evolving SP Group’s enterprise Data & AI platform to support mission-critical analytics, regulatory compliance, and AI-driven innovation. This role focuses on ensuring the platform is scalable, reliable, secure, and performant, while enabling data engineers, scientist and analysts to deliver business value efficiently. What You'll Do • Ensure high availability and reliability of the enterprise data platform (Cloudera and Talend Big Data). • Deliver scalable and performant data infrastructure supporting batch and near real-time workloads. • Enable secure and compliant data operations aligned with enterprise and regulatory requirements. • Improve platform automation and operational efficiency through DevOps and Infrastructure-as-Code. • Provide strong platform support and partnership to data engineering, analytics, and business teams. Data Platform Engineering & Operations • Design, deploy, and manage large-scale data platform environments (primarily Cloudera and Talend Big Data). • Operate and optimize Hadoop ecosystem components (HDFS, Hive, Spark, HBase, Impala, Iceberg, Ozone). • Manage cluster lifecycle including provisioning, scaling, patching, and upgrades. • Ensure platform uptime, stability, and service reliability through proactive monitoring and maintenance. Performance, Scalability & Capacity Management • Monitor workload performance and optimize data processing efficiency. • Perform capacity planning to support increasing data volumes and usage. • Troubleshoot performance bottlenecks across compute, storage, and query layers. Platform Delivery & Execution Planning • Plan and coordinate execution of platform upgrades, patching, and scaling activities. • Manage deployment schedules and ensure minimal service disruption. • Coordinate cross-team dependencies (infrastructure, network, security). • Track and report execution progress against platform roadmap. DevOps, Automation & Platform Engineering • Implement Infrastructure-as-Code using tools such as Terraform and Ansible. • Design and maintain CI/CD pipelines for platform configuration. • Automate platform provisioning, deployment, and operational workflows. • Improve observability through monitoring, alerting, and logging practices. Security, Governance & Compliance • Implement and manage platform security controls including authentication (Kerberos), authorization (e.g. Ranger), and encryption. • Ensure compliance with enterprise data governance, security policies, and regulatory requirements. • Support auditability, access control, and data protection mechanisms. Collaboration & Platform Enablement • Work closely with data engineers, data scientists, and business teams to enable data solution delivery. • Partner with infrastructure, network, and security teams to ensure platform integration and stability. • Provide guidance, troubleshooting, and best practices to users of the platform. • Document platform architecture, processes, and operational procedures. What You'll Need • 5- 10 years of experience in data platform engineering, big data operations, or infrastructure engineering. • Strong hands-on experience managing big data platforms (Cloudera CDP, Hadoop ecosystem). • Deep understanding of Hadoop components: HDFS, Hive, Spark, HBase, and related services. • Experience with cluster operations, performance tuning, and troubleshooting. • Proficiency in automation and scripting (Python, Bash, Terraform, Ansible). • Experience with CI/CD pipelines and DevOps practices. • Solid understanding of data platform security and access control concepts. • Strong analytical and problem-solving skills with a proactive mindset. • Ability to work in a fast-paced, cross-functional environment. • Good communication skills with ability to translate technical topics to stakeholders. • Ownership mentality with focus on platform reliability and continuous improvement. Good to have • Experience with Databricks, Talend, or hybrid data platforms. • Knowledge of containerization and orchestration. • Experience with streaming platforms (e.g. Kafka) or real-time processing. • Familiarity with modern data lake-house concepts (e.g. Iceberg, Delta). • Exposure to cloud or hybrid environments (Azure preferred). What We'll Provide • Join us to empower the future of energy. • You will work in teams that thrive on collaboration for outstanding achievements. • We have regular wellness programmes to promote healthy living