Data Engineer

🎯 Job Summary We are looking for an experienced Data Engineer with 7+ years of experience in designing, developing, and maintaining scalable data pipelines and data infrastructure. The candidate will be responsible for building ETL/ELT processes, managing large datasets, optimising data workflows, and supporting data analytics and business intelligence initiatives. 🔧 Key Responsibilities • Design, develop, and maintain scalable data pipelines and data integration solutions. • Build and manage ETL/ELT workflows for structured and unstructured data. • Develop data models and optimise data warehouse solutions. • Integrate data from multiple sources, including databases, APIs, and cloud platforms. • Implement data processing solutions using batch and real-time streaming technologies. • Ensure data quality, accuracy, consistency, and reliability. • Work with data scientists, analysts, architects, and business stakeholders. • Develop and optimise SQL queries and database performance. • Implement data governance, security, and access control practices. • Monitor, troubleshoot, and resolve data pipeline failures and performance issues. • Work with cloud-based data platforms and distributed processing frameworks. • Maintain technical documentation and follow software development best practices. 💻 Technical Skills & Requirements • Strong proficiency in SQL and Python. • Hands-on experience with ETL/ELT tools and data pipeline development. • Experience with Apache Spark, PySpark, and Hadoop. • Knowledge of Apache Airflow or similar workflow orchestration tools. • Experience with data warehouses such as Snowflake, Amazon Redshift, Google BigQuery, or Azure Synapse. • Hands-on experience with AWS, Microsoft Azure, or Google Cloud Platform. • Knowledge of relational and NoSQL databases. • Experience with data integration, data modelling, and data transformation. • Familiarity with Kafka or other real-time data streaming technologies. • Experience with Git, CI/CD, and automated deployment processes. • Understanding of data governance, data security, and data quality frameworks. 🎓 Education & Experience • Bachelor's degree in Computer Science, Information Technology, Data Engineering, or a related discipline. • Minimum 7 years of relevant experience in Data Engineering. • Proven experience developing and managing enterprise-scale data pipelines. • Experience working with cloud data platforms and big data technologies. 🤝 Required Skills • Strong analytical and problem-solving abilities. • Excellent SQL and programming skills. • Ability to handle large and complex datasets. • Strong communication and collaboration skills. • Ability to work independently and manage multiple priorities. ⭐ Preferred Certifications • AWS Certified Data Engineer. • Microsoft Certified: Azure Data Engineer Associate. • Google Cloud Professional Data Engineer. • Databricks Data Engineer Certification.