Data Engineer

Job Responsibilities • Design and optimise scalable data workflows with subject matter expertise in data loading patterns, data architecture etc. • Develop and maintain robust data pipelines that support both real-time and batch processing • Extract and integrate data from various sources, developing and scaling models in real-time business conditions. This includes interfacing (via API) between the data layer and application layer. • Leverage on SQL to ensure efficient data transformation and analytics • Implement data modelling best practices to support AI/ML applications, including schema design, partitioning strategies and indexing for performance optimisation • Integrate diverse data sources (e.g. structured, semi-structured and unstructured) into unified data architectures that power AI-driven applications and insights • Train advanced models and algorithms suitable for business use – Models include computer vision models, forecasting models, route optimisation etc. • Work as a team to ensure data readiness, feature engineering pipelines and reproducibility of experiments • Champion data quality and governance by implementing validation, lineage tracking, and compliance standards • Serve as data engineer, software engineer, cloud engineer and AI scientist all-in-one • Job Requirements • At least Bachelor Degree in Engineering, Economics, Mathematics, Computer Science or related areas • Minimum 5 years of work experience in a related role dealing with artificial intelligence • Advanced understanding in software engineering practices (e.g. Version Control), containerization, CI/CD pipelines • Experience with deploying models using frameworks like TensorFlow, PyTorch, and extremely familiar with platforms like Azure, Databricks, Fabric • Advanced proficiency in SPARK, Python, Java, SQL, Pandas, Hugging Face, Open AI APIs • Familiarity with data manipulation, statistical analysis and machine learning theories. This includes extracting and integration of data from various sources, developing and scaling models in real-time business conditions. This includes interfacing (via API) between the data layer and application layer. • Advanced proficiency with both full stack software engineering and data engineering with solid experience with working with Azure etc. • Familiar with data ethics and governance including best practices on how data should be managed securely