We are working with a leading global Financial Services business to hire a Senior Data Engineer into their growing data engineering practice. This is a hands-on technical role sitting at the centre of a major enterprise data platform build, with scope to influence architecture, drive pipeline development, and contribute to AI initiatives across a complex, high-volume financial data environment.
Responsibilities:
-
Design and develop scalable data solutions on a Lakehouse architecture platform, supporting enterprise-wide data processing and analytics
-
Build, optimise, and maintain ETL/ELT pipelines and structured streaming workflows for both batch and real-time data ingestion
-
Configure and tune clusters and Spark jobs to deliver consistent performance at scale
-
Utilise Delta Live Tables and Unity Catalog to manage data ingestion, transformation, and access governance
-
Apply IAM best practices and maintain compliance with data security standards across the platform
-
Support infrastructure provisioning and resource management using Terraform
-
Implement monitoring frameworks covering pipeline performance, data quality, and operational health
-
Contribute to code reviews, technical documentation, and team knowledge-sharing
-
Work within an Agile delivery model, collaborating closely with data scientists, analysts, and business stakeholders
-
Explore emerging technologies and AI tooling to enhance development productivity and platform capability
Requirements
-
6+ years in data engineering, with at least 2 years hands-on experience with Databricks
-
Strong Python and Spark programming skills
-
Solid AWS experience across core services including S3, Glue, and Lambda
-
Deep understanding of data modelling, SQL, and ETL/ELT design patterns
-
Experience with Delta Lake, Lakehouse architecture, and Git-based version control
-
Demonstrable use of AI tools within a professional development workflow
-
Strong communication skills and the ability to work effectively across technical and non-technical teams
Desirable:
-
Financial services or fund administration background
-
Exposure to AI/ML implementation patterns and real-time data processing frameworks
-
Multi-cloud experience beyond AWS
-
API development or data governance framework experience
-
Experience mentoring junior engineers
Benefits
Salary: to be discussed, depending on experience
Length: Permanent contract
