JobHunter AI
⚠️ This position is no longer accepting applications. See similar jobs below.
Lead Data Engineer(Partner Company Role)
name
Location
United Kingdom
Work Mode
Remote
Type
Contract
Sector
Tech
First Seen
2026-07-03
Source
himalayas
Remote United Kingdom IT Finance MEAL Engineering Deadline Unclear Remote
Job Description
<em>We're looking for the right person to build that foundation from the ground up.</em><br><em>This is a hands-on leadership role with real ownership: you'll shape how data is collected, structured, and delivered — and grow a team around you as the function matures.</em><br><h3><strong>WHAT YOU'LL BE WORKING ON</strong></h3><ul style="margin-bottom:4px;"><li style="margin-bottom:4px;">Design the target-state data architecture: Postgres CDC → Kafka → Snowflake → client-facing data products.</li><li style="margin-bottom:4px;">Own tooling decisions across ingestion, orchestration, transformation, and quality layers.</li><li style="margin-bottom:4px;">Implement CDC-based ingestion from PostgreSQL services (RDS, Aurora, EC2, K8s operator) using Debezium or equivalent.</li><li style="margin-bottom:4px;">Build streaming and near-real-time pipelines with defined SLAs.</li><li style="margin-bottom:4px;">Build a data quality control layer: checksums, reconciliation, schema validation, anomaly detection, and alerting.</li><li style="margin-bottom:4px;">Define quality checkpoints across the full pipeline — from source capture through Snowflake to client delivery.</li><li style="margin-bottom:4px;">Define and enforce data contracts with service-owning teams for core entities: transaction, merchant, settlement, and processing status.</li><li style="margin-bottom:4px;">Build the external data delivery layer: financial settlement, transaction status, processor reconciliation, and client analytics.</li><li style="margin-bottom:4px;">Design tenant separation and implement replay/reload mechanisms for failure recovery.</li><li style="margin-bottom:4px;">Start hands-on, then gradually hire and grow a small data engineering team as the function matures.<br>Build a pragmatic roadmap with concrete deliverables at 3, 6, and 12 months.</li></ul><h3><strong>WHAT YOU NEED TO SUCCEED IN THIS ROLE</strong></h3><ul style="margin-bottom:4px;"><li style="margin-bottom:4px;">5+ years in data engineering with end-to-end ownership of production pipelines.</li><li style="margin-bottom:4px;">Hands-on with Snowflake, PostgreSQL CDC (Debezium preferred), and Kafka.</li><li style="margin-bottom:4px;">Solid AWS experience — S3, RDS, Aurora, and cloud data infrastructure.</li><li style="margin-bottom:4px;">Data quality engineering mindset: monitoring, reconciliation, lineage.</li><li style="margin-bottom:4px;">Comfortable defining data contracts and driving requirements with backend engineering teams.</li><li style="margin-bottom:4px;">Technical leadership experience: project ownership, cross-team alignment, delivery under constraints.</li><li style="margin-bottom:4px;">Kubernetes, Airflow/Prefect/Dagster, and dbt are strong pluses.</li><li style="margin-bottom:4px;">Payments domain knowledge — settlement, transaction lifecycle, processor integrations — is a strong plus.</li><li style="margin-bottom:4px;">Familiarity with gRPC, RabbitMQ, and reading Go/Python service code is expected.</