Principal Data Engineer, Data Quality & Datasets Pipelines

🧠 Your day-to-day

• Implement and iterate filtering, retention and offload policies across hot, cold, archive and delete storage

• Build the data movement and pipelines that our pre-ingestion selection model depends on

• Decide where and how selection models run, balancing compute cost against data value

• Build early, low-compute decision points that avoid expensive standardisation and enrichment of low-value data

• Work with the storage and data infra teams, and monitor data quality to make sure our datasets reflect what model training needs

🧩 What you’ll be working on

• Turning our dataset strategy into working systems: moving, filtering, retaining and offloading driving data at hundreds of petabytes

• The engineering behind our pre-ingestion selection mode(s)l: how data moves, where the model runs and how offload policies work

• Delivering large, measurable storage-cost savings through early, low-compute decisions

• Making sure our datasets contain the most useful behavioural and environmental coverage for training. This is as much a data quality challenge as a software engineering one

• Working as a pair with our dataset strategy lead (who design and builds the models) and closely with the data infra team, shipping practical solutions quickly

👤 You should apply if

• You’re a senior data or software engineer who has run data ingestion or data quality pipelines at petabyte scale

• You’re strong in Python and Spark, with cloud data engineering experience

• You have good judgement on data filtering and quality, and know how to check you haven’t discarded valuable data

• You ship practical solutions quickly and are comfortable iterating on live systems

• You’re a software engineer at heart, building the systems around models rather than the models themselves

• You enjoy working closely with data infra and storage teams

• You bring experience in AV, robotics, fleet data or high-volume IoT, or with retention, offload and data lifecycle policies (a bonus rather than a requirement)

🌱 Not ticking every box? That’s totally okay! If you’re passionate about autonomy and keen to learn, we encourage you to apply even if you don’t meet every requirement.