Senior Data Labeler

In this role, you will

• Build from scratch and lead the Data Labeling team (hiring, coaching, and performance management)

• Define annotation guidelines, quality standards, and evaluation frameworks

• Develop quality assurance processes, calibration sessions, and auditing systems

• Partner with AI researchers and engineers to translate research objectives into labeling workflows

• Prioritise labeling projects based on business and research needs

• Monitor operational metrics including quality, consistency, throughput, and cost

• Improve annotation tooling, automation, and workflow efficiency

• Lead complex AI evaluation projects, including safety, preference ranking, RLHF, policy evaluation, and benchmark creation

• Analyse disagreement patterns and edge cases to improve guidelines and model performance

• Manage vendor relationships and ensure consistent quality across distributed teams

• Build reporting dashboards and communicate operational insights to leadership

• Foster a culture of continuous improvement, accountability, and operational excellence

We're looking for someone who

• Has experience leading data annotation or AI evaluation teams

• Has strong operational and people management skills

• Understands AI model evaluation, LLM behaviour, and modern annotation workflows

• Can design scalable processes without sacrificing quality

• Communicates clearly across technical and non-technical teams

• Thrives in fast-moving startup environments

You might be a great fit if you

• Have managed annotation programs for LLMs, generative AI, or machine learning

• Have experience with RLHF, preference data collection, safety evaluations, or benchmark creation

• Have worked in Trust & Safety, AI Safety, Content Moderation, or ML Ops

• Have managed distributed or global annotation teams

• Have experience with vendor management and outsourcing operations

Bonus points

• Familiarity with prompt engineering and AI safety policies

• SQL, Python, or data analysis experience

• Experience building internal annotation platforms or workflow automation

• Background in linguistics, cognitive science, machine learning, or data operations

Compensation & benefits

• Competitive compensation, including equity

• Flexible time off

• A spacious office in the heart of Paris’s 2nd arrondissement, with a flexible hybrid setup

• Relocation support if you’re moving to Paris, available after your probationary period

• Premium private health insurance

• Mental health support, including coverage for therapy when you need it

• Language lessons to help you improve your English or French

• Lunch and dinner covered when you work from the office

• Learning and development support for courses, conferences, and opportunities to grow your skills

• All the hardware, subscriptions, tools, and services you need

• Team off-sites twice a year: we’ve recently been to the Alps, Saint-Tropez, and Marbella