TuringCoding & software eval
Engineering Manager – LLM Evaluation
Pay not published
About this role, in Turing's words
We are looking for an Engineering Manager to lead teams creating high-quality conversational data for training and evaluating advanced Large Language Models (LLMs). The role involves managing the design of realistic, multi-turn interactions between users and AI assistants, including tool-based workflows involving calendars, email, maps, cloud storage, and other applications.
What you'll do
- Lead and manage teams responsible for creating and evaluating multi-turn LLM conversations.
- Define workflows, priorities, quality standards, and delivery goals for conversation data projects.
- Guide the design of realistic assistant interactions involving APIs, JSON, function calls, and external tools.
Show all 10 itemsHide the other 7 in What you'll do
- Review complex scenarios involving decision-making, clarification, error handling, constraints, and task completion.
- Ensure conversations accurately reflect realistic user expectations and assistant capabilities.
- Establish quality-control processes and monitor team performance, consistency, and deliverable quality.
- Coach team members on technical reasoning, conversation design, tool usage, and adherence to project guidelines.
- Collaborate with engineering, product, operations, and quality teams to resolve issues and improve workflows.
- Analyze reviewer feedback and implement process improvements to increase realism, accuracy, and training value.
- Manage project risks, timelines, escalations, and stakeholder communication.
Candidate background
What We’re Looking For
- 5+ years of professional experience in software engineering, technical operations, data, AI, or a related technical field.
- Prior experience managing, mentoring, or leading technical teams.
- Strong understanding of APIs, JSON, data formats, logical reasoning, and structured workflows.
- Ability to guide teams in breaking down complex tasks into realistic conversational and tool-use flows.
- Strong understanding of assistant behavior, including tool selection, clarification, limitations, and error handling.
- Excellent written and verbal English communication skills.
- Strong attention to detail and ability to establish and enforce quality standards.
- Proven ability to manage multiple priorities, provide actionable feedback, and deliver projects within deadlines.
- Ability to collaborate effectively with cross-functional teams and external stakeholders.
Quoted from selected sections of Turing's posting. The full posting may state other conditions. Read the full posting
A closer look
More context, when you need it.
Full job descriptionThe original posting, in Turing's words
Posted by Turing, reproduced here so you can judge the role before clicking. Original posting ↗
About Turing
Turing is a leading AI company accelerating the advancement and deployment of frontier AI systems. We work with the world’s top AI labs and enterprises to improve capabilities in reasoning, coding, agentic behavior, multimodality, and other advanced AI domains.
Role Overview
We are looking for an Engineering Manager to lead teams creating high-quality conversational data for training and evaluating advanced Large Language Models (LLMs). The role involves managing the design of realistic, multi-turn interactions between users and AI assistants, including tool-based workflows involving calendars, email, maps, cloud storage, and other applications.
The manager will oversee project execution, maintain quality standards, guide technical decision-making, and collaborate with engineering, operations, and review teams to deliver reliable training and evaluation datasets.
What You’ll Do
• Lead and manage teams responsible for creating and evaluating multi-turn LLM conversations.
• Define workflows, priorities, quality standards, and delivery goals for conversation data projects.
• Guide the design of realistic assistant interactions involving APIs, JSON, function calls, and external tools.
• Review complex scenarios involving decision-making, clarification, error handling, constraints, and task completion.
• Ensure conversations accurately reflect realistic user expectations and assistant capabilities.
• Establish quality-control processes and monitor team performance, consistency, and deliverable quality.
• Coach team members on technical reasoning, conversation design, tool usage, and adherence to project guidelines.
• Collaborate with engineering, product, operations, and quality teams to resolve issues and improve workflows.
• Analyze reviewer feedback and implement process improvements to increase realism, accuracy, and training value.
• Manage project risks, timelines, escalations, and stakeholder communication.
What We’re Looking For
• 5+ years of professional experience in software engineering, technical operations, data, AI, or a related technical field.
• Prior experience managing, mentoring, or leading technical teams.
• Strong understanding of APIs, JSON, data formats, logical reasoning, and structured workflows.
• Ability to guide teams in breaking down complex tasks into realistic conversational and tool-use flows.
• Strong understanding of assistant behavior, including tool selection, clarification, limitations, and error handling.
• Excellent written and verbal English communication skills.
• Strong attention to detail and ability to establish and enforce quality standards.
• Proven ability to manage multiple priorities, provide actionable feedback, and deliver projects within deadlines.
• Ability to collaborate effectively with cross-functional teams and external stakeholders.
Perks of Freelancing With Turing
• Work in a fully remote environment.
• Opportunity to work on cutting-edge AI projects with leading LLM companies.
Offer Details
• Commitments Required: At least 4 hours per day and minimum 40 hours per week with overlap of 4 hours with PST.
• Duration of Contract: 3 months (adjustable based on engagement)
Working with TuringPay, getting started and work, from Turing's own pages
What the platform publishes in general. These are not extra terms confirmed for this particular role.
- Getting paid
- Paid monthly, around the 5th, straight to your bank account, or inside Deel on projects run through Deel. Its help articles disagree on whether that is the 5th calendar day or business day. help.turing.com/articles/9691407757-project-onboarding, help.turing.com/articles/6582454854-what-if-i-haven-t-rec..., help.turing.com/articles/1226770947-project-offboarding
- Before your first paid task
- A short application, with tests on some roles. Turing's review article says an interview is typically required for all roles. Onboarding covers the terms, ID verification with a driver's license or passport, and tax and payment details. help.turing.com/articles/2020987592-apply-for-jobs-at-turing, help.turing.com/articles/9179629201-how-long-does-the-app..., help.turing.com/articles/9691407757-project-onboarding
- How much work
- Review takes from 48 hours to 6 weeks after you finish applying, and the client makes the final choice. No reply within 6 weeks usually means you were not selected. help.turing.com/articles/9179629201-how-long-does-the-app..., help.turing.com/articles/2020987592-apply-for-jobs-at-turing
- Contract
- Depending on the project, the engagement rests on Turing's Terms of Service and a Service Agreement, on Deel, or on a partner that provides the contract. help.turing.com/articles/6582454854-what-if-i-haven-t-rec...
Read on the pages linked above on 23 Sep 2026. Platforms change these terms, so check the linked page before you rely on one.
How does the pay compare?Across Coding & software eval roles, not this role alone
From our own daily reads of every open listing on the platforms we follow, not from Turing's posting. How we count.
Pay across the field: Coding & software eval
Across 392 roles in this field with a stated hourly rate in US dollars, on 14 platforms, the middle half pays $65 to $120 an hour at the top of the range, and the median is $95.
| Platform | Roles | Top of range, per hour |
|---|---|---|
| Alignerr | 137 | $50 to $400 · median $100 |
| micro1 | 44 | $60 to $350 · median $120 |
| Handshake AI | 43 | $47 to $125 · median $80 |
| Meridial (Invisible) | 34 | $22 to $150 · median $60 |
| Mercor | 33 | $60 to $200 · median $105 |
Sources and listing historyWhere this information comes from
This listing's history
- First seen here on 19 Sep 2026, and still on Turing's list when we last read it on 13 Oct 2026.
- Posted by Turing
- First seen by AIPayList
- Last seen live
- Listing ID
- 106212
Keep exploring
More like this
Related field, different requirements. Check each role's own page before you apply.