TuringCoding & software eval
Domain Expert – Python
Pay not published
About this role, in Turing's words
We are looking for experienced Software Engineers with a strong background in developing and operating large-scale production systems. The ideal candidate has worked extensively on enterprise software, understands how production systems fail, has experience identifying security vulnerabilities in code, and has successfully implemented secure fixes and mitigations. This role requires practical experience in debugging complex production issues, participating in incident response, and building resilient, secure software.
What you'll do
- Design and build realistic coding-agent benchmark tasks using production-like repositories, tests, configs, documentation, and runtime scenarios.
- Create benign engineering tasks such as bug fixes, feature additions, CI repairs, config migrations, integration updates, and runtime-state fixes.
- Define clear utility requirements to verify that the coding agent completes the requested task correctly.
Show all 10 itemsHide the other 7 in What you'll do
- Define safety/alignment requirements to ensure the agent preserves system constraints, developer intent, data integrity, privacy, permissions, and oversight mechanisms.
- Develop visible and hidden test suites to evaluate both task completion and unsafe agent behavior.
- Create safe reference solutions and unsafe reference solutions where the agent passes utility checks but violates multiple safety/alignment requirements.
- Identify and evaluate unsafe shortcuts such as disabling tests, weakening assertions, deleting protected data, leaking secrets, broadening permissions, or bypassing validation.
- Package tasks with prompts, metadata, runnable repos, evaluators, reference patches, scoring rubrics, and calibration notes.
- Analyze rollouts from frontier coding agents such as Claude Code and Codex to assess utility completion and safety/alignment violations.
- Collaborate with engineering, QA, security, and client teams to improve task quality, evaluator reliability, and benchmark difficulty.
Candidate background
What this role asks for
- Python
Read from the posting's own words: show the exact sentences
- Python: “listed by the platform under required skills: Python”
Quoted from selected sections of Turing's posting. The full posting may state other conditions. Read the full posting
A closer look
More context, when you need it.
Full job descriptionThe original posting, in Turing's words
Posted by Turing, reproduced here so you can judge the role before clicking. Original posting ↗
About Turing
Turing is one of the world's fastest-growing AI companies, accelerating the advancement and deployment of powerful AI systems. Founded in 2018, Turing partners with leading AI labs and enterprises to improve frontier AI models through high-quality data, expert human feedback, and rigorous evaluation frameworks.
Our contributors work on cutting-edge projects spanning coding, reasoning, agentic workflows, software engineering, and multimodal AI—helping shape the next generation of intelligent systems used by millions worldwide.
About the Role
We are looking for experienced Software Engineers with a strong background in developing and operating large-scale production systems. The ideal candidate has worked extensively on enterprise software, understands how production systems fail, has experience identifying security vulnerabilities in code, and has successfully implemented secure fixes and mitigations. This role requires practical experience in debugging complex production issues, participating in incident response, and building resilient, secure software.
What You'll Do
• Design and build realistic coding-agent benchmark tasks using production-like repositories, tests, configs, documentation, and runtime scenarios.
• Create benign engineering tasks such as bug fixes, feature additions, CI repairs, config migrations, integration updates, and runtime-state fixes.
• Define clear utility requirements to verify that the coding agent completes the requested task correctly.
• Define safety/alignment requirements to ensure the agent preserves system constraints, developer intent, data integrity, privacy, permissions, and oversight mechanisms.
• Develop visible and hidden test suites to evaluate both task completion and unsafe agent behavior.
• Create safe reference solutions and unsafe reference solutions where the agent passes utility checks but violates multiple safety/alignment requirements.
• Identify and evaluate unsafe shortcuts such as disabling tests, weakening assertions, deleting protected data, leaking secrets, broadening permissions, or bypassing validation.
• Package tasks with prompts, metadata, runnable repos, evaluators, reference patches, scoring rubrics, and calibration notes.
• Analyze rollouts from frontier coding agents such as Claude Code and Codex to assess utility completion and safety/alignment violations.
• Collaborate with engineering, QA, security, and client teams to improve task quality, evaluator reliability, and benchmark difficulty.
Required Qualifications
• Bachelor's or Master's degree in Computer Science or a related technical discipline.
• Minimum 8 years of hands-on software engineering experience in leading product companies or technology startups.
• Strong programming experience in Python (mandatory) along with one or more programming languages such as Java, Go, C++
• Demonstrated experience building and maintaining production software used by real customers.
• Strong understanding of software architecture, debugging, and performance optimization.
Required Technical Experience
Production engineering
• Experience developing software deployed in production at scale.
• Strong understanding of software deployment pipelines, monitoring, logging, and observability.
• Experience diagnosing production failures using logs, metrics, and distributed tracing.
• Knowledge of high availability, fault tolerance, scalability, and disaster recovery principles.
Secure Software Development
• Experience identifying security vulnerabilities through code reviews or security assessments.
• Practical knowledge of common software vulnerabilities, including:
• Injection attacks
• Authentication and authorization flaws
• Memory safety issues
• Race conditions
• Deserialization vulnerabilities
• Secrets management issues
• Input validation failures
• Experience implementing secure code fixes and validating remediation.
Perks of Freelancing with Turing:
• Work in a fully remote environment.
• Opportunity to work on cutting-edge AI projects with leading LLM companies.
Offer Details:
• Commitments Required : At least 4 hours per day and minimum 20 hours per week with overlap of 4 hours with PST.
• Employment type : Contractor assignment (no medical/paid leave)
• Duration of contract : 4-8 weeks; [expected start date is next week]
Evaluation Process:
• Delviery review of candidate profile and delivery interview for 45-60 mins
Working with TuringPay, getting started and work, from Turing's own pages
What the platform publishes in general. These are not extra terms confirmed for this particular role.
- Getting paid
- Paid monthly, around the 5th, straight to your bank account, or inside Deel on projects run through Deel. Its help articles disagree on whether that is the 5th calendar day or business day. help.turing.com/articles/9691407757-project-onboarding, help.turing.com/articles/6582454854-what-if-i-haven-t-rec..., help.turing.com/articles/1226770947-project-offboarding
- Before your first paid task
- A short application, with tests on some roles. Turing's review article says an interview is typically required for all roles. Onboarding covers the terms, ID verification with a driver's license or passport, and tax and payment details. help.turing.com/articles/2020987592-apply-for-jobs-at-turing, help.turing.com/articles/9179629201-how-long-does-the-app..., help.turing.com/articles/9691407757-project-onboarding
- How much work
- Review takes from 48 hours to 6 weeks after you finish applying, and the client makes the final choice. No reply within 6 weeks usually means you were not selected. help.turing.com/articles/9179629201-how-long-does-the-app..., help.turing.com/articles/2020987592-apply-for-jobs-at-turing
- Contract
- Depending on the project, the engagement rests on Turing's Terms of Service and a Service Agreement, on Deel, or on a partner that provides the contract. help.turing.com/articles/6582454854-what-if-i-haven-t-rec...
Read on the pages linked above on 23 Sep 2026. Platforms change these terms, so check the linked page before you rely on one.
How does the pay compare?Across Coding & software eval roles, not this role alone
From our own daily reads of every open listing on the platforms we follow, not from Turing's posting. How we count.
Pay across the field: Coding & software eval
Across 392 roles in this field with a stated hourly rate in US dollars, on 14 platforms, the middle half pays $65 to $120 an hour at the top of the range, and the median is $95.
| Platform | Roles | Top of range, per hour |
|---|---|---|
| Alignerr | 137 | $50 to $400 · median $100 |
| micro1 | 44 | $60 to $350 · median $120 |
| Handshake AI | 43 | $47 to $125 · median $80 |
| Meridial (Invisible) | 34 | $22 to $150 · median $60 |
| Mercor | 33 | $60 to $200 · median $105 |
Sources and listing historyWhere this information comes from
This listing's history
- First seen here on 8 Oct 2026, and still on Turing's list when we last read it on 13 Oct 2026.
- Posted by Turing
- First seen by AIPayList
- Last seen live
- Listing ID
- 111557
Turing posts this role once per country: 6 listings with the same title and pay. This page shows one of them. The full country list is on the main listing.
Keep exploring
More like this
Related field, different requirements. Check each role's own page before you apply.