Jobs / Mercor

Study Domain Expert: Judge Calibration

$90/hr

source wording: “90 USD HOUR”

Platform: Mercor
Posted: 5h ago · seen live 16 min ago
Eligibility: United States
Except 74 countries Mercor names. Show them
Mercor says it cannot hire residents of 86 countries and regions. The full list is on our Mercor page. Checked 2026-09-05. Check the current list.
To start: Account and CV, then an AI interview of about 20 minutes. Mercor says replies take 2 to 4 weeks.Want to know exactly what the process looks like? Step by step for Mercor.

Apply on Mercor ↗

What happens after you apply →

This site earns a referral commission from Mercor. That commission keeps the site running and pays for new features.

Platforms sometimes change a rate after posting. Every listing here is re-read at least once a day and corrected when its source moves, so a difference you see on the platform is usually a change made since the last read. How freshness works.

What this role asks for

Accounting

Read from the posting's own words: show the exact sentences

Accounting: “…Resources, Legal, Operations, IT, Administration, Marketing, Sales, Accounting, and Finance. What You'll Do • Sign an NDA before receiving any materials. • Complete…”

The role, as Mercor describes it

Role Overview

Mercor is running a structured judge-calibration study to measure how frontier AI models perform on realistic professional work and how experienced practitioners judge that work. We are sourcing experts across Human Resources, Legal, Operations, IT, Administration, Marketing, Sales, Accounting, and Finance.

What You'll Do

• Sign an NDA before receiving any materials.

• Complete a realistic, domain-specific task using supplied source files.

• Review model-generated outputs for the task, rank and score them, and provide evidence-based rationales.

• Provide structured feedback on the task and study design.

Time Commitment 13 hours during the study week, September 14–18, 2026.

Study Conditions

Read the rest of the description (7 more paragraphs)

• No rubric or golden answer is provided; professional judgment is the measurement.

• Use of LLMs or other AI assistants is prohibited at every stage.

Who We're Looking For

• 5+ years of hands-on professional experience in one of the listed domains.

• Currently or recently practicing, with clear, evidence-grounded written judgment.

• Able to complete the study within the stated work window.

Eligibility Restriction People who have worked on Project Alchemy in any capacity, or had access to its source-world data, are not eligible for this study.

Posted by Mercor, reproduced here so you can judge the role before clicking. Original posting ↗

Apply on Mercor ↗What happens after you apply →

Source: platform job feed · first seen 5h ago · ID list_AAABoIh-F0qzz9jk4eFB6bDa

Study Domain Expert: Judge Calibration

The role, as Mercor describes it

More like this