Handshake AIRating & model evaluation
Domain Expert, AI Evaluation (US)
up to $200/hr
About this role, in Handshake AI's words
Handshake is recruiting Master's and PhD holders residing in the United States to partner with leading AI labs and join our AI research community. This program brings subject-matter experts together to enhance the capabilities of Large Language Models (LLMs) across specialized scientific and technical domains. The Handshake AI Fellowship program runs year-round, though project availability will vary by discipline.
Candidate background
Qualifications
- Education : PhD students and graduates (including those currently enrolled in a PhD program or in postdoctoral positions), or Master’s degree holders with relevant experience, in Mathematics, Chemistry, Biology, Physics, Engineering, Computer Science, Medicine, etc.
- Experience : 2+ years of research or industry experience in your field, in addition to your degree
- Subject expertise : Deep subject knowledge and careful attention to detail, and confidence that your domain expertise can outmatch current AI systems in understanding and explaining key concepts in your field
- Written communication : Strong written communication, with the ability to translate complex research knowledge into clear, structured task instructions and evaluation criteria
Required for all applicants
- Must be based in the United States and have US work authorization
- Ability to work independently in a remote, asynchronous environment
- Availability of approximately 5 to 20 hours per week
Quoted from selected sections of Handshake AI's posting. The full posting may state other conditions. Read the full posting
A closer look
More context, when you need it.
Full job descriptionThe original posting, in Handshake AI's words
Posted by Handshake AI, reproduced here so you can judge the role before clicking. Original posting ↗
Overview
Handshake is recruiting Master's and PhD holders residing in the United States to partner with leading AI labs and join our AI research community. This program brings subject-matter experts together to enhance the capabilities of Large Language Models (LLMs) across specialized scientific and technical domains. The Handshake AI Fellowship program runs year-round, though project availability will vary by discipline.
Details
• The position is remote and asynchronous - work independently from wherever you are.
• Flexible hours and the ability to work remotely, with a commitment of approximately 5 to 20 hours per week.
• Project work includes, but is not limited to, writing and solving challenging problems in your area of expertise, developing domain-specific prompts and evaluating LLM responses for correctness and quality of reasoning.
• Dedicate time researching topics that are interesting to you, with the assistance of AI.
• Learn new skills while contributing to the changing world of AI across various disciplines.
• Placement into a project will be dependent on project availability. You'll apply once and, if qualified, be considered for projects as they open.
Qualifications
• Education : PhD students and graduates (including those currently enrolled in a PhD program or in postdoctoral positions), or Master’s degree holders with relevant experience, in Mathematics, Chemistry, Biology, Physics, Engineering, Computer Science, Medicine, etc.
• Experience : 2+ years of research or industry experience in your field, in addition to your degree
• Subject expertise : Deep subject knowledge and careful attention to detail, and confidence that your domain expertise can outmatch current AI systems in understanding and explaining key concepts in your field
• Written communication : Strong written communication, with the ability to translate complex research knowledge into clear, structured task instructions and evaluation criteria
Required for all applicants
• Must be based in the United States and have US work authorization
• Ability to work independently in a remote, asynchronous environment
• Availability of approximately 5 to 20 hours per week
Working with Handshake AIPay, getting started and work, from Handshake AI's own pages
What the platform publishes in general. These are not extra terms confirmed for this particular role.
- Getting paid
- Paid weekly through Stripe. Handshake says it usually sends payouts by Wednesday afternoon for the previous Monday to Sunday week, and a payment can often take 48 hours to process and appear. Some projects set their own schedule. Non-US tax residents face 30% US withholding unless they certify the work was done outside the US. ai-support.joinhandshake.com/articles/32263614087575, ai-support.joinhandshake.com/articles/33238954868759, ai-support.joinhandshake.com/articles/39964143294615
- Before your first paid task
- Its pages give different answers on who can join. Its AI page says you must be based in the US, its help center says people outside the US can join by invitation only, and some of its postings ask for Indian work authorization or a base in Canada. Identity verification, tax details and a payout method come before paid work, and a failed assessment usually cannot be retaken. joinhandshake.com/ai, ai-support.joinhandshake.com/articles/37145099086743, joinhandshake.com/ai/opportunities/physics-expert-india, joinhandshake.com/ai/opportunities/ai-evaluation-speciali..., ai-support.joinhandshake.com/articles/32244754462615, ai-support.joinhandshake.com/articles/36935929970711, ai-support.joinhandshake.com/articles/41129932721431
- How much work
- Applying does not guarantee a project, and there is no fixed timeline for an invitation. Projects can start, pause or end at short notice. ai-support.joinhandshake.com/articles/32244169079703, ai-support.joinhandshake.com/articles/33238954868759, ai-support.joinhandshake.com/articles/38150760132119
- Contract
- Contractor, under an agreement with Handshake AI Solutions, LLC that says you are not an employee. joinhandshake.com/legal/contractor-agreement
Read on the pages linked above on 24 Sep 2026. Platforms change these terms, so check the linked page before you rely on one.
How does the pay compare?Across Rating & model evaluation roles, not this role alone
From our own daily reads of every open listing on the platforms we follow, not from Handshake AI's posting. How we count.
Pay across the field: Rating & model evaluation
Across 156 roles in this field with a stated hourly rate in US dollars, on 19 platforms, the middle half pays $30 to $80 an hour at the top of the range, and the median is $45. This role's top rate, $200, is in the top quarter. Of the 70 distinct rates we count, 68 are lower and 1 are higher.
| Platform | Roles | Top of range, per hour |
|---|---|---|
| Meridial (Invisible) | 44 | $15 to $65 · median $30 |
| micro1 | 28 | $15 to $140 · median $45 |
| Prolific | 18 | $20 to $216 · median $40 |
| Alignerr | 11 | $120 |
| Innodata | 11 | $15 |
Sources and listing historyWhere this information comes from
This listing's history
- First seen here on 20 Nov 2026, and still on Handshake AI's list when we last read it on 21 Nov 2026.
- The listed pay has not changed since then.
- Posted by Handshake AI
- First seen by AIPayList
- Last seen live
- Listing ID
- 12015470
Pay as the source states it: “Up to $200/hr”.
Platforms sometimes change a rate after posting. Every listing here is re-read at least once a day and corrected when its source moves, so a difference you see on the platform is usually a change made since the last read. How freshness works.
Keep exploring
More like this
Related field, different requirements. Check each role's own page before you apply.