iMerit ScholarsRating & model evaluation
AI Response Evaluation Analyst
$25/hr
About this role, in iMerit Scholars' words
We are looking for detail oriented analysts to evaluate and rank AI generated responses to text based prompts. You will judge answers on accuracy, relevance, clarity, conciseness, safety, localization, and how well they follow the user's instructions, then explain your reasoning in writing. Much of the job comes down to this: look at the image, look at what the model said about it, and decide whether the two actually match.
Candidate background
What this role asks for
- French
Read from the posting's own words: show the exact sentences
- French: “listed by the platform under required skills: French”
A closer look
More context, when you need it.
Full job descriptionThe original posting, in iMerit Scholars' words
Posted by iMerit Scholars, reproduced here so you can judge the role before clicking. Original posting ↗
We are looking for detail oriented analysts to evaluate and rank AI generated responses to text based prompts. You will judge answers on accuracy, relevance, clarity, conciseness, safety, localization, and how well they follow the user's instructions, then explain your reasoning in writing. Much of the job comes down to this: look at the image, look at what the model said about it, and decide whether the two actually match.
What you bring A sharp eye for visual detail and the patience to look twice Strong critical thinking, research skills, and reading comprehension Sound judgment and well supported decision making Professional written English, including the ability to describe what you see precisely The consistency to apply detailed guidelines the same way, task after task Experience in image annotation, content review, editing, research, design, or localization is welcome but not essential.
Review images alongside the prompts and responses attached to them.
Interpret both visual and conversational context before scoring anything.
Compare multiple responses and rank them against defined criteria.
Confirm that descriptions and claims about an image reflect what is actually there.
Verify factual claims using approved research sources.
Flag tasks that cannot be reliably evaluated.
Write clear justification for every ranking you submit
Independent contractor engagement for the length of the project.
Fully remote with flexible scheduling around agreed deadlines.
Tasks include visual content, so a device and connection that display images comfortably are required.
Volume varies with project demand.
A paid qualification assessment is required before live work begins.
Location: France
Working with iMerit ScholarsPay, getting started and work, from iMerit Scholars' own pages
What the platform publishes in general. These are not extra terms confirmed for this particular role.
- Getting paid
- Direct bank transfer in US dollars only, started every Friday, with arrival committed within 21 days of timesheet approval, according to its older knowledge base. imerit-docs.gitbook.io/scholars-legacy-kb/working/invoice...
- Before your first paid task
- According to its older knowledge base, selection may include online assessments, interviews or practice work and normally takes up to four weeks, then onboarding with a criminal record check and ID verification. imerit-docs.gitbook.io/scholars-legacy-kb/selection-and-o..., imerit-docs.gitbook.io/scholars-legacy-kb/getting-started..., imerit-docs.gitbook.io/scholars-legacy-kb/getting-started...
- How much work
- According to its older knowledge base, there is no guarantee of work at selection or after onboarding, and volume can be uncertain even on a confirmed project. imerit-docs.gitbook.io/scholars-legacy-kb/selection-and-o..., imerit-docs.gitbook.io/scholars-legacy-kb/selection-and-o...
- Contract
- Contractor under a Master Services Agreement with iMerit, which sets no rate or hours and does not commit iMerit to offer work, according to its older knowledge base. imerit-docs.gitbook.io/scholars-legacy-kb/getting-started...
Read on the pages linked above on 23 Sep 2026. Platforms change these terms, so check the linked page before you rely on one.
How does the pay compare?Across Rating & model evaluation roles, not this role alone
From our own daily reads of every open listing on the platforms we follow, not from iMerit Scholars's posting. How we count.
Pay across the field: Rating & model evaluation
Across 162 roles in this field with a stated hourly rate in US dollars, on 13 platforms, the middle half pays $30 to $65 an hour at the top of the range, and the median is $45. This role's top rate, $25, is in the bottom quarter. Of the 48 distinct rates we count, 8 are lower and 38 are higher.
| Platform | Roles | Top of range, per hour |
|---|---|---|
| Meridial (Invisible) | 98 | $15 to $150 · median $50 |
| Rise Data Labs | 15 | $40 to $50 · median $45 |
| micro1 | 10 | $15 to $150 · median $45 |
| Alignerr | 9 | $120 |
| Handshake AI | 7 | $15 to $200 · median $60 |
Sources and listing historyWhere this information comes from
This listing's history
- First seen here on 13 Sep 2026, and still on iMerit Scholars's list when we last read it on 13 Oct 2026.
- The listed pay has not changed since then.
- Posted by iMerit Scholars
- Not stated
- First seen by AIPayList
- Last seen live
- Listing ID
- FRA-046
Pay as the source states it: “$25/Hour”.
Platforms sometimes change a rate after posting. Every listing here is re-read at least once a day and corrected when its source moves, so a difference you see on the platform is usually a change made since the last read. How freshness works.
Keep exploring
More like this
Related field, different requirements. Check each role's own page before you apply.