AlignerrCoding & software eval
Machine Learning Evaluation Specialist
up to $400/hr
About this role, in Alignerr's words
What if your years of research and domain expertise could directly shape how the next generation of AI models think and reason? That's exactly what this role offers.
What you'll do
- Design complex, original machine learning problems rooted in your specialized domain of expertise
- Craft evaluation tasks that go beyond standard ML pipelines and require genuine advanced knowledge to solve
- Draw from your own research experience to build challenges that would stump a highly capable AI
- Write clear, precise problem statements with defined evaluation criteria and gold-standard solutions
- Assess AI-generated ML solutions for correctness, creativity, and methodological rigor
- Document problem difficulty, required domain knowledge, and expected failure modes
- Collaborate asynchronously with a global team of researchers and engineers
Candidate background
Who You Are
- You hold graduate-level expertise (MS or PhD preferred) in a scientific or technical domain intersecting with machine learning
- You have strong working knowledge of core ML methods — model selection, feature engineering, evaluation metrics, and pipeline design
- You're deeply familiar with active, open research problems in your field
- You can pinpoint exactly where general ML knowledge falls short and specialized domain insight becomes essential
- You have experience conducting or publishing original research (highly valued)
- You communicate complex ideas clearly and precisely in writing
- You're self-motivated and thrive when working independently on intellectually demanding problems
Quoted from selected sections of Alignerr's posting. The full posting may state other conditions. Read the full posting
A closer look
More context, when you need it.
Full job descriptionThe original posting, in Alignerr's words
Posted by Alignerr, reproduced here so you can judge the role before clicking. Original posting ↗
Machine Learning Evaluation Specialist (AI Training)
About the Role
What if your years of research and domain expertise could directly shape how the next generation of AI models think and reason? That's exactly what this role offers.
We're looking for ML experts and domain specialists to design fiendishly difficult evaluation problems that push state-of-the-art AI systems to their limits. This isn't about textbook prompts — it's about crafting the kind of challenges that only someone with deep, hard-earned expertise can build and judge.
Your work will directly influence how leading AI models are measured, improved, and deployed.
- Organization: Alignerr - Type: Hourly Contract - Location: Remote - Commitment: 10–40 hours/week
What You'll Do
* Design complex, original machine learning problems rooted in your specialized domain of expertise * Craft evaluation tasks that go beyond standard ML pipelines and require genuine advanced knowledge to solve * Draw from your own research experience to build challenges that would stump a highly capable AI * Write clear, precise problem statements with defined evaluation criteria and gold-standard solutions * Assess AI-generated ML solutions for correctness, creativity, and methodological rigor * Document problem difficulty, required domain knowledge, and expected failure modes * Collaborate asynchronously with a global team of researchers and engineers
Who You Are
* You hold graduate-level expertise (MS or PhD preferred) in a scientific or technical domain intersecting with machine learning * You have strong working knowledge of core ML methods — model selection, feature engineering, evaluation metrics, and pipeline design * You're deeply familiar with active, open research problems in your field * You can pinpoint exactly where general ML knowledge falls short and specialized domain insight becomes essential * You have experience conducting or publishing original research (highly valued) * You communicate complex ideas clearly and precisely in writing * You're self-motivated and thrive when working independently on intellectually demanding problems
Example Domains (Not Exhaustive)
We're actively seeking specialists across a wide range of fields, including:
* Computational biology, genomics, or bioinformatics * Climate science and environmental modeling * Medical imaging and healthcare ML * Materials science and computational chemistry * Astrophysics and signal processing * NLP for low-resource or specialized corpora * Robotics, control theory, or reinforcement learning * Financial modeling and quantitative analysis
If your domain isn't listed, we still want to hear from you — if your expertise is deep and niche, it's valuable.
Why Join Us
* Work at the frontier — contribute directly to AI evaluation and safety research that matters * Make your expertise count — finally, a role that rewards deep domain knowledge, not just generalist skills * Collaborate globally — work asynchronously alongside top researchers and engineers around the world * Full autonomy — flexible schedule, no micromanagement, work when and how you do your best thinking * Ongoing opportunity — strong potential for contract extension and deeper research involvement * Build your profile — be recognized as a contributor to cutting-edge AI development
Working with AlignerrPay, getting started and work, from Alignerr's own pages
What the platform publishes in general. These are not extra terms confirmed for this particular role.
- Getting paid
- Billed weekly, Monday to Sunday, with approved work paid by the following Friday, according to Labelbox's help center. Payment goes through Stripe Connect or PayPal depending on your location, with one provider per country. intercom.help/labelbox/en/articles/13600616-alignerr-paym..., alignerr.com/process
- Before your first paid task
- Sign up, apply, verify your identity, set up billing and onboard. Some jobs add a Zara AI interview or domain assessments. alignerr.com/process
- How much work
- Time to your first project depends on demand and your skills, and there is no minimum number of hours. alignerr.com/faqs
- Contract
- Independent contractor, under terms with ALIGNERR LLC. alignerr.com/faqs, alignerr.com/privacy
Read on the pages linked above on 24 Sep 2026. Platforms change these terms, so check the linked page before you rely on one.
How does the pay compare?Across Coding & software eval roles, not this role alone
From our own daily reads of every open listing on the platforms we follow, not from Alignerr's posting. How we count.
Pay across the field: Coding & software eval
Across 393 roles in this field with a stated hourly rate in US dollars, on 14 platforms, the middle half pays $65 to $120 an hour at the top of the range, and the median is $95. This role's top rate, $400, is in the top quarter. Of the 140 distinct rates we count, 140 are lower and 0 are higher.
| Platform | Roles | Top of range, per hour |
|---|---|---|
| Alignerr (this platform) | 136 | $50 to $200 · median $100 |
| micro1 | 46 | $35 to $350 · median $110 |
| Handshake AI | 43 | $47 to $125 · median $80 |
| Meridial (Invisible) | 34 | $22 to $150 · median $60 |
| Mercor | 33 | $60 to $200 · median $105 |
Sources and listing historyWhere this information comes from
This listing's history
- First seen here on 27 Aug 2026, and still on Alignerr's list when we last read it on 12 Oct 2026.
- The listed pay has not changed since then.
- Posted by Alignerr
- First seen by AIPayList
- Last seen live
- Listing ID
- 0e671fa7-f786-42be-9d53-6f97cff0a18f
Pay as the source states it: “$200-400/hr”.
Platforms sometimes change a rate after posting. Every listing here is re-read at least once a day and corrected when its source moves, so a difference you see on the platform is usually a change made since the last read. How freshness works.
Keep exploring
More like this
Related field, different requirements. Check each role's own page before you apply.