BraintrustCybersecurity
Senior Application Security Engineer - AI Code Evaluation
up to $90/hr
About this role, in Braintrust's words
Help a top AI lab evaluate and improve large language models through security-focused coding tasks. Bring your software-engineering judgment and hands-on security experience to work involving vulnerabilities, exploit verification and security patches.
What you'll do
- Evaluate coding tasks involving software vulnerabilities, exploit verification and security patches.
- Create high-quality coding prompts and reference answers for benchmark-style problems.
- Evaluate model outputs for code generation, refactoring, debugging and implementation.
Show all 7 itemsHide the other 4 in What you'll do
- Identify and document model failures, edge cases and reasoning gaps.
- Compare private language models with leading external models.
- Build or configure coding environments for evaluation and reinforcement learning.
- Follow detailed annotation and evaluation guidelines consistently.
Candidate background
What this role asks for
- Python
- Data Annotation
- Software Engineering
Read from the posting's own words: show the exact sentences
- Python: “listed by the platform under required skills: Python”
- Data Annotation: “listed by the platform under required skills: Data Annotation”
- Software Engineering: “listed by the platform under required skills: Software Engineering”
Quoted from selected sections of Braintrust's posting. The full posting may state other conditions. Read the full posting
A closer look
More context, when you need it.
Full job descriptionThe original posting, in Braintrust's words
Posted by Braintrust, reproduced here so you can judge the role before clicking. Original posting ↗
Help a top AI lab evaluate and improve large language models through security-focused coding tasks. Bring your software-engineering judgment and hands-on security experience to work involving vulnerabilities, exploit verification and security patches.
This is a contracting engagement, with potential for a longer-term engagement. Remote candidates in the selected countries are elegible.
What you will do
• Evaluate coding tasks involving software vulnerabilities, exploit verification and security patches.
• Create high-quality coding prompts and reference answers for benchmark-style problems.
• Evaluate model outputs for code generation, refactoring, debugging and implementation.
• Identify and document model failures, edge cases and reasoning gaps.
• Compare private language models with leading external models.
• Build or configure coding environments for evaluation and reinforcement learning.
• Follow detailed annotation and evaluation guidelines consistently.
What you bring
• At least five years of professional software-development experience and strong Python skills.
• Hands-on experience with vulnerability research, exploit reproduction or verification, or implementing, backporting or validating security patches.
• The ability to apply structured evaluation criteria and write clear technical feedback.
• Fluency in written and spoken English.
Helpful, not required
• Professional code review, coding annotation, LLM/code evaluation or benchmark design.
• Knowledge of another programming language.
• Team leadership or mentoring experience.
Working with BraintrustPay, getting started and work, from Braintrust's own pages
What the platform publishes in general. These are not extra terms confirmed for this particular role.
- Getting paid
- Through a processor such as Stripe or Wise. On hourly contracts you invoice through Braintrust and payment starts within 30 days, or later if the client has not paid. Its AI training page says gig tasks are paid weekly. usebraintrust.com/site-service-fees-terms, usebraintrust.com/ai-training
- Before your first paid task
- A profile and a short AI interview, usually under an hour, then ID verification and skills validation. Certification takes 1 to 2 days, and matching starts after it. usebraintrust.com/for-talent
- How much work
- No guaranteed minimum or regular pay. Its terms say Braintrust does not introduce talent to clients or find projects for them. usebraintrust.com/talent-terms, usebraintrust.com/terms
- Contract
- Independent contractor. Each project is a contract directly between you and the client, under terms with Freelance Labs, Inc. usebraintrust.com/talent-terms, usebraintrust.com/terms
Read on the pages linked above on 23 Sep 2026. Platforms change these terms, so check the linked page before you rely on one.
How does the pay compare?Across Cybersecurity roles, not this role alone
From our own daily reads of every open listing on the platforms we follow, not from Braintrust's posting. How we count.
Pay across the field: Cybersecurity
Across 54 roles in this field with a stated hourly rate in US dollars, on 10 platforms, the middle half pays $60 to $120 an hour at the top of the range, and the median is $90. This role's top rate, $90, is in the middle half. Of the 27 distinct rates we count, 12 are lower and 13 are higher.
| Platform | Roles | Top of range, per hour |
|---|---|---|
| Alignerr | 32 | $55 to $120 · median $60 |
| Mercor | 4 | $90 to $250 |
| micro1 | 4 | $95 to $175 |
| AfterQuery | 4 | $75 to $135 |
| DataAnnotation | 3 | $125 |
Sources and listing historyWhere this information comes from
This listing's history
- First seen here on 29 Sep 2026, and still on Braintrust's list when we last read it on 13 Oct 2026.
- The listed pay has not changed since then.
- Posted by Braintrust
- First seen by AIPayList
- Last seen live
- Listing ID
- 17919
Pay as the source states it: “USD 75–90 hourly”.
Platforms sometimes change a rate after posting. Every listing here is re-read at least once a day and corrected when its source moves, so a difference you see on the platform is usually a change made since the last read. How freshness works.
Keep exploring
More like this
Related field, different requirements. Check each role's own page before you apply.