PassRidge Flex: new 3-4h/day tasks every week, paid hourly or per project. See open tasks →

SPECIALIST QA · 5-DAY SHORTLIST

Hire an AI / LLM Evaluation QA

PassRidge sends a first shortlist for an AI / LLM Evaluation QA in about 5 working days from a complete brief. Each candidate has completed a role-specific assignment that we review. A mid-level hire in the US has a market median of $100k a year; the placement fee is 15-22% of first-year pay, due after the start date.

Request an AI / LLM Evaluation QA Cost calculator

Reply within one working day. Candidates stay anonymous until they agree to an introduction.

Hiring models

Four ways to hire an AI / LLM Evaluation QA.

The same vetted candidates under four contracts. Figures are for a mid-level hire, from the market median for this role.

Pricing by region

Permanent hire

A one-time placement fee after the start date. The engineer joins your payroll.

US, mid-level: $100k median pay, fee $18,000 (18%)

Pricing

Contract

An engineer on a fixed term, billed monthly at pay rate plus 22%.

US, mid-level: $82 an hour to you, $67 an hour to the engineer

Pricing

Outsourced team

Augmentation, a dedicated team, project work or managed QA from one hourly rate.

Mid-level from $38 an hour, $90 in the US

Models

Flex capacity

Vetted testers for 3-4 hours a day on a scoped task, for one to three weeks.

Task rate plus a 30% fee, billed on approved hours

Pricing
Cost calculator

What would your AI / LLM Evaluation QA team cost?

Compare a permanent hire, a contractor and each outsourcing model for this role. Change the inputs and the figures update. It is the same calculator as on the pricing page.

Pricing page

Your team

Used for permanent and contract hire.
Used for the five outsourcing models.

The calculator needs JavaScript. The tables below list the same figures for AI / LLM Evaluation QA by region and level.

Estimates for planning, not a quote. Permanent hire includes employer on-costs (15-32% by region), the placement fee and $3,600 a year for equipment and software. Outsourcing models bill 150 hours a month on average (1,800 a year after leave and public holidays). Management time, severance and exchange-rate movements are not included.

Rates by region

AI / LLM Evaluation QA pay and rates, published.

Pay is the market median for the role and level. Every rate below sits on top of it; none is lower.

AI / LLM Evaluation QA salary guide

Permanent

US dollars a year: market median pay by region and level, and the placement fee under it. Ranges span the countries in each region.

AI / LLM Evaluation QA permanent rates by region and level
RegionJuniorMidSeniorLead
United StatesNew York, Austin, San Francisco$71kfee $11k$100kfee $18k$150kfee $30k$190kfee $42k
United KingdomLondon, Manchester, Edinburgh$42kfee $7k$61kfee $11k$88kfee $18k$114kfee $26k
Western EuropeBerlin, Amsterdam, Lisbon, Barcelona$24–43kfee $4–7k$35–64kfee $7–12k$50–90kfee $10–18k$65–120kfee $15–27k
Central & Eastern EuropeWarsaw, Krakow, Prague, Bucharest$22–29kfee $4–5k$31–43kfee $6–8k$44–61kfee $9–13k$56–79kfee $13–18k
Latin AmericaSao Paulo, Buenos Aires, Mexico City, Bogota$28–34kfee $5k$42–49kfee $8–9k$60–70kfee $12–14k$78–91kfee $18–20k
AsiaBengaluru, Pune, Manila, Kuala Lumpur$12–17kfee $2–3k$17–25kfee $3–5k$24–34kfee $5–7k$32–45kfee $7–10k

Swipe the table sideways to see every level.

Contract

US dollars an hour. The large figure is what you pay PassRidge. The small figure is the engineer's pay rate.

AI / LLM Evaluation QA contract rates by region and level
RegionJuniorMidSeniorLead
United StatesNew York, Austin, San Francisco$58pay $48$82pay $67$122pay $100$155pay $127
United KingdomLondon, Manchester, Edinburgh$34pay $28$50pay $41$72pay $59$93pay $76
Western EuropeBerlin, Amsterdam, Lisbon, Barcelona$20–35pay $16–29$29–52pay $24–43$41–73pay $34–60$53–98pay $43–80
Central & Eastern EuropeWarsaw, Krakow, Prague, Bucharest$18–23pay $15–19$25–35pay $21–29$36–50pay $30–41$46–64pay $38–53
Latin AmericaSao Paulo, Buenos Aires, Mexico City, Bogota$23–28pay $19–23$35–40pay $28–33$49–57pay $40–47$64–74pay $52–61
AsiaBengaluru, Pune, Manila, Kuala Lumpur$10–14pay $8–11$14–20pay $12–17$20–28pay $16–23$26–37pay $21–30

Swipe the table sideways to see every level.

Outsourced

US dollars an hour for staff augmentation. Dedicated teams and managed QA are on the pricing page.

AI / LLM Evaluation QA outsourced rates by region and level
RegionJuniorMidSeniorLead
United StatesNew York, Austin, San Francisco$64$90$135$170
United KingdomLondon, Manchester, Edinburgh$37$53$76$99
Western EuropeBerlin, Amsterdam, Lisbon, Barcelona$33–41$38–61$50–85$62–114
Central & Eastern EuropeWarsaw, Krakow, Prague, Bucharest$33$38$50–55$59–70
Latin AmericaSao Paulo, Buenos Aires, Mexico City, Bogota$33$41–47$59–68$76–88
AsiaBengaluru, Pune, Manila, Kuala Lumpur$33$38$50$59

Swipe the table sideways to see every level.

Contract and outsourced rates include on-costs and margin; the method is on the pricing page. Prices in US dollars.

Selection stages

How we select an AI / LLM Evaluation QA.

Six stages, with the assignment written for this role. We describe the stages instead of quoting pass rates: they change with the market and the brief.

  1. Brief

    A 15-minute call or the request form: role, level, stack, time zone, budget range. We tell you the same day whether the range fits the market.

  2. Source

    We search our network of QA professionals by stack and evidence, not keyword. Recruiters who have tested software read the profiles.

  3. Technical assignment

    Each candidate completes a short, role-specific assignment. We review the output ourselves, and you get the results with the profile.

    For this role: Design an evaluation set for a described assistant, score a batch of responses against a rubric and explain disagreements between raters.

  4. Recruiter interview

    Motivation, availability, pay expectations and right to work are confirmed, and the candidate agrees to the introduction.

  5. Shortlist

    Up to three profiles, anonymised until the candidate consents, each with assignment results, expected pay and notice period.

  6. Offer and start

    We coordinate interviews, references and the offer. For contract roles we handle onboarding and contracts. The fee is due after the start date.

Tip for your brief: Tell us what the model does, who uses it and what a bad answer costs. The rubric and the rater's domain knowledge follow from that.

Profile

What a shortlisted profile looks like.

An anonymised profile typical for this role. Real profiles carry the same four parts, and the name and employer stay hidden until the candidate agrees to an introduction.

  • Assignment results. The reviewed output of the role-specific exercise, not a self-rated skill list.
  • Evidence from past work. Three concrete facts about products, volumes and outcomes, confirmed in the recruiter interview.
  • Pay expectation and notice. What the candidate expects to earn and when they can start, so budget fit is known before you meet.
  • Consent to introduction. Identity and employer stay anonymised until the candidate agrees to be introduced.
AnonymisedCandidate A

Mid-level AI Evaluation QA

Experience
4 years in QA
Based in
Toronto, Canada
Notice period
3 weeks
Stack
PythonJiraPostman

Evidence

  • Built a 600-prompt evaluation set with a written rubric and rater guide
  • Runs red-team sessions for harmful, biased and off-policy outputs
  • Tracks regressions between model versions in a score sheet reviewed weekly

Assignment: Design an evaluation set for a described assistant, score a batch of responses against a rubric and explain disagreements between raters.

Job description

A AI / LLM Evaluation QA job description you can copy.

Choose a level and a country. The pay line starts at the market median for that level, so candidates see an honest range. Edit the team line to fit your product.

AI / LLM Evaluation QA (mid level)
Location: Remote, based in United States
Pay: $100k–$130k a year
Experience: 3-5 years in software testing

About the role
We are hiring an AI / LLM Evaluation QA (mid level) to join our product engineering team. Evaluates model outputs, writes rubrics and builds regression sets for AI features.

What you will do
- Write evaluation rubrics and golden datasets
- Rate and compare model outputs at scale
- Red-team prompts for safety and policy gaps
- Build regression suites for prompt and model changes
- Report quality metrics to product teams

What we are looking for
- 3-5 years in software testing
- Prompting
- Evaluation design
- Python
- Domain expertise
- Annotation tools
- Hands-on with Python, Jira, Postman, or tools like them
- Clear written English: bug reports and test notes others can act on

How we hire
1. A conversation with a recruiter about the role, your pay expectations and your start date
2. Role-specific exercise. Design an evaluation set for a described assistant, score a batch of responses against a rubric and explain disagreements between raters.
3. An interview with the engineers you will work with
4. An offer, with the pay range stated above

FAQ

Hiring an AI / LLM Evaluation QA: your questions.

What is LLM evaluation?

Measuring how a model or an assistant behaves against defined criteria: accuracy, safety, tone and consistency. It combines test design, a rubric, rater calibration and automated checks.

Do evaluators need to code?

Basic Python helps with batch scoring and analysis. Many evaluators are domain specialists, in law, health or finance, working from a rubric.

How fast can you send a first shortlist for an AI / LLM Evaluation QA?

The median for this role is 5 working days from a complete brief. The assignment review is the longest step, and a brief with a clear stack and pay range shortens it. If your requirements are unusual we say so on the first call.

How much does it cost to hire an AI / LLM Evaluation QA?

Permanent placement is 15-22% of first-year base salary by level, paid after the hire starts. For a mid-level hire in the US that is $18,000 on a market median of $100k. Contract engineers are billed at the pay rate plus 22%. Use the calculator above for your country and team size.

What if the hire does not work out?

If a permanent hire leaves, or is dismissed for performance, within 90 days of starting, we run a replacement search at no extra fee. This is a clause in the client agreement, not a marketing promise.

Do you only send CVs?

No. Every shortlisted candidate has completed a role-specific technical assignment that we review, and a recruiter who has worked in testing has spoken to them. You get the assignment results with the profile.

Can I hire outside my country?

Yes. We recruit in 53 countries. For permanent hires abroad we can act as employer of record where you have no local entity. Contract engineers invoice through our local entity.

Request

Request an AI / LLM Evaluation QA.

Send level, country and a budget range. A recruiter confirms within one working day whether the range fits the market, and the first shortlist follows in about 5 working days.

JuniorMidSeniorLead

Related pages

Other roles
Manual QA TesterQA Automation EngineerSDETQA Lead / ManagerMobile QA EngineerPerformance Test EngineerSecurity / Penetration TesterGame QA TesterAccessibility TesterBusiness / Test AnalystData / ETL QA EngineerLocalization QA TesterRelease / DevOps EngineerTechnical Writer (QA / Docs)
PassRidge / start here

Test the market, not your patience.

Tell us the role you want or the role you need. A QA-only recruiter replies within one working day.

Get a personal recruiter I'm hiring

Choose your location

We use your location to show local pay, jobs and the right PassRidge legal entity. It never changes the page URL.

Search PassRidge