Expert Talent Pool, Evals and Benchmarks (Associate)

Posted:
11/16/2026, 7:29:24 AM

Location(s):
San Francisco, California, United States ⋅ California, United States

Experience Level(s):
Mid Level ⋅ Senior

Field(s):
AI & Machine Learning

Workplace Type:
Remote

Pay:
$30–$60/hr

About the Role

Join a team that designs and runs human-led evaluations of AI models. You will help measure model capabilities by creating rigorous questions, rubrics, and comparative assessments, contributing to better understanding of AI performance.

What You'll Do

  • Write questions and tasks that test specific model capabilities.

  • Develop rubrics that define strong, partial, and weak answers.

  • Create reference answers and edge-case scenarios to challenge models.

  • Grade model outputs against rubrics and explain each score clearly.

  • Analyze incorrect answers, group error patterns, and document findings.

  • Compare multiple models using the same prompts.

  • Validate benchmarks and flag questions that may no longer provide reliable measurements.

  • Support small studies of how people evaluate or interact with model outputs.

What We're Looking For

  • Zero to five years of professional experience; prior AI-data work is not required.

  • Experience or training in QA, test engineering, assessment design, psychometrics, UX research, data analysis, ML evaluation, research support, or a subject area involving rigorous question writing.

  • Experience writing test cases, exam questions, rubrics, or evaluation tasks, and reviewing work consistently against written criteria.

  • Ability to use spreadsheets or Python to organize and analyze results, and to identify and document error causes.

  • Experience with evaluation tools such as Inspect, lm-eval-harness, HELM, or promptfoo, model comparisons, or user and rating studies is useful.

  • Clear, professional written communication.

Compensation & Benefits

Compensation is $30 to $60 per hour, depending on project, field, and experience. Payment is for hours worked.

Location

Fully remote. Projects typically run from one week to several months.

Clera

Website: https://www.getclera.com/

Headquarter Location: San Francisco, California, United States

Employee Count: 1-10

Year Founded: 2025

IPO Status: Private

Last Funding Type: Pre-Seed

Industries: Artificial Intelligence (AI) ⋅ Career Planning ⋅ Information Technology ⋅ Recruiting