<div><p><a href="https://himalayas.app/companies/epoch-ai">Epoch AI</a> is looking for a Researcher to develop and publish critiques and reviews of AI benchmarks. </p><h3>About the role</h3><p>We are looking for a Researcher to produce a steady stream of benchmark reviews. You will closely analyze a wide variety of new benchmarks, evaluate their methodologies, and write up your findings in public-facing research. You should be comfortable using coding agents to help you, without delegating your judgment.</p><p>Examples of the kind of reports you would produce include our reviews of <a href="https://epoch.ai/publications/what-skills-does-swe-bench-verified-evaluate" rel="nofollow ugc noopener noreferrer" target="_blank">SWE-bench Verified</a>, <a href="https://epoch.ai/publications/what-does-osworld-tell-us-about-ais-ability-to-use-computers" rel="nofollow ugc noopener noreferrer" target="_blank">OSWorld</a>, and <a href="https://epoch.ai/publications/what-do-economic-value-benchmarks-tell-us" rel="nofollow ugc noopener noreferrer" target="_blank">economic value benchmarks</a>.</p><p>This role is fully remote; we are able to hire in many countries. We invite anyone who is interested to apply, regardless of background, experience, or credentials. Please do not include a cover letter, photograph, or headshot of yourself, or any personal information that is not relevant to the role for which you're applying (including marital status, age, identity traits, etc.). </p><p>If this role sounds interesting, we are also looking for <a href="https://jobs.lever.co/epoch-ai/de7b4c71-ece2-454a-be70-e7b75c5f3b23" rel="nofollow ugc noopener noreferrer" target="_blank">researchers on multiple other teams</a>. </p><h3>Applications are rolling.</h3></div><h3>Key Responsibilities</h3><ul><div><li><strong>Review benchmarks:</strong> Assess a new benchmark at least every two weeks. You’ll evaluate their methodology and what a “good performance” would imply about AI capabilities.</li><li><strong>Publish and maintain reports:</strong> Write up public-facing reports on new benchmarks and periodically update them as new versions release and models make progress.</li><li><strong>Dig into data:</strong> Examine individual tasks within benchmarks in detail, using coding agents to assist while retaining your own critical oversight.</li></div></ul><h3>What we're looking for</h3><ul><div><li><strong>You excel at written communication. </strong>Your first drafts rarely need much polishing; they’re clear and publication-quality with minimal revision.</li><li><strong>You think critically about research methodologies. </strong>You have strong opinions about what benchmarks and evaluations should look like, and can recognize when something seems wrong. </li><li><strong>You’re already familiar with existing AI benchmarks</strong>, their methodologies, strengths, and weaknesses. You have ideas for new benchmarks that don’t exist yet, but should. </li><li><strong>You’re comfortable work