Member of Technical Staff, AI Evaluation

We are looking for a Member of Technical Staff, AI Evaluation with 5+ years of experience, blending physical engineering (mechanical, electrical, aerospace, robotics) and AI evals, to own how we measure Archie, our AI engineer for the physical world. You'll build the evaluation framework that shows how Archie compares to human engineers and what it takes to grow him from junior to senior engineer, and eventually to engineering superintelligence.

What will you be doing?

• Own Archie's evaluation framework from scratch, defining what "good" engineering design looks like when there's no single right answer
• Work with engineering SMEs, customers, partners and internal teams to learn how top engineers work and where Archie succeeds or fails
• Recreate real-world engineering tasks and failure modes in evaluation environments that the dev teams can rerun
• Design and refine automated judges and human evaluation processes, and check that judges match expert human judgment
• Turn eval results into clear priorities for improving Archie's engineering reasoning

Key Requirements

• 5+ years of experience across physical engineering and/or AI work. The ideal candidate is strong in both, but deep strength in one with working knowledge of the other is fine
• Hands-on AI evaluation experience: building eval tasks, designing judges, and running human evaluation for agentic systems
• Physical engineering background (mechanical, electrical, aerospace, robotics, power systems), enough to recognize strong engineering design
• The presence and confidence to lead work across SMEs, customers and cross-functional teams in a high-ownership role
• Experience in scrappy, early-stage environments building new things from scratch (physical AI or robotics companies a plus)