About the role
As a Member of Technical Staff on Evals, you'll own how we measure AI Employee performance on the job.
Areas you may work in:
- Evaluation systems: datasets, offline replay, scorers, and regression alerts
- Feedback loops for models and agents
- Autonomous failure analysis
- Defining what good, degraded, and failed runs look like, and alerting on them
You may be a good fit if you:
- Have built evaluation or measurement systems, such as AI evals, experimentation, ranking, or search quality
- Can turn ambiguous quality questions into concrete metrics and decisions
- Have strong software engineering fundamentals and ship production systems
- Are comfortable where there are few established patterns
- Want to work in person in San Francisco
Even better:
- You've shipped LLM systems to production and learned where they break
- You have strong data instincts and work well with researchers
- You contribute to open source projects
About Jarmin AI
Jarmin AI is a Y Combinator-backed startup (F25 batch) building autonomous AI employees for human-level roles including ML Engineer, Financial Analyst, Account Executive, Paralegal, and Compliance Analyst. Founded in 2025 by three engineers with backgrounds at Meta SuperIntelligence Labs, Apple, and AWS, the company's core thesis is that every company should have access to top-tier AI/ML capabilities regardless of whether they can recruit or afford specialized talent. The product learns its assigned responsibilities, connects to a company's internal data and systems, and operates autonomously within existing workflows.
Required skills
Other roles at Jarmin AI
Refer someone
Know someone great for this role?
Share your link. If they get hired through Standout, you get $3,000.
Job details
Salary
$200,000 - $300,000
Equity
0.25% - 1%
Location
San Francisco, CA
Experience
3+ years
Funding
Total raised
$500K
Last stage
Seed
Investors
Founders
What happens next.
No applications, no recruiter spam. Just the intro.
Confirm the fit
A few questions to make sure this role is the right shape for you. Two minutes.
I pitch you to the company
I write the intro, send it to the founder, and handle the back-and-forth.
A meeting lands on your calendar
If they’re a yes, I book the chat. You show up — that’s the whole job-hunt.