About the role
Own the machinery that makes Roark honest: the evaluation pipelines that score every call, the audio-native metrics that hear what transcripts miss, and the simulation engine that dials agents like a real caller. This is the product — if it is wrong, everything above it is wrong.
What you'll do: Build and scale evaluation pipelines that score thousands of production calls in near real time. Work on audio-native metrics — pronunciation, emotion, vocal stress — alongside LLM-based evaluators, and prove they agree with human judgment. Evolve the simulation engine: realistic personas, accents and noise, dialing real agents over the phone and WebRTC. Own analytics at scale: ClickHouse pipelines, DynamoDB streams, and the query layer that keeps dashboards honest. Orchestrate long-running work with Temporal — replays, backfills and load tests that converge instead of double-count. What you'll bring: 4+ years building backend or ML/audio systems in production, ideally in TypeScript or Python. Experience with at least one of: speech/audio processing, LLM evaluation pipelines, or high-volume data infrastructure. Rigour about correctness — you think in idempotency, replays and ground truth, not just happy paths. Comfort operating what you build: dashboards, alerts, and the 2am curiosity to find out why. Clear written communication — designs and post-mortems people actually want to read. Stack: TypeScript · Bun · AWS (SST) · ClickHouse · DynamoDB · Temporal · Postgres · WebRTC/telephony.
About Roark
Roark helps teams test, monitor, and improve their voice agents.
Voice AI is exploding, but reliability is still the #1 challenge. Teams spend hours manually testing agents and still miss failures. In the last 6 months, Roark has processed over 10M minutes of calls, powering monitoring and simulation for teams across the Voice AI ecosystem (including YC companies).
We close that gap by combining:
📊 Monitoring & Evaluation - 40+ built-in metrics (latency, instruction-follow, sentiment, etc.), custom dashboards, alerts, and the ability to define your own. Up to 15 speakers with automatic speaker identification.
🎭 Simulations & Personas - end-to-end phone/WebSocket simulations for inbound and outbound agents, with configurable personas (accents, languages, speech & behavior profiles). Define tests as conversations with a graph-based approach, so you can easily branch into edge cases and variants.
🔁 Full lifecycle loop - failed calls automatically become repeatable tests, so every failure makes your agent stronger.
Roark is the missing QA layer for Voice AI - helping teams ship agents they can trust.
Other roles at Roark
Refer someone
Know someone great for this role?
Share your link. If they get hired through Standout, you get $3,000.
Job details
Location
Remote
Funding
Total raised
$2M
Last stage
Seed
Investors
Founders
What happens next.
No applications, no recruiter spam. Just the intro.
Confirm the fit
A few questions to make sure this role is the right shape for you. Two minutes.
I pitch you to the company
I write the intro, send it to the founder, and handle the back-and-forth.
A meeting lands on your calendar
If they’re a yes, I book the chat. You show up — that’s the whole job-hunt.