We are looking for an Applied AI Engineer with 1-10 years of industry experience to join our team at Judgment Labs. We are seeking intellectually curious individuals with high agency who can navigate open-ended problems and build AI systems end-to-end. This role is critical to enhancing the accuracy and scalability of our AI agents, which analyze hundreds of millions of data points to provide crucial insights for our customers.
What will you be doing?
Build better retrieval mechanisms and search algorithms to ensure our AI agent can access the right information quickly and make correct judgments.
Develop custom agent-specific harnesses optimized for manipulating and understanding agent trajectories, as out-of-the-box solutions are insufficient.
Design and build a sandboxed environment where our evaluation agent can run 24/7, enabling autonomous, self-improving systems for our customers.
Work hands-on with large, messy, real-world agent interaction data to extract meaningful evaluation signals and improve data quality through curation, pipelining, and benchmarking.
Develop agent-based systems to analyze and evaluate complex, long-running agent behaviors in production environments.
About Judgment Labs
Judgment Labs builds infrastructure for Agent Behavior Monitoring (ABM). While traditional observability focuses on logging exceptions and latency, our ABM surfaces behavioral anomalies such as instruction drifts and context retrieval loss in scaled production environments.
Hundreds of teams building autonomous agents rely on Judgment to understand how their systems are behaving post-deployment. Instead of reactive incident triage, they cluster patterns across conversations and workflows, correlate regressions to specific interaction types, and pinpoint where reliability breaks down in their usage context.
We’ve raised $30M+ across two rounds in the past five months. Our investors include Lightspeed, SV Angel, Valor Equity Partners, Nova Global, Chris Manning, Michael Ovitz, Michael Abbott, Cory Levy, Kevin Hartz, and others.
