Engineering Manager, Evaluations & Observability

You’ll guide engineers focused on:

• Evaluation platforms: building the frameworks that let us test and compare performance across LLM providers and model versions
• Quality systems: defining and enforcing rubrics, metrics, and evaluation loops that answer the hardest question in AI: “Is this actually good?”
• Data curation: managing the datasets that power and test our AI models, sourced from real-world usage to keep our systems grounded and relevant
• Search & retrieval quality: owning the retrieval layer that underpins both AI and non-AI experiences ensuring results are relevant, accurate, and fast
• Reusable AI quality infrastructure: creating the building blocks (evaluation tools, pipelines, and feedback systems) that other teams can leverage to maintain quality across Retool’s AI surface area
• Culture of continuous improvement: embed a data-driven approach to AI quality, where experimentation and measurement are the default as we scale our capabilities

IN THIS ROLE, YOU WILL:

• Communicate and collaborate effectively with Product and other Engineering counterparts
• Manage a team of engineers; support the team by identifying growth opportunities, providing continuous feedback, and performance management as appropriate
• Understand the needs of our Assist roadmap, helping define rubrics & automated systems that allow engineers to iterate quickly on product features, with confidence
• Establish and define your team's strategy to ensure execution maximizes business impact
• Introduce scalable, repeatable processes that help engineering and product teams deliver a successful product
• Partner with recruiting on building out a diverse team of exceptionally motivated engineers

**THE SKILLSET YOU'LL BRING: **

• 3+ years of experience successfully leading and managing teams
• Familiarity with AI evaluation & observability systems – we use Braintrust, but exposure to general evaluation & LLM observability systems is a plus
• Deep technical curiosity. You’ll be writing code, engaging deeply on design and architecture, and tinkering to uncover what’s really happening under the hood
• A strong ability to champion an engineering team through macro process changes
• A history of orienting teams and setting strategy toward engineering goals, with a bias toward impact
• A propensity to operate as a business-owner—caring deeply about our customers, product, and team
• Thoughtfulness around engineering culture, process, and identity
• Proficiency in navigating through ambiguity, managing stakeholders, communicating in a structured manner, as well as driving maximal accountability and excellence
• A collaborative skillset to partner effectively with product, design, and go-to-market teams

For candidates based in the United States, the pay range(s) for this role is listed below and represents base salary range for non-commissionable roles or on-target earnings (OTE) for commissionable roles. This salary range may be inclusive of several career levels at Retool and will be narrowed during the interview process based on a number of factors such as (but not limited to), scope and responsibilities, the candidate’s experience and qualifications, and location.

Additional compensation in the form(s) of equity and/or commission are dependent on the position offered. Retool provides a comprehensive benefit plan, including medical, dental, vision, and 401(k). Pay and benefits are subject to change at any time, consistent with the terms of any applicable compensation or benefit plans.

The base pay range for this role is $260,100 – $306,000 per year.

Retool offers generous benefits to all employees and hybrid work location. For more information, please visit the benefits and perks section of our careers page!

Retool is currently set up to employ all roles in the US and specific roles in the UK and Mexico. To find roles that can be employed in the UK and Mexico, please refer to our careers page and review the indicated locations.

A meeting lands on your calendar

If they’re a yes, I book the chat. You show up — that’s the whole job-hunt.