▲Agent harness optimizationSeen Aug 12 – Aug 28
The agent misplaces components in 71% of layout calls — the tool sends no units
High confidence1,240 sessions$12K conversion
Infrastructure for continuous agent improvement
Know how your agent behaves once it ships.
See what’s working for your users, from anonymized UI and agent traces.Connect your coding agents and track what every change does.
Your dashboards are green. Your users aren’t.
Get the full story behind every finding.
◆Product suggestionSeen Aug 01 – Aug 29
Translation requests grew 5× in a month — nothing in the product does it
Medium confidence1,140 sessions$50K conversion
▲AI cost and valueSeen Aug 18 – Aug 29
72% of token spend goes to free users active for over three months
High confidence1,847 users$34K conversion
●Agents as usersSeen Aug 10 – Aug 29
Agents chain three CLI commands into one you never built — a third of them fail
High confidence40 accounts$22K churn at risk
▲Retention and churnSeen Jul 30 – Aug 29
Chat discovery in week 1 lifts 30-day retention by 14% — onboarding never mentions it
High confidence2,100 users$50K churn at risk
●Cross-surface frictionSeen Aug 15 – Aug 29
Users ask the chat to “mark all as read” — when the UI option is right there
High confidence340 users$18K churn at risk
●Cross-surface frictionSeen Aug 15 – Aug 29
Users ask the chat to “mark all as read” — when the UI option is right there
High confidence340 users$18K churn at risk
TL;DR
- ►
340 of the 1,210 users who opened the inbox asked chat instead — even though the UI was visible in every session.
- ►
This detour costs 15 extra seconds — and 20% of times cancel is hit before completion.
- ►
They are not new users — 71% had been active for over a month, so they are familiar with the rest of the product. Just not this layout.
- ►
Chat answers correctly every time — the request succeeds, which is why nothing in your error rates or support volume moves.
◆
What to do about it
1
Put the button where users are looking.
Users open chat from the lower right and scan left. The button sits top-left, outside that path.
Prototype ›2
Let chat teach the behavior.
When chat handles it, have it also point out the UI. The detour then costs 15 seconds once instead of every time.
Explore fix ›Who took the detour
New users12%
Power users47%
Account champions41%
When the cancel happens
Before any result52%
Part-way down the list31%
At the confirm step17%
Compound what you learn in production.
Every surface, human and agent.
Your customers are both human and agent. Otis analyzes their interactions with your AI product across every surface — UI, chat, agentic workflows, MCP and CLIs — and finds what they need.
Evidence, not dashboards.
Otis gives you stories, not dashboards. It suggests actions to fix issues, improve your agent, simplify your UI, and drive user value. You get to focus on the decision, not data-gathering.
What your evals can’t tell you.
Otis watches every PR and experiment you ship, and learns which ones actually worked. Its understanding of your customers compounds with each one. That learning stays private to your product.
Bring your judgement — we’ve done the plumbing.
Every product signal, integrated.
Connect telemetry from every product surface with context from support tickets, forums, and call transcripts.
Your dev stack, integrated.
Otis connects to GitHub, and your tools for releases, experiments and evals.
Privacy for your users. Always.
The Otis SDK anonymizes all telemetry and redacts PII. Otis never records your users’ browsers.
Statistics, handled.
Otis takes care of the statistical rigor, so you can trust the insights.
Your product’s ledger. No-one else’s.
Otis’ memory builds a deep understanding of your product and priorities. It’s used to improve your product, never anyone else’s.
Works where you do.
Otis is for engineering teams. Get daily briefings in Slack. Use Otis through your agents with MCP and skills.
Ship with evidence, not hope.
Where your agent fails without erroringWhich users to go talk toWhere users see most frictionWhich changes paid off in productionWhat agent behavior predicts user churn
Learn more about Otis with one of our founding team.
