▲Agent harness optimizationSeen Aug 12 – Aug 28
The agent misplaces components in 71% of layout calls — the tool sends no units
High confidence1,240 sessions$12K conversion
Infrastructure for continual product improvement
Act on the hidden signal in your agentic product.
Turn anonymized UI and agent traces into product opportunities.Connect your coding agents and measure the outcomes.
Focus your judgement on what matters.
Act on opportunities immediately.
◆Product suggestionSeen Aug 01 – Aug 29
Translation requests grew 5× in a month — nothing in the product does it
Medium confidence1,140 sessions$50K conversion
▲AI cost and valueSeen Aug 18 – Aug 29
72% of token spend goes to free users active for over three months
High confidence1,847 users$34K conversion
●Agents as usersSeen Aug 10 – Aug 29
Agents chain three CLI commands into one you never built — a third of them fail
High confidence40 accounts$22K churn at risk
◆Retention and churnSeen Jul 30 – Aug 29
Week-one chat discovery lifts 30-day retention 14% — onboarding never mentions it
High confidence2,100 users$50K churn at risk
●Cross-surface frictionSeen Aug 15 – Aug 29
Users ask chat to “mark all as read” — the UI control is on screen every time
High confidence340 users$18K churn at risk
●Cross-surface frictionSeen Aug 15 – Aug 29
Users ask chat to “mark all as read” — the UI control is on screen every time
High confidence340 users$18K churn at risk
TL;DR
- ►
340 of the 1,210 users who opened the inbox asked chat instead — the control was rendered and on screen in every one of those sessions.
- ►
The detour costs 15 seconds against under a second for the control — and 20% of times cancel is hit before completion.
- ►
They are not new users — 71% had been active for over a month, so this is unfamiliarity with the layout rather than with the product.
- ►
Chat answers correctly every time — the request succeeds, which is why nothing in your error rates or support volume moves.
◆
What to do about it
Proposed solutions to explore
1
Put the control where the eye already is
Users open chat from the lower right and scan left. The control sits top-left, outside that path. Mirror it into the list header.
Prototype ›2
Let chat teach the control
When chat handles something the UI already does, have it name the control in the reply. The detour then costs 15 seconds once instead of every time.
Explore fix ›Who took the detour
New users12%
Power users47%
Account champions41%
When the cancel happens
Before any result52%
Part-way down the list31%
At the confirm step17%
Compound your product learning.
Cross-surface product intelligence
Otis analyzes users’ interactions with your AI product across every surface — UI, chat, agentic workflows, MCP and CLIs. Otis discovers what your customers need, both human and agent.
Rigorous, actionable insights
Otis gives you stories, not dashboards. It suggests actions to fix issues, improve your agent, simplify your UI, and drive user value. You get to focus on decision-making, not data-gathering.
Continual product eval and improvement
Otis watches every PR and experiment. It sees what works, and what doesn’t. This learning is private to your product, and compounds Otis’ understanding of your customers.
Bring your taste. We’ve done the heavy lifting.
Every product signal, integrated.
Connect telemetry from every product surface with context from support tickets, forums, and call transcripts.
Your dev stack, integrated.
Otis connects to GitHub, and your tools for releases, experiments and evals.
Privacy for your users. Always.
The Otis SDK anonymizes and PII redacts all telemetry. Otis never records your user’s browser.
Statistics, handled.
Otis takes care of the statistical rigor, so you can trust the insights.
Your product ledger. No-one else’s.
Otis’ agentic memory builds a deep understanding of your priorities and product. It’s used to improve your product, never anyone else’s.
Works where you are.
Otis is for PMs and engineers. Get daily briefings in Slack, personalized to your role. Use Otis through your agents with MCP and skills.
Start listening now.
The features most likely to improve retentionWhich users to go talk toWhere users see most frictionWhich users are getting most valueWhat agent behavior predicts user churn
Learn more about Otis with one of our founding team.
