December 11, 2025
Article
The Dialogue Dance: A New Framework for Measuring AI Agents
Recently, I was asked to design metrics for a set of early-stage AI agents.
In true Corporate Bard fashion, I wrote an opera.
The team needed two chords; I handed them a full overture with leitmotifs and mood lighting.
But beneath the theatrical flourish was a real problem: we’re still measuring AI agents like Web 2.5 products.
Recommendation quality. Relevance. Click-through. Discoverability.
As if someone slapped a conversational UI over the Amazon homepage and called it the future.
