December 11, 2025

Article

The Dialogue Dance: A New Framework for Measuring AI Agents

Recently, I was asked to design metrics for a set of early-stage AI agents.

In true Corporate Bard fashion, I wrote an opera.

The team needed two chords; I handed them a full overture with leitmotifs and mood lighting.

But beneath the theatrical flourish was a real problem: we’re still measuring AI agents like Web 2.5 products.

Recommendation quality. Relevance. Click-through. Discoverability.

As if someone slapped a conversational UI over the Amazon homepage and called it the future.