SALESBENCH: A Long-Horizon, Agent-to-Agent Benchmark for Tool-Use Agent Behavior
Om Shastri ⋅ Hamza Mostafa ⋅ Dennis Lee
Abstract
Most agent benchmarks ask \emph{what} an agent achieves against a static rubric or fixed verifier. For deployed agents, an equally important question is \emph{how} they behave: how they allocate a budget, where they break, whether they remain compliant under optimization pressure, and whether a learned policy survives a counterparty it did not train against. We introduce \salesbench{}, a long-horizon, agent-to-agent benchmark for measuring those behaviors. A seller agent operates inside a deterministic insurance-sales simulator with ${\sim}100$ procedurally generated leads and a fixed time budget. It must maximize converted monthly recurring revenue by searching a customer database, quoting plans, placing calls, and closing deals, while a separate buyer language model, instantiated with one of ten personality archetypes, decides whether to accept, reject, or hang up. Because the world is a deterministic state machine, variance comes from the seller and buyer models; seller mistakes are separated from buyer-model failures; and four buyer-prompt variants turn counterparty overfitting into a quantitative test. We use \salesbench{} as both a training signal and an evaluation suite, training a 2B-parameter model with online reinforcement learning and analyzing its behavior. The trained policy learns in two phases, tool-interface mastery then selling skill, and at full scale is productive but brittle: it converts ${\sim}18$ leads using only ${\sim}11\%$ of the time budget before terminating on the invalid-action ceiling rather than on time. Compliance holds under revenue optimization (do-not-call violations stay near zero), and the policy transfers to three unseen buyer decision styles ($8\text{--}20\%$ per-lead conversion), degrading less than the base model as the pipeline doubles. Finally, reward shaping is fragile under RL: auxiliary terms become floor strategies once the model can collect them without selling. We release the environment, curriculum, behavioral instrumentation, and harness.
Chat is not available.
Successful Page Load