Active Agent Research Environment: Simulating Active Users to Evaluate Active Assistants
Active agents that anticipate user needs and perform tasks automatically hold great promise as digital assistants, yet the lack of realistic user simulation frameworks hinders their development. Existing modeling applications such as APIs for calling tools, fail to capture the state and sequence of user interactions in digital environments and make realistic simulation of the user impossible. Introducing the Proactive Agent Research Environment (Pare), a framework for building and testing proactive agents in digital environments. Pare models applications as finite state machines with structured navigation and a state-dependent action space for the user simulator, which allows for active user simulation. Building on this foundation, we present Pare-Bench, a benchmark for 143 different tasks including communication, productivity, planning, and lifestyle applications, designed to test content detection, goal prediction, intervention time, and orchestration of multiple applications.
- † University of California, Santa Barbara
- ‡ Independent Researcher
- § University of Washington
- ** Work done while at Apple
