SQUIRE: Interactive UI Authoring with Slot QUery Intermediate Representations

Frontend developers build UI prototypes to test alternatives, which is a time-consuming process of iteration and refinement. Productive AI coding assistants enable rapid prototyping by providing a conversational interface instead of writing code. However, while this interaction gives developers the ability to be flexible as they can write any information they desire, it makes it challenging to control what is produced. First, natural language itself can be confusing, making it difficult for developers to clearly communicate their intentions. Second, the model may respond unpredictably, requiring the developer to iterate through trial and error to correct any unwanted changes. To address this weakness, we introduce Squire, a system designed for guided prototype testing and refinement. In Squire, the developer gradually builds the UI component tree by pointing and clicking on alternatives suggested by the program. An additional charge allows the developer to customize the appearance of the target UI. All interactions are clearly defined, with guarantees about which parts of the UI will and won't be changed. The system is supported by a novel central representation called SquireIR which is supported by a language for controlled testing and refinement. Through a user study where 11 frontend developers use Squire to implement mobile web prototypes, we find that developers are successfully testing and iterating on alternative UI approaches with high levels of visual control. The developers also scored the Squire well for usability and general satisfaction. Our findings suggest that strong coding capabilities are managed in fast UI modeling tools by combining conversation with clearly defined capabilities.