As manufacturing models become ubiquitous, there is a significant need for fine-grained control over the manufacturing process. However, while controlled manufacturing methods from fine-tuning are proliferating, an important question remains unanswered: are these models really controllable in the first place? In this work, we provide a theoretical framework to formally address this question. To frame human-model interaction as a control process, we propose a novel algorithm to estimate controllable sets of models in a conversational setting. Notably, we provide formal guarantees on the measurement error as a function of the complex sample: we obtain approximately correct bounds for the controllable fixed measurements that can be distributed, we do not use assumptions outside the output limit, and we work with any black-box nonlinear control system (that is, any generative model). We convincingly demonstrate the theoretical framework for different tasks in the management of conversational processes, in both language and text-to-image production. Our results show that the control of the model is remarkably fragile and highly dependent on the experimental setting. This highlights the need for rigorous regulatory analysis, shifting the focus from merely trying to regulate to first understanding its fundamental limitations.
- † University of Pompeu Fabra
- ‡ Stanford University
