Basic speech models, such as Hubert and diversity, are trained first in large numbers of random communication and used for defeating of Downsmream. These models use the purpose of prediction mask, where the model is learning to predict details about the masked entries from the unattended context. The choice of purposes for predicting this framework affects their performance in good works. For example, the first-trained models receive representations read by the speaker related activities, while those first trained in Phonetics read appropriate representations. In addition, the targeted for predicting may vary on the extent possible. Previously trained models of acoustic good features do better in Douing activities, while those who are first trained in higher level are more effective in the content related activities. Despite the importance of predictive purposes, affecting design decisions were not properly learned. This work tests the design and their impact on the performance of the work. Our results indicate that commonly used for habert decisions can be the Suppertimal. We propose ways to construct more educational mechanisms and show their performance in progress in various activities.
