Umhlahlandlela Wesethulo Wokuqoshwa Kwekhodi Okusebenzayo

# Isingeniso

Ukukhipha amakhodi okuvimbezela okusebenzayookwaziwa nangokuthi ukukhiqiza okuhleliwe noma ukususa amakhodi okuqondisiwe, kuhlanganisa amasu onjiniyela okuphoqa imodeli yolimi enkulu (LLM) ukuthi ikhiqize imiphumela yombhalo ethobela ngokuqinile i-schema yedatha eshiwo, uhlelo lolimi, noma isisho esivamile (regex) esigabeni sokukhetha amathokheni.

Ngomhlahlandlela oyisingeniso wokwenza amakhodi okusebenzayo kulesi sihloko, ngeke usadinga ukuncenga imodeli yakho ukuthi “ikhiphe i-JSON evumelekile ngaphandle kokufaka noma yikuphi ukubeka phansi”, ukuze nje wenze isibonelo. Ukukhipha amakhodi okucindezelayo kwenza kungenzeki ngokwezibalo ukuthi i-LLM ilethe noma yini ngaphandle kwezingqinamba ezichaziwe.

# Isebenza Kanjani I-Practical Constraint Decoding?

Ngenkathi inqubo evamile yokukhiqiza i-LLM isebenza “njengesenzo sokholo” lapho udlulisela khona ukwaziswa kumodeli futhi ingase ikhiphe lokho okufunayo (noma okungenzeka ungakukhiphi), ukukhishwa kwekhodi okusebenzayo kuthatha indlela ehluke ngokucashile. Ibheka ukwaziswa nokukhiqizwa kombhalo njengohlelo oluhlukile, olunezihibe. Lokhu kwenza kube nokwenzeka ukukhiya izinhlamvu ezithile ezingukhiye ekugcineni i-syntax ethile edingekayo, okuvumela imodeli ukuthi “igcwalise izikhala” phakathi.

Ingabe sizongena emininingwaneni eyengeziwe? Lapho i-LLM ikhipha ithokheni elandelayo yempendulo yayo, iqala ikhiqiza ivekhtha yezikolo ezingavuthiwe, noma amalogi – eyodwa yawo wonke amathokheni angaba khona kusilulumagama esiseduze. Lokhu ngokuvamile kubandakanya izinkulungwane zezinketho ongakhetha kuzo.

Kepha uma usebenzisa ukuqoshwa kokucindezela okusebenzayo, okuthile kwenzeka ngaphambili, ngaphambi kokuthi inqubo yokusho iqale: kwakhiwa umshini wesimo esilinganiselwelapho kuhlanganiswa khona umkhawulo oqondiwe — ngokwesibonelo, ngemodeli yePydantic kuPython. Esinyathelweni esinqunyiwe, umshini wesimo esilinganiselwe uhlola isimo samanje futhi unikeze a uhlu lwamathokheni alandelayo avunyelwe. Lolu “luhlu olumhlophe” luyi esetshenziswa njengemaski kuVector ye-LLM yamalogikangangoba kuwo wonke amathokheni angaphandle kwalolo hlu, ukungena kwawo kusethelwe kokungapheli okunegethivu, isb -inf ePython.

Ngemuva kwenqubo yokufihla ubuso, imodeli iyaqhubeka nokusebenzisa inqubo yayo ye-softmax evamile kanye nesampula njengenjwayelo (ngokusekelwe kumapharamitha afana nezinga lokushisa, i-top-p, noma i-top-k) “kumathokheni asaphila” ukuze ekugcineni ukhethe okungenzeka kakhulu futhi uyikhiqize.

Kungase kuzwakale sengathi ukusebenzisa le nqubo kulo lonke ulwazimagama oluhlanganisa izinkulungwane zamagama kungase kwehlise isivinini sokucabanga kwemodeli. Izindaba ezinhle: akunjalo. Imitapo yolwazi yesimanje yePython ithatha ithuba lokuthi isilulumagama se-LLM simile futhi asihlanganise kusengaphambili ngaphambi kokuba umsebenzisi aqale ukuthayipha ukwaziswa kwakhe. Umshini wombuso ngeke udinge ukubheka lonke ulwazimagama, futhi i-latency overhead yehliswa kakhulu.

Ngakho-ke, liyini izinga legolide lamanje lokuqalisa ukuqopha okusebenzayo kokucindezela? Ngokungangabazeki, i uhlaka umtapo wolwazi uthole lo mehluko. Kusivumela ukuthi sichaze futhi sidlule amamodeli e-Pydantic, izikimu ze-JSON, noma i-regex ngokuqondile enguqulweni esongwe yemodeli eqeqeshwe kusengaphambili, ngaleyo ndlela ivimbele inkululeko yayo ekukhiqizeni okuphumayo.

# Isibonelo

Ake sihambe ngesibonelo. Okokuqala, faka izinhlaka:

pip install outlines[transformers]

Manje ngekhodi:

from pydantic import BaseModel
import outlines
from transformers import AutoTokenizer, AutoModelForCausalLM

class UserProfile(BaseModel):
    name: str
    age: int
    is_active: bool

model_name = "TinyLlama/TinyLlama-1.1B-Chat-v1.0"

llm = AutoModelForCausalLM.from_pretrained(model_name)
tokenizer = AutoTokenizer.from_pretrained(model_name)

model = outlines.from_transformers(llm, tokenizer)
result = model("Extract the user: John is a 34 year old pilot.", UserProfile)

print(result)

Okukhiphayo:

{"name": "John", "age": 34, "is_active": true}

Lesi sibonelo sibonise indlela yokusebenzisa umtapo wohlaka ukuze usonge imodeli eqeqeshwe kusengaphambili kanye ne-tokenizer yayo, futhi siyiphoqelele ukuthi ikhiphe izinto ze-JSON ezichazwe ikilasi langokwezifiso esilichazile – eliqanjwe igama. UserProfile kanye nefa lePydantic BaseModel.

# Esonga

Ukukhishwa kwekhodi okusebenzayo kwesithiyo kunochungechunge lokuhwebelana. Ake sibheke ezinye zazo.

Amandla:

• Uma isetshenziswe ngendlela efanele, inikeza isiqinisekiso esingu-100% se-syntax efanele, isusa isidingo sokuhlaziya amabhlogo kukhodi yakho.
• Kusiza kakhulu ukulondoloza amathokheni ekwazisweni kwakho, okungasadingi izibonelo zezithombe ezimbalwa ezidla amathokheni ukuze kuboniswe imodeli ukuthi into elungile ye-JSON kufanele ibukeke kanjani, isibonelo.
• Inikela embusweni wentando yeningi wamamodeli amancane, iguqule imodeli “encane” yepharamitha engu-1B engabeka engozini izimo zokusetshenziswa kokukhiqizwa kwe-JSON ibe umakhi wedatha ongaphambuki.

Imikhawulo:

• Uma i-LLM ibidinga ukusho ukuthi ayikwazi ukuphendula okuthile, kodwa i-schema iphoqa ukuthi ikhiphe inombolo ephelele, ngokwesibonelo, izokwenza kanjalo, iyenze ingasathembekile ezimweni ezingavamile.
• Ekuqhutshweni kokuqala kwe-schema ye-Pydantic ngokumelene ne-LLM, kungase kube khona ukushuba kwamasekhondi ambalwa ukuze kwakhiwe umshini wesimo esilinganiselwe, okwenza umshini wokuqala ugijime kancane kakhulu – nakuba okuzolandela kuzoba bushelelezi.

Lesi sihloko sethula ukuqoshwa kwekhodi okusebenzayo, kwembula ukuthi kungani kudingekile ezimweni ezithile eziqhutshwa yi-LLM, ukuthi kusebenza kanjani, nokuthi isiphi isisombululo esamukelwa kabanzi kukwakheka kwezwe kwamanje: umtapo wolwazi. Isibonelo sokusetshenziswa kwayo sinikezwe ngokufanayo.

U-Iván Palomares Carrascosa ungumholi, umbhali, isikhulumi, kanye nomeluleki ku-AI, ukufunda ngomshini, ukufunda okujulile nama-LLM. Uqeqesha futhi aqondise abanye ekusebenziseni i-AI emhlabeni wangempela.