Kubiza Malini Ngempela Ukuqalisa I-LLM Yasendaweni? (Ama-Euro ngeMillion Tokens, Kukaliwe)
ngakho empeleni kumahhala.” Sengikushilo lokho, mhlawumbe ukushilo lokho, futhi wuhlobo lwesimangalo oluzwakala luyiqiniso kuze kube yilapho umuntu esilinganisa. Ngakho ngayilinganisa.
Ibhokisi lithi “ardi” — umshini owodwa ovulekileSUSE one-RTX 3090 eyodwa (24 GB). Kuyo ngiphethe ibhentshimakhi elawulwayo: amamodeli amathathu endawo ahlinzekwa ngu-ollama, ngalinye lishayelwa ngaphansi komthwalo ofanayo ongaguquki, kuyilapho ideshibhodi inenani ngalinye eliqhutshwa yi- amandla we-GPU wangempela washisa. Akusona isilinganiso se-thermal-design-power, hhayi ukuqagela kwe-back-of-envelope – isampula yamandla evela ku-nvidia-smi njalo ngemizuzwana eyi-10 futhi ihlanganiswe phezu kwesiqalo esinembile sokugijima ngakunye→iwindi lokugcina, emalini yami yangempela kagesi yosuku/yasebusuku: 0.30 BGN usuku / 0.18 BGN ebusuku (I-Bulgarian lev — isilinganiso sami sangempela sokusetshenziswa; siguqulelwe ku-EUR kulo lonke lolu cezu esikhonkwaneni esimisiwe se-ECB, 1 BGN ≈ €0.5113).
Umphumela uyinombolo eyodwa ngemodeli ngayinye: ama-euro ngesigidi amathokheni okukhiphayo. Amamodeli amahlanu kwayisishiyagalombili engiwalinganise afika eshibhile kune-API yefu esingethwe. Abathathu abakwazanga – futhi akubona abathathu ongabaqagela ekubaleni kwepharamitha kuphela.
Isimangalo bengifuna ukusihlola
Ukuhlakanipha kwabantu ukuthi ukucatshangelwa kwendawo kuyindlela eshibhile – uthenge i-GPU, ithokheni engaseceleni imahhala, ifu yinto esebenza imitha. Yonke ingxenye yalokho ifanelwe inombolo, ngakho nansi engiyilandele: izindleko zamandla amancane zokukhiqiza ithokheni endaweniimodeli ngemodeli, nokuthi inqwabelana kanjani kulokho okukhokhiswa i-API yesigaba esisingethwe esithi “Flash” ngethokheni efanayo.
Ukuze ngenze leso silinganiso hhayi ingxabano, ngangidinga izinto ezintathu ezingaguquki: umthwalo ofanayo wokusebenza kuwo wonke amamodeli, ihadiwe efanayo, nemitha engangiyethemba ukufunda i-GPU phakathi nokugijima ngakunye. Okwesithathu yingxenye enzima ekhaya – yilapho insimbi ingena khona.
Indlela
Amamodeli amathathu, akhethelwe ukwelula ububanzi bosayizi engiwagcina ebhokisini: gemma3:1b (amancanyana), gemma4:26b (Gemma 4 yakwaGoogle-25.8B amapharamitha ngaphansi kwe-hood; u-ollama uvele ayisondeze ku-“26b” kuthegi), kanye ne-gemma3:27b (enkulu endala). Zontathu ziyizisindo ze-Q4_K_M-quantized GGUF ezidonswe ku-ollama, ngakho-ke ukuqhathanisa kufana ne-quantization. Ngamunye uthole umsebenzi ofanayo ongaguquki – iluphu yesizukulwane samathokheni angama-256 sihamba ngebhayisikili ngokusebenzisa imiyalo emihlanu engaguquki – igcinelwe ~4 amaminithi (240s)ngakho-ke i-GPU ifinyelele esimweni esiqinile kunokuba yahlulelwe ocingweni olubandayo lokuqala.
Ngenkathi imodeli ngayinye igijima, ngayilandela njenge ukuhlolwa okunenani ku-HomeLab Monitor – ideshibhodi yami, umthombo ovulekile, i-MIT, isitsha esisodwa. Isampula amandla e-GPU asuka ku-nvidia-smi njalo ngemizuzwana eyi-10, futhi ekugijimeni okulandelwayo ihlanganisa amandla phezu kokuqala kokuqalisa→iwindi lokugcina libe yi-kWh, bese iphindaphinda nganoma iyiphi intela ebisebenza (usuku vs ebusuku) – ngemali engiyilungise ngayo, i-BGN . Lokho kuguqula ukuthi “ibize malini le modeli?” ukusuka ekuqageleni ukuya kusibalo esilinganisiwe; Bese ngiguqulela ku-EUR ukuze ngifundeke.
Bese kuba i-arithmetic eshintsha ukugijima kwemizuzu emi-4 ibe yinani eliqhathanisekayo:
€ per 1M output tokens = (run_cost_BGN × 0.5113) ÷ (output_tokens / 1,000,000)
Ibhentshimakhi yamamodeli amathathu engezansi ingaphinda ikhiqizeke ekugcineni ukuya ekugcineni. Nansi indlela yokuyimisa ebhokisini lakho.
Isinyathelo 1 – letha imonitha (isitsha esisodwa)
Emshinini one-GPU:
COMPOSE_URL=
curl -fsSLO "$COMPOSE_URL"
docker compose up -dVula i-http://
Isinyathelo sesi-2 – hlanganisa ukhiye wokungenisa
Iskripthi sebhentshimakhi sidinga ukuphusha singene kuhabhu, ngakho sidinga ukhiye. Izilungiselelo → Ukuhlanganiswa → Dala ukhiye. Ukhiye (uchungechunge lwe-hlm_…) uboniswa kanye — kopisha bese, ngeke uphinde uwubone. Yigcine ku-env var; ungayinamathiseli embhalweni.
export HOMELAB_KEY="hlm_xxxxxxxxxxxxxxxxxxxx"
Iphaneli efanayo ine a Landa iklayenti isixhumanisi sefayela lePython ozolisebenzisa ku-Isinyathelo sesi-3, kanye nesiqalo sokukopisha-namathisela – ngakho lesi sikrini esisodwa simboza izinyathelo ezimbili ezilandelayo.
Isinyathelo sesi-3 – bamba iklayenti elincane
Ihabhu isebenzela iklayenti layo lePython. I-stdlib-kuphela – akukho okungafakwa ngepayipi:
CLIENT_URL=http://:9800/static/homelab_run.py
curl -O "$CLIENT_URL" Lelo fayela elilodwa iyonke i-API yokulandelela. Iphethini incane ngamabomu: lungiselela kanye, bese ugoqa noma iyiphi ibhulokhi yomsebenzi kumphathi wokuqukethwe sebenzisa futhi ungene kumamethrikhi ngaphakathi kwayo.
import homelab_run as homelab
homelab.configure(url="http://:9800", key="hlm_...")
with homelab.run("llm-cost-gemma3-1b",
params={"model": "gemma3:1b", "engine": "ollama",
"gpu": "RTX 3090 24GB", "num_predict": 256}) as r:
# ... do the work ...
r.log_metric("tokens_per_sec", tps, step=i) Uma i- with block ivuleka, ihabhu iphawula isiqalo sokugijima; uma ivala, iphawula ukuphela futhi ibiza amandla e-GPU ashiswe phakathi. Iwindi yilona isilinganiso.
Isinyathelo sesi-4 – bhala ibhentshimakhi
Umthwalo womsebenzi ungoka-ollama/api/generate in loop, ukufunda amasimu u-ollama ubuyisela emuva ukuze sithole okwangempela amathokheni-ngesekhondi kunesilinganiso sewashi lodonga. Izinkambu ezimbili ezibalulekile ziyi eval_count (amathokheni akhiqiziwe) kanye eval_duration (ama-nanosecond asetshenziswe ukuwakhiqiza):
import json, time, urllib.request
OLLAMA = "
def ollama_generate(model, prompt):
body = json.dumps({
"model": model, "prompt": prompt, "stream": False,
"options": {"num_predict": 256, "temperature": 0.7},
}).encode()
req = urllib.request.Request(OLLAMA + "/api/generate", data=body,
headers={"Content-Type": "application/json"})
with urllib.request.urlopen(req, timeout=600) as resp:
return json.loads(resp.read())Bese ushayela imodeli ngayinye ngaphansi komthwalo isikhathi esinqunyiwe, esongwe ngokulandela ngomkhondo:
def bench(model, duration_s=240):
ollama_generate(model, "Say hello.") # warm up - exclude the cold start
run = homelab.run(f"llm-cost-{model.replace(':', '-')}",
params={"model": model, "engine": "ollama",
"gpu": "RTX 3090 24GB", "num_predict": 256,
"workload": "fixed-prompt-loop"},
tags=["llm-cost", "benchmark"])
tot_out = 0
t0 = time.time(); step = 0
with run as r:
while time.time() - t0 < duration_s:
res = ollama_generate(model, PROMPTS[step % len(PROMPTS)])
ec, ed = res.get("eval_count", 0), res.get("eval_duration", 1) # ed in ns
tps = ec / (ed / 1e9) if ed else 0.0
tot_out += ec
r.log_metric("tokens_per_sec", round(tps, 2), step=step)
step += 1
r.log_params({"total_output_tokens": tot_out})
return run.id, tot_outUcingo lokuzifudumeza ngaphambi kwewindi elinesikhathi ligcina ukubambezeleka kokuqala okubandayo ngaphandle kwesilinganiso. Isikhathi.ukulala(15) engikubeke phakathi kwamamodeli (angaboniswa) kugcina iwindi lentengo ngalinye lihlukene ngokuhlanzekile ukuze umsila wemodeli eyodwa ungangeni enkokhelweni elandelayo.
Isinyathelo sesi-5 – funda ukubuyela emuva, okunentengo
Qalisa iskripthi sawo womathathu amamodeli bese uvula ifayela Izivivinyoithebhu. Ukugijima ngakunye manje kuwumugqa ophethe amandla awo angempela nezindleko:
Chofoza ku-run bese uthola ijika lamandla e-GPU kulelo windi ngqonamagama-ncazo anentengo ngezansi – impahla eluhlaza inani le-euro libalwa kusukela:
Ihabhu lidalula izindleko zokuqalisa ngakunye futhi ngokohlelo, ukuze ukuhlukaniswa kokugcina kwenzeke kusikripthi:
pulled = homelab.pull(run_id) # cost, energy_kwh, avg_w, power curve - cost is in BGN
eur_per_mtok = round((pulled["cost"] * 0.5113) / (tot_out / 1e6), 4)Izinombolo
Kamuva ngabuyela emuva ngengeza amanye amamodeli amahlanu – i-GLM-4.5-Air (106B), i-DeepSeek-R1-Distill (32.8B), i-Seed-OSS (36B), i-Devstral (24B), ne-Qwen3-Coder (30.5B) – ikhishwe kubhentshimakhi ye-coding-ejenti ehlukile, bengizosebenzisa i-GPU efanayo, bese ngisebenzisa i-GPU efanayo. indlela, kodwa imisebenzi ye-ejenti enezinyathelo eziningi zangempela esikhundleni seluphu yesizukulwane samasekhondi angama-240 – kuphawulwe lapho okubalulekile, emihumeni engezansi). Amamodeli ayisishiyagalombili, isikali esisodwa:
Nereferensi yefu engangilinganisa nayo (uhlu lwamanani, Juni 2026): imodeli ye-Flash-class esingethwe – Gemini 2.5 Flash, GPT-4o-mini – isebenza cishe $0.60 amathokheni okukhiphayo angu-1M (~€0.55); i-tier eshibhe kakhulu (Gemini 3.1 Flash-Lite) isiseduze $0.40.
Isimanga
Ezinhlanu kweziyisishiyagalombili zishibhe kunereferensi yamafu. Abathathu abakho—nabathathu abangekho hhayi okuthathu ongakhetha ngokubala kwepharamitha.
I-GLM-4.5-Air iyimodeli enkulu kunazo zonke ohlwini, amapharamitha angu-106B, futhi ibiza u-€1.040/1M — ngokweqiniso ngaphezu kwe-API yamafu, ugesi wodwa, ngaphambi kokuba kubalwe isenti lentengo yokuthenga ye-GPU. Leyo ngxenye ifanelana ne-intuition yasekuqaleni: imodeli enkulu, ihamba kancane, iyabiza.
Kodwa i-DeepSeek-R1-Distill ayiyona ingxenye yesithathu yosayizi we-GLM – amapharamitha angu-32.8B – futhi iyona. iningi imodeli ebizayo kulo lonke uhlu, at €1.526/1M. Idonsa amandla amancane kune-gemma3:27b (155 W vs 283 W) – futhi iseyi- kubiza kakhulu ithokheni ngayinye yanoma iyiphi imodeli ehloliwe .
Inothi lendlela eyodwa elibalulekile lapha, ngaphambili kunokungcwatshwa: amamodeli amathathu e-gemma akalwe ngebhentshimakhi yesizukulwane esiqhubekayo samasekhondi angu-240 engiyakhele lolu cezu. Ezinye ezinhlanu – ezihlanganisa i-DeepSeek-R1-Distill – zivela kubhentshimakhi yangaphambili, ehlukile yemisebenzi ye-ejenti yezinyathelo eziningi zangempela (izingcingo zamathuluzi, ukucabanga okuningiliziwe) ku-GPU efanayo nentela, kusetshenziswa ifomula efanayo yamathokheni e-cost÷output. Lowo mehluko yisona sizathu i-DeepSeek ebukeka ngayo: yayo isivinini isizukulwane eluhlaza lapho ukukhiqiza amathokheni ngenkuthalo kuyinto ehloniphekile yamathokheni angu-6.7/sec. Kodwa isivinini sokukhiqiza akusona esikhokhiswa imitha. Imitha ishaja isikhathi sewashi lasodongeni ekhadini, futhi i-DeepSeek — imodeli ye-reaction-distilled – ichitha isikhathi esiningi ixoxa phakathi kwezizukulwane, ingakhiqizi. Yakho izinga lokulethwa elisebenzayo(amathokheni okukhiphayo ÷ isamba sesikhathi sokusebenza, izingxoxo zifakiwe) kusebenza 3.7 amathokheni/umzuzwana ephansi kunanoma iyiphi imodeli ehloliwe — ngaphansi kakhulu kokusebenza kwe-Seed-OSS okungu-7.1 kanye ne-GLM's 5.0. Yilokho inani elilandelekayo ngempela.
Usayizi awuzange ube umshini. Isivinini esisebenzayo sasiyi – futhi isivinini esisebenzayo akuyona inombolo lawa mamodeli avame ukuphawulwa ngayo.
Indlela
Ithokheni ngalinye libiza ama-watts ÷ ukuphuma.Yiyo yonke leyo equation – iqhinga ukusebenzisa inombolo efanele yokuphuma:
I-gemma3:1b iyashesha (136 tok/s) futhi ilula (154 W) — ishibhile kuzo zombili izimbazo, ishibhile kukonke. I-Devstral idonsa amandla amaningi anoma iyiphi imodeli ehloliwe (320 W) kodwa ihlala imaphakathi netafula ngoba isashesha ngokufanele. Kumamodeli amathathu e-gemma eyakhelwe inhloso, isivinini sesizukulwane esingahluziwe kuyinto inombolo ebeka inani – ama-watts ÷ ama-tok/s aluhlaza akha kabusha isilinganiso sezindleko cishe ngokunembile. Kumamodeli amahlanu ebhentshimakhi, inombolo ebeka intengo ithi ukusebenza okusebenzayo kwewashi odongeni hhayi isivinini sesizukulwane esingavuthiwe abavame ukucashunwa kuso — futhi ngaleyo nombolo, i-Seed-OSS (7.1 tok/s esebenzayo), GLM-4.5-Air (5.0), ne-DeepSeek-R1-Distill (3.7) amamodeli amathathu anensa kakhulu ahlolwe, ngokulandelana ngqo izindleko zawo eziwalinganisa. Akekho kubo odonsa amandla angajwayelekile (141–186 W, empeleni ngaphansi kunamamodeli ashibhile amaningana); ayanensa uma nje amakholi wamathuluzi nezikhala zokucabanga zibaliwe, futhi imodeli ehamba kancane ikhokhela umzuzwana ngamunye ephethe ikhadi, kungakhathaliseki ukuthi ipharamitha ingakanani.
Engingakufuniyo
Lena ingxenye eyigcina ithembekile, ngakho-ke ngizobe ngingagwegwesi mayelana nemiphetho:
· Lona ugesi we-GPU kuphela – ukaliwe, unqenqema, futhi mncane.Ukugijima okugcwele kuzungeza ingxenye yesenti. Angilinganisi i-CPU noma i-DRAM (i-RAPL ibingafundeki kulo msingathi, ngakho-ke lezo zibuyile ziyize), futhi ngikubeka ngokusobala. hhayi ukubala inani lokuthenga le-GPU, ukudonsa kokungenzi lutho, ukupholisa, noma isikhathi sami. Izibalo ze-€/1M yilezi isilinganiso samandla amancane iwusizo kokuthi “iyiphi imodeli eshibhile ukuqhubeka uyikhiqiza,” hhayi izindleko eziphelele zobunikazi.
· Izindleko zangempela zendawo yi-GPU oyithengile, hhayi ama-watts. I-3090 inciphisa kuphela ukusetshenziswa okuphezulu, okungaguquki. Ibhokisi elimatasatasa imizuzu embalwa ngosuku likhokhela kakhulu i-silicon engenzi lutho, futhi akukho sibalo sikagesi esithwebula lokho. I-marginal-energy lens ithembekile mayelana nobubanzi bayo: iphansi, hhayi imali.
· Izindlela ezimbili zokulinganisa (bona “isimangaliso” ngenhla ukuthi kungani kubalulekile). Amamodeli amathathu e-gemma: ibhentshimakhi yesizukulwane esiqhubekayo samasekhondi angama-240. Ezinye ezinhlanu: ibhentshimakhi ye-ejenti yekhodi ehlukile, ifomula efanayo ye-cost÷tokens, kodwa ngempumelelo ukufakwa kwewashi odongeni kunejubane lesizukulwane esingavuthiwe – ngokuphikiswayo okumelela kakhulu ukuthi la mamodeli asetshenziswa kanjani ngempela, hhayi ngaphansi, njengoba imithwalo yemisebenzi yama-ejenti wangempela ihlanganisa izingcingo zamathuluzi nezingxoxo iluphu yesizukulwane esingenasici.
·I-GPU eyodwa, injini eyodwa, izisindo ze-GGUF ezilinganiselwe.Inani lakho lentengo, imodeli, usayizi weqoqo, inani kanye nekhadi kuzohambisa zonke izinombolo lapha. Itafula okwami ibhokisi; indlela iyingxenye edluliswayo.
Ngakho-ke phatha ama-euro aphelele njengomfanekiso kanye umumo njengokutholwayo: izindleko zethokheni ngayinye zilandela isivinini sokulethwa esisebenzayo, futhi lokho akulandeleli ukubala kwepharamitha ngendlela obungayilindela.
I-takeway
Khetha imodeli encane kunazo zonke, eshesha kakhulu esula ibha yekhwalithi yakho – yilapho “indawo” ikhokha khona ngempela, ngomkhawulo obanzi, futhi ayihlangene nokukhetha imodeli encane. ngoba kuncane. I-Qwen3-Coder ku-30.5B ishibhile ngethokheni ngayinye kune-gemma4:26b, futhi i-gemma3:27b kokuthi 27B ihlula yonke imodeli yesitayela sokucabanga kuhlu naphezu kokuthi iyimodeli emincane engenalutho engenawo amaqhinga akhethekile okusebenza kahle. Okuguquguqukayo okubalulekile amathokheni alethwa ngomzuzwana wewashi eliwudonga, isitobhi esigcwele – hhayi inombolo yesivinini sesizukulwane esingavuthiwe imodeli evame ukuphawulwa ngayo – futhi okuwukuphela kwendlela yokuyazi ngemodeli yakho nebhokisi lakho ukuyikala.
Ithuluzi engilisebenzisile i-HomeLab Monitor – umthombo ovulekile, i-MIT, isiqukathi esisodwa – futhi amanani okuqalisa ngenhla yilokho kanye ithebhu yayo Yokuhlola ekwenzayo ngaphandle kwebhokisi:
Engeza umsingathi wakho we-GPU, faka ukhiye, kanye nezintengo zakho zokuma ezilandelayo uqobo – kunoma yiluphi uhlobo lwemali olulungisile, okufanele luhlolwe ngaphambi kokuthi ushicilele inombolo.
Ngakho-ke: uma ufinyelela imodeli yendawo nge-API, ingabe uyayazi inombolo yakho yethokheni ngayinye, ngohlobo lwemali olufanele – noma ingabe, njengoba nganginjalo, uthatha nje ukuthi i-GPU iyenza ibe mahhala?
