Thuthela amaseva okulandelela i-MLflow uye ku-Amazon SageMaker AI nge-MLflow engenaseva

0 1 6 minutes read

Thuthela amaseva okulandelela i-MLflow uye ku-Amazon SageMaker AI nge-MLflow engenaseva

Ukusebenzisa iseva yokulandelela i-MLflow ezilawulwa yona iza nesihloko esiphezulu sokuphatha, okuhlanganisa ukugcinwa kweseva nokukalwa kwensiza. Njengoba amaqembu ekala ukuhlola kwawo kwe-ML, ukuphatha kahle izinsiza ngesikhathi sokusetshenziswa okuphezulu nezikhathi zokungenzi lutho kuyinselele. Izinhlangano ezisebenzisa i-MLflow ku-Amazon EC2 noma ezindlini zingathuthukisa izindleko nezinsiza zobunjiniyela ngokusebenzisa i-Amazon SageMaker AI ene-MLflow engenaseva.

Lokhu okuthunyelwe kukubonisa ukuthi uyithutha kanjani iseva yakho yokulandelela i-MLflow ezilawulayo iye kuhlelo lokusebenza lwe-MLflow – iseva yokulandelela engenasiphakeli ku-SageMaker AI ekala ngokuzenzakalelayo izinsiza ngokusekelwe ekudingeni kuyilapho ikhipha ukupeshiswa kweseva nokuphatha isitoreji mahhala. Funda ukuthi ungalisebenzisa kanjani ithuluzi lokungenisa le-MLflow Export ukuze udlulisele ukuhlolwa kwakho, ukugijima, amamodeli, nezinye izinsiza ze-MLflow, okuhlanganisa imiyalelo yokuqinisekisa impumelelo yakho yokufuduka.

Nakuba lokhu okuthunyelwe kugxile ekufudukeni kusuka eziphakelini zokulandela ngomkhondo ze-MLflow ezizilawulayo kuye ku-SageMaker ene-MLflow, ithuluzi Lokuthekelisa Le-MLflow linikeza usizo olubanzi. Ungasebenzisa indlela efanayo ukuze uthuthe amaseva okulandelela e-MLflow aphethwe yi-SageMaker uye kukhono elisha le-MLflow engenasiphakeli ku-SageMaker. Ithuluzi lisiza futhi ngokuthuthukiswa kwenguqulo nokusungula imizila yokusekelayo yokutakula inhlekelele.

Umhlahlandlela wesinyathelo nesinyathelo: Ukulandelela ukuthuthela kweseva ku-SageMaker nge-MLflow

Umhlahlandlela olandelayo unikeza imiyalelo yesinyathelo ngesinyathelo yokuthutha iseva ekhona yokulandelela i-MLflow iye ku-SageMaker nge-MLflow. Inqubo yokuthutha iqukethe izigaba ezintathu eziyinhloko: ukuthumela ama-artifact akho e-MLflow kusitoreji esimaphakathi, ukulungisa Uhlelo Lokusebenza Lwe-MLflow, nokungenisa ama-artifact akho. Ungakhetha ukwenza inqubo yokuthutha kusukela kusibonelo se-EC2, ikhompuyutha yakho yomuntu siqu, noma incwadi yokubhalela ye-SageMaker. Noma ngabe iyiphi indawo oyikhethayo kufanele igcine ukuxhumeka kukho kokubili iseva yakho yokulandelela umthombo kanye neseva yakho yokulandelela okuqondiwe. I-MLflow Export Import isekela ukuthunyelwa kwamanye amazwe okuvela kuzo zombili iziphakeli zokulandela ezilawulwa ngokwazo kanye namaseva okulandelela i-Amazon SageMaker MLflow (kusuka ku-MLflow v2.16 kuya phambili) kuya ku-Amazon SageMaker Serverless MLflow.

Umfanekiso 1: Inqubo yokufuduka ngethuluzi lokungenisa le-MLflow Export

Okudingekayo

Ukuze uhambisane nalokhu okuthunyelwe, qiniseka ukuthi unezidingo ezilandelayo:

Isinyathelo 1: Qinisekisa ukuhambisana kwenguqulo ye-MLflow

Ngaphambi kokuqala ukuthutha, khumbula ukuthi akuzona zonke izici ze-MLflow ezingasekelwa kunqubo yokuthutha. Ithuluzi Lokuthekelisa Le-MLflow lisekela izinto ezihlukene ngokusekelwe kunguqulo yakho ye-MLflow. Ukuze ulungiselele ukufuduka okuyimpumelelo:

Qinisekisa inguqulo yamanje ye-MLflow yeseva yakho ekhona yokulandelela i-MLflow:
Buyekeza inguqulo yakamuva esekelwayo ye-MLflow kumadokhumenti e-Amazon SageMaker MLflow. Uma usebenzisa inguqulo endala ye-MLflow endaweni ezilawulayo, sincoma ukuthi uthuthukele enguqulweni yakamuva esekelwa i-Amazon SageMaker MLflow ngaphambi kokuqhubeka nokuthuthela:
```
pip install --upgrade mlflow=={supported_version}
```
Ukuze uthole uhlu olusesikhathini samanje lwezinsiza ze-MLflow ezingadluliswa kusetshenziswa i-MLflow Export Import, sicela ubheke amadokhumenti Okuthunyelwa Kwempahla Kwe-MLflow.

Isinyathelo sesi-2: Dala i-MLflow App entsha

Ukuze ulungise indawo okuyo, udinga kuqala udale i-SageMaker Serverless MLflow App entsha.

Ngemva kokusetha i-SageMaker AI (bheka futhi Umhlahlandlela wokusetha nge-Amazon SageMaker AI), ungakwazi ukufinyelela i-Amazon SageMaker Studio futhi esigabeni se-MLflow, udale i-MLflow App entsha (uma ingazange idalwe ngokuzenzakalelayo ngesikhathi sokusetha isizinda sokuqala). Landela imiyalelo echazwe kumadokhumenti e-SageMaker.
Uma uhlelo lwakho lokusebenza oluphethwe lwe-MLflow seludaliwe, kufanele luvele kukhonsoli yakho ye-SageMaker Studio. Khumbula ukuthi inqubo yokudala ingathatha imizuzu emi-5.

Umfanekiso 2: Uhlelo lokusebenza lwe-MLflow ku-SageMaker Studio Console

Kungenjalo, ungayibuka ngokwenza umyalo olandelayo we-AWS Command Line Interface (CLI):

aws sagemaker list-mlflow-tracking-servers

Kopisha Igama Lensiza Ye-Amazon (ARN) leseva yakho yokulandela ngomkhondo kudokhumenti, liyadingeka Esinyathelweni sesi-4.
Khetha Vula i-MLflowokukuholela kudeshibhodi ye-MLflow engenalutho. Ezinyathelweni ezilandelayo, singenisa izivivinyo zethu nama-artifact ahlobene kusuka kuseva yethu yokulandelela i-MLflow ezilawulayo lapha.

Umfanekiso 3: I-MLflow interface yomsebenzisi, ikhasi lokufika

Isinyathelo sesi-3: Faka i-MLflow kanye ne-plugin ye-SageMaker MLflow

Ukuze ulungisele indawo yakho yokusebenzisa ukuze uthuthe, udinga ukusungula ukuxhumana neziphakeli zakho ezikhona ze-MLflow (bona izimfuneko) futhi ufake futhi ulungiselele amaphakheji nama-plugin e-MLflow adingekayo.

Ngaphambi kokuthi uqale ngokufuduka, udinga ukusungula ukuxhumana futhi uqinisekise endaweni esingethe iseva yakho ekhona ezilawulayo yokulandelela i-MLflow (isb, umshini obonakalayo).
Uma usukwazi ukufinyelela iseva yakho yokulandelela, udinga ukufaka i-MLflow kanye ne-plugin ye-SageMaker MLflow endaweni yakho yokusayinda. I-plugin iphethe ukusungulwa kokuxhumana kanye nokuqinisekisa kuhlelo lwakho lokusebenza lwe-MLflow. Yenza umyalo olandelayo (bheka futhi imibhalo):

pip install mlflow sagemaker-mlflow

Isinyathelo sesi-4: Faka ithuluzi lokungenisa le-MLflow Export

Ngaphambi kokuthi uthumele izinsiza zakho ze-MLflow, udinga ukufaka ithuluzi Lokuthekelisa le-MLflow.

Zijwayeze ngethuluzi lokungenisa le-MLflow Export namandla alo ngokuvakashela ikhasi layo le-GitHub. Kulezi zinyathelo ezilandelayo, sisebenzisa amathuluzi ayo ngobuningi (okungukuthi export-all futhi import-all), okukuvumela ukuthi udale ikhophi yeseva yakho yokulandela ngomkhondo nokuhlola kwayo nezinto zobuciko ezihlobene. Le ndlela igcina ubuqotho bereferensi phakathi kwezinto. Uma ufuna ukuthutha izivivinyo ezikhethiwe kuphela noma uguqule igama lokuhlolwa okukhona, ungasebenzisa amathuluzi Awodwa. Sicela ubuyekeze imibhalo ye-MLflow Export Import ukuze uthole ulwazi olwengeziwe ngezinto ezisekelwayo kanye nemikhawulo.
Faka ithuluzi Lokungenisa I-MLflow Export endaweni yakho, ngokwenza umyalo olandelayo:

pip install git+

Isinyathelo sesi-5: Khipha izinsiza ze-MLflow kuhla lwemibhalo

Manje njengoba indawo yakho isilungisiwe, singakwazi ukuqala inqubo yokuthutha yangempela ngokuthumela izinsiza zakho ze-MLflow zisuka endaweni yakho yomthombo.

Ngemva kokufaka ithuluzi Lokungenisa Okuthekelisa I-MLflow, ungakha uhla lwemibhalo oluqondiwe endaweni yakho yokusebenzisa njengendawo okuyiwa kuyo yezinsiza, ozikhipha esinyathelweni esilandelayo.
Hlola izivivinyo zakho ezikhona kanye nezinsiza ezihambisanayo ze-MLflow ofuna ukuzithekelisa. Esibonelweni esilandelayo, sifuna ukuthekelisa izinto ezigcinwe njengamanje (isibonelo, ukuhlola namamodeli abhalisiwe).

Umfanekiso 4: Izivivinyo ezigcinwe ku-MLflow
Qala ukuthutha ngokulungiselela Isihlonzi Sensiza Efanayo (i-URI) seseva yakho yokulandelela njengento eguquguqukayo yemvelo futhi usebenzise ithuluzi elilandelayo lokuthekelisa ngobuningi ngamapharamitha weseva yakho ekhona yokulandelela i-MLflow kanye nohla lwemibhalo oluqondiwe (bona futhi imibhalo):

# Set the tracking URI to your self-managed MLflow server
export MLFLOW_TRACKING_URI=

# Start export
export-all --output-dir mlflow-export

Linda kuze kuqede ukuthunyelwa ukuze uhlole uhla lwemibhalo lokuphumayo (esimweni esandulele: mlflow-export).

Isinyathelo 6: Ngenisa izinsiza ze-MLflow kuhlelo lwakho lokusebenza lwe-MLflow

Ngesikhathi sokungenisa, izibaluli ezichazwe ngumsebenzisi ziyagcinwa, kodwa amathegi akhiqizwa uhlelo (isb, creation_date) azilondolozwanga I-MLflow Export Import. Ukuze ulondoloze izibaluli zesistimu yoqobo, sebenzisa i- --import-source-tags inketho njengoba kukhonjisiwe esibonelweni esilandelayo. Lokhu kubalondoloza njengamathegi nge mlflow_exim isiqalo. Ukuze uthole ulwazi olwengeziwe, bona i-MLflow Export Import – Governance and Lineage. Qaphela imikhawulo eyengeziwe echazwe lapha: Imikhawulo yokungenisa.

Inqubo elandelayo idlulisela izinsiza zakho ze-MLflow ezithunyelwe ku-MLflow App yakho entsha:Qala ukungenisa ngokulungiselela i-URI yohlelo lwakho lokusebenza lwe-MLflow. Ungasebenzisa i-ARN–oyilondoloze kusinyathelo 1–ukwenzela lokhu. I-plugin ye-SageMaker MLflow efakwe ngaphambilini ihumusha ngokuzenzakalelayo i-ARN ku-URI evumelekile futhi idale isicelo esiqinisekisiwe ku-AWS (khumbula ukulungisa izifakazelo zakho ze-AWS njengeziguquguqukayo zemvelo ukuze i-plugin ikwazi ukuzilanda).

# Set the tracking URI to your MLflow App ARN
export MLFLOW_TRACKING_URI=arn:aws:sagemaker:::mlflow-app/app- 

# Start import
import-all --input-dir mlflow-export

Isinyathelo sesi-7: Qinisekisa imiphumela yakho yokuthutha

Ukuze uqinisekise ukuthi ukuthutha kwakho kube yimpumelelo, qinisekisa ukuthi izinsiza zakho ze-MLflow zidluliselwe ngendlela efanele:

Uma isikripthi sokungenisa kwakho konke sesithuthele ukuhlolwa kwakho, ukugijima, nezinye izinto kuseva entsha yokulandela ngomkhondo, ungaqala ukuqinisekisa impumelelo yokuthutha, ngokuvula ideshibhodi yohlelo lwakho lokusebenza lwe-MLflow olungenasiphakeli (oluvule esinyathelweni sesi-2) futhi uqinisekise ukuthi:
- Izinsiza ze-MLflow ezithunyelwe zikhona namagama azo angempela kanye nemethadatha
- Imilando yokuqalisa iphelele ngamamethrikhi namapharamitha
- Ama-artifact angamamodeli ayafinyeleleka futhi ayalayisheka
- Omaka namanothi agciniwe
  
  Umfanekiso 5: I-MLflow interface yomsebenzisi, ikhasi lokufika ngemva kokufuduka
Ungaqinisekisa ukufinyelela okuhleliwe ngokuqala incwajana entsha ye-SageMaker nokusebenzisa ikhodi elandelayo:

import mlflow

# Set the tracking URI to your MLflow App ARN 
mlflow.set_tracking_uri('arn:aws:sagemaker:::mlflow-app/app-')

# List all experiments
experiments = mlflow.search_experiments()
for exp in experiments:
    print(f"Experiment Name: {exp.name}")
    # Get all runs for this experiment
    runs = mlflow.search_runs(exp.experiment_id)
    print(f"Number of runs: {len(runs)}")

Ukucatshangelwa

Lapho uhlela ukufuduka kwakho kwe-MLflow, qinisekisa indawo yakho yokusebenzisa (ukuthi i-EC2, umshini wendawo, noma amabhukumaka e-SageMaker) inesitoreji esanele nezinsiza zekhompuyutha zokusingatha umthamo wedatha yeseva yakho yokulandelela umthombo. Nakuba ukuthutha kungasebenza ezindaweni ezihlukahlukene, ukusebenza kungase kuhluke ngokusekelwe ekuxhumekeni kwenethiwekhi nezisetshenziswa ezitholakalayo. Ngokufuduka kwenani elikhulu, cabanga ukuhlukanisa inqubo ibe amaqoqo amancane (isibonelo, ukuhlolwa ngakunye).

Hlanza

Iseva yokulandelela i-MLflow ephethwe yi-SageMaker izongenisa izindleko uze uyisuse noma uyimise. Ukukhokhiswa kwamaseva okulandelela kusekelwe esikhathini iziphakeli ebezisebenza ngaso, usayizi okhethiwe, kanye nenani ledatha efakwe eziphakelini zokulandelela. Ungakwazi ukumisa amaseva ukulandelela lapho engasetshenziswa ukulondoloza izindleko, noma ungawasusa usebenzisa i-API noma i-SageMaker Studio UI. Ukuze uthole imininingwane eyengeziwe ngamanani, bheka amanani entengo e-Amazon SageMaker.

Isiphetho

Kulokhu okuthunyelwe, sibonise ukuthi singathutha kanjani iseva yokulandelela i-MLflow ezilawulayo iye ku-SageMaker ene-MLflow kusetshenziswa ithuluzi lomthombo ovulekile lokungenisa i-MLflow Export. Ukuthuthela kuhlelo lokusebenza lwe-MLflow olungenasiphakeli ku-Amazon SageMaker AI kunciphisa i-overhead yokusebenza ehlotshaniswa nokugcina ingqalasizinda ye-MLflow kuyilapho ihlinzeka ngokuhlanganisa okungenazihibe ne-AI/ML ebanzi esebenza ku-SageMaker AI.

Ukuze uqalise ngokufuduka kwakho, landela umhlahlandlela wesinyathelo ngesinyathelo futhi ubheke imibhalo ebaluliwe ukuze uthole imininingwane eyengeziwe. Ungathola amasampula ekhodi nezibonelo endaweni yethu ye-AWS Samples GitHub. Ukuze uthole ulwazi olwengeziwe mayelana namakhono e-Amazon SageMaker AI nezinye izici ze-MLOps, vakashela imibhalo ye-Amazon SageMaker AI.

Mayelana nababhali

Rahul Easwar Ungumphathi Womkhiqizo Omkhulu kwa-AWS, ohola i-MLflow ephethwe kanye Nezinhlelo Zokusebenza Zozakwethu ze-AI ngaphakathi kwethimba le-SageMaker AIOps. Ngesipiliyoni seminyaka engaphezu kwengu-20 kusukela ekuqaleni kuya kubuchwepheshe bebhizinisi, usebenzisa isizinda sakhe sezamabhizinisi kanye ne-MBA yaseChicago Booth ukuze akhe amapulatifomu e-ML alula enza kube lula ukwamukelwa kwe-AI ezinhlanganweni emhlabeni wonke. Xhuma no-Rahul ku-LinkedIn ukuze ufunde kabanzi ngomsebenzi wakhe kumapulatifomu e-ML kanye nezixazululo ze-AI zebhizinisi.

U-Roland Odorfer i-Solutions Architect kwa-AWS, ezinze eBerlin, eJalimane. Usebenza nemboni yaseJalimane namakhasimende akhiqizayo, ebasiza ukuba bakhe izixazululo ezivikelekile nezingalawuleki. U-Roland unentshisekelo ezinhlelweni ezisabalalisiwe nokuphepha. Uyakujabulela ukusiza amakhasimende ukuthi asebenzise ifu ukuze axazulule izinselele eziyinkimbinkimbi.

Anurag Gajam ungunjiniyela Wokuthuthukiswa Kwesoftware neqembu le-Amazon SageMaker MLflow kwa-AWS. Izithakazelo zakhe zobuchwepheshe zihlanganisa ingqalasizinda ye-AI/ML nezinhlelo ezisabalalisiwe, lapho engumnikeli owaziwayo we-MLflow othuthukise ithuluzi le-mlflow-export-import ngokungeza ukusekelwa kwezinto ezengeziwe ze-MLflow ukuze anike amandla ukufuduka okungenamthungo phakathi kwezinsizakalo ze-SageMaker MLflow. Usebenza ngokukhethekile ekuxazululeni izinkinga eziyinkimbinkimbi nokwakha isofthiwe ethembekile enika amandla imithwalo yemisebenzi ye-AI ngezinga. Ngesikhathi sakhe sokuphumula, uthanda ukudlala i-badminton nokuhamba ngezinyawo.