Ukukala Isegimenti nge-Blender: Uyenza Kanjani Indalo Yedathasethi | nguVincent Vandenbussche | Jan, 2025

Umhlahlandlela wesinyathelo ngesinyathelo sokukhiqiza idatha yokwenziwa yokuqeqesha amamodeli e-AI

Uma uke waqeqesha imodeli yokuhlukanisa iphrojekthi entsha, cishe uyazi ukuthi ayimayelana nemodeli. Imayelana nedatha.

Ukuqoqa izithombe ngokuvamile kuqondile; ngokuvamile ungathola okuningi kumapulatifomu afana ne-Unsplash, noma usebenzise amathuluzi e-Generative AI afana ne-Stable Diffusion ukuze ukhiqize okwengeziwe:

Inselele enkulu ngokuvamile isekulebulani. Ukuchasisa izithombe zokuhlukaniswa kudla isikhathi kakhulu. Ngisho nangamathuluzi athuthukile njenge-SAM2 nge-Meta, ukudala idathasethi enezichasiselo ezigcwele, eqinile, nehlukahlukene kusadinga isikhathi esikhulu.

Kulesi sihloko, sizohlola enye, inketho evame ukuhlolisiswa kancane: sisebenzisa amathuluzi e-3D, njenge-Blender. Ngempela, izinjini ze-3D ziya ngokuya ziba namandla futhi zingokoqobo. Ngaphezu kwalokho, banikeza inzuzo ephoqelelayo: ikhono lokukhiqiza amalebula ngokuzenzakalelayo ngenkathi udala idathasethi, ususa isidingo sezichasiselo ezenziwa ngesandla.

Kulesi sihloko, sizobeka isixazululo esiphelele sokudala imodeli yokuhlukanisa isandla, ehlukaniswe izingxenye ezibalulekile ezilandelayo:

• Ukukhiqiza izandla nge-Blender nokuthi ungakuthola kanjani ukuhlukahluka kokuma kwesandla, indawo kanye namathoni esikhumba
• Ukukhiqiza idathasethi kusetshenziswa izithombe ze-Blender ezikhiqiziwe nemifanekiso engemuva ekhethiwe, kusetshenziswa i-OpenCV
• Ukuqeqesha nokuhlola imodeli nge-PyTorch

Vele, yonke ikhodi esetshenziswe kulokhu okuthunyelwe iyatholakala ngokuphelele futhi ingasetshenziswa kabusha, kule ndawo yokugcina ye-GitHub.

Ukukhiqiza izithombe zezandla, masisebenzise i-Blender. Angiyena uchwepheshe ngalolu hlobo lwethuluzi, kodwa linikeza izici eziwusizo kakhulu ngenjongo yethu:

• Kumahhala — ayikho ilayisensi yokuhweba, noma ubani angayilanda futhi ayisebenzise ngokushesha
• Kunomphakathi omkhulu futhi amamodeli amaningi angatholakala ku-inthanethi, amanye amahhala, amanye awakho
• Ekugcineni, ifaka i-Python API, evumela ukuzenzakalelayo kokukhiqizwa kwesithombe ngezici ezahlukahlukene

Njengoba sizobona, lezo zici ziwusizo kakhulu, futhi zizosivumela ukuthi senze idatha yokwenziwa kalula. Ukuqinisekisa ukuhlukahluka okwanele, sizohlola ukuthi singenza kanjani ngokungahleliwe amapharamitha alandelayo ezandleni zethu ezikhiqiziwe:

• Ukuma kweminwe: sifuna ukuba nezithombe zezandla ezindaweni eziningi ezahlukene
• Indawo yekhamera: sifuna ukuba nezithombe zezandla emibonweni eyahlukene
• Ithoni yesikhumba: sifuna ukuhlukahluka kwethoni yesikhumba, ukwenza imodeli iqine ngokwanele

Qaphela: Indlela ehlongozwayo lapha ayinakho ukuchema okungaba khona okusekelwe esikhumbeni futhi ayisho ukuthi ayinakuchema. Noma yimuphi umkhiqizo osuselwe kule ndlela kufanele uhlolwe ngokucophelela ngokumelene nanoma yikuphi ukuchema kwezimiso zokuziphatha.

Ngaphambi kokungena kulezi zinyathelo, sidinga imodeli yesandla ye-3D. Kunamamodeli amaningi kumawebhusayithi afana ne-Turbosquid, kodwa ngisebenzise imodeli yesandla etholakala mahhala umuntu angayithola lapha. Uma uvula leli fayela nge-Blender, uzothola okuthile okufana nesithombe-skrini esilandelayo.

Njengoba kubonisiwe, imodeli ayihlanganisi nje kuphela ukwakheka kwesandla nokuthungwa kodwa futhi nesakhiwo samathambo, esivumela ukulingiswa kokunyakaza kwesandla. Masisebenze kusukela kulokho ukuze sithole isethi yezandla ehlukahlukene ngokudlala ngokuma kweminwe, amathoni esikhumba kanye nokuma kwekhamera.

Ukushintsha Izikhundla Zeminwe

Isinyathelo sokuqala siwukuqinisekisa isethi ehlukahlukene kodwa engokoqobo yezindawo zeminwe. Ngaphandle kokuhlolisisa imininingwane eminingi (njengoba lokhu kuhlobana kakhulu ne-Blender ngokwayo), sidinga ukudala izilawuli zokunyakaza futhi sibeke imigoqo ekunyakazeni okuvumelekile. Ngokuyisisekelo, asifuni iminwe isonge emuva noma igobe ibheke eziqondisweni ezingezona ezangempela. Ukuze uthole imininingwane eyengeziwe ngalezi zinyathelo, bheka lesi sifundo se-YouTube, esingisizile ukuba ngizisebenzise ngomzamo omncane.

Lapho ifayela le-Blender lisethwe kahle ngezingqinamba ezifanele, singasebenzisa umbhalo wePython ukuze senze ngokuzenzakalelayo noma iyiphi indawo yomunwe:

Njengoba sibona, esikwenzayo nje ukubuyekeza izindawo zezilawuli ngokungahleliwe, okuvumela ukuhambahamba kweminwe ngaphansi kwemikhawulo. Ngesethi efanele yemikhawulo, sithola izindawo zeminwe ezibukeka kanjena:

Lokhu kukhiqiza ukuma kweminwe okungokoqobo nokuhlukile, ekugcineni kuvumela ukukhiqizwa kwesethi ehlukahlukene yezithombe zezandla. Manje, asidlale ngethoni yesikhumba.

Ukushintsha Ithoni Yesikhumba

Uma udala idathasethi yesithombe esisha enabantu, enye yezici eziyinselele kakhulu kungaba ukuzuza ukumelwa okwanele kwamathoni esikhumba. Ukuqinisekisa ukuthi amamodeli asebenza kahle kuwo wonke amathoni esikhumba ngaphandle kokuchema kuyinto ebaluleke kakhulu. Nakuba ngingasho ukuthi ngilungisa noma yikuphi ukuchema, indlela engiyiphakamisayo lapha ivumela ukuba nesixazululo sokusebenza ngokushintsha ngokuzenzakalelayo ithoni yesikhumba.

QAPHELA: Le ndlela ayisho ukuthi yenza amamodeli angabi nakuchema. Noma iyiphi imodeli yokukhiqiza kufanele ihlolwe ngokucophelela ngokuhlaziywa kokulunga. Umuntu angabheka okwenziwe yi-Google kumamodeli abo okubona ubuso njengesibonelo.

Engikwenzayo lapha ukubala okuhlanzekile kokucubungula isithombe esithombeni. Umbono ulula: uma kubhekwa umbala oqondiwe kanye nombala omaphakathi wesandla esinikeziwe, ngizovele ngibale umehluko phakathi kwaleyo mibala emibili. Ngizobe sengisebenzisa lo mehluko esandleni esinikeziwe ukuze ngithole ithoni yesikhumba esisha:

Ngenxa yalokho, inikeza izithombe ezilandelayo zezandla:

Nakuba imiphumela ingaphelele, ikhiqiza izithombe ezinengqondo ezinengqondo ezinamathoni esikhumba ahlukahlukene, isebenzisa ukucubungula izithombe okuqondile. Kusele isinyathelo esisodwa kuphela sokuba nesethi eyanele yezithombe: indawo yokubuka.

Ukulungisa Indawo Yekhamera

Okokugcina, masilungise ukuma kwekhamera ukuze sithwebule izandla ngemibono eminingi. Ukuze kuzuzwe lokhu, ikhamera itholakala endaweni engahleliwe kuyindilinga emaphakathi nesandla. Lokhu kungafinyelelwa kalula nje ngokudlala ngama-engeli amabili wezixhumanisi eziyindilinga. Kule khodi elandelayo ngikhiqiza indawo engahleliwe endaweni:

Bese, ngisebenzisa lokhu futhi ngengeza izithiyo ezimbalwa endaweni eyindilinga, ngingakwazi ukubuyekeza indawo yekhamera esandleni nge-Blender:

Ngenxa yalokho, manje sithola isampula elandelayo yezithombe:

Manje sinezandla ezineminwe ehlukahlukene, ithoni yesikhumba kanye nemibono ehlukahlukene. Ngaphambi kokuqeqesha imodeli yokuhlukanisa, isinyathelo esilandelayo ukukhiqiza izithombe zezandla ezingemuva nezimo ezihlukahlukene.

Ukuze sikhiqize izithombe ezihlukene nezingokoqobo ezanele, sizohlanganisa izandla zethu ezikhiqiziwe nesethi yezithombe zangemuva ezikhethiwe.

Ngithathe izithombe ku-Unsplash, ngaphandle kwamalungelo njengezithombe ezingemuva. Ngiqinisekise ukuthi lezi zithombe aziqukethe izandla. Ngizobe sengingeza ngokungahleliwe izandla ezikhiqizwe yi-Blender kulezi zithombe ezingemuva:

Lo msebenzi, nakuba umude, wenza izenzo ezilula:

• Layisha isithombe sesandla esingahleliwe kanye nemaski
• Layisha isithombe sangemuva esingahleliwe
• Shintsha usayizi wesithombe sangemuva
• Khetha indawo engahleliwe esithombeni esingemuva ukuze ubeke isandla
• Bala imaski entsha
• Bala isithombe esihlanganisiwe sengemuva nesandla

Ngenxa yalokho, kulula ukukhiqiza amakhulukhulu noma izinkulungwane zezithombe ezinamalebula azo ngomsebenzi wokuhlukanisa. Ngezansi isampula yezithombe ezikhiqiziwe:

Ngalezi zithombe namamaski akhiqiziwe, manje singadlulela esinyathelweni esilandelayo: ukuqeqesha imodeli yokuhlukanisa.

Manje njengoba sesikhiqize idatha ngendlela efanele, ake siqeqeshe imodeli yokuhlukanisa kuyo. Ake siqale sikhulume ngepayipi lokuqeqesha, bese sihlola izinzuzo zokusebenzisa le datha ekhiqiziwe.

Ukuqeqesha Imodeli

Sizosebenzisa i-PyTorch ukuqeqesha imodeli, kanye nelabhulali ye-Segmentation Models Pytorch, evumela ukuqeqesha kalula amamodeli amaningi okuhlukanisa.

Amazwibela wekhodi alandelayo avumela ukuqeqeshwa kwemodeli:

Le khodi yenza izinyathelo ezijwayelekile zokuqeqeshwa okuyimodeli:

• Qinisekisa isitimela namasethi edatha avumelekile, kanye nezilayishi zedatha
• Faka imodeli ngokwayo
• Chaza ukulahlekelwa nesilungiseleli
• Qeqesha imodeli futhi uyigcine

Imodeli ngokwayo ithatha ama-agumenti okokufaka ambalwa:

• Isifaki khodi, sokukhetha kulolu hlu lwamamodeli asetshenzisiwe, njenge-MobileNetV3 engiyisebenzisa lapha.
• Izisindo zokuqalisa kudathasethi ye-ImageNet
• Inani lamashaneli okokufaka, lapha 3 kusuka ku-RGB njengoba sisebenzisa izithombe ezinemibala
• Inani lamashaneli okukhiphayo, lapha 1 njengoba kunesigaba esisodwa kuphela
• Umsebenzi wokwenza kusebenze okukhiphayo: i-sigmoid lapha, futhi njengoba kunesigaba esisodwa kuphela

Ukuqaliswa okugcwele kuyatholakala ku-GitHub uma ufuna ukwazi okwengeziwe.

Ukuhlola Imodeli

Ukuze uhlole imodeli, kanye nentuthuko evela ezithombeni ezihlanganisiwe, masenze lokhu kuqhathanisa okulandelayo:

• Qeqesha futhi uhlole imodeli kudathasethi ye-Ego Hands
• Qeqesha futhi uhlole imodeli efanayo kudathasethi ye-Ego Hands, nedatha yethu ekhiqiziwe ehlanganisiwe yengezwe kusethi yesitimela.

Kuzo zombili izimo, ngizohlola imodeli kusethi engaphansi efanayo yedathasethi ye-Ego Hands. Njengemethrikhi yokuhlola, ngizosebenzisa i-Intersection over Union (IoU) (ebuye ibizwe ngokuthi I-Jaccard Index). Ngezansi imiphumela:

• Kusethi yedatha ye-Ego Hands iyodwa, ngemva kwezinkathi ezingu-20: IoU = 0.72
• Kudathasethi ye-Ego Hands + izithombe ezikhiqizwe yi-Blender, ngemva kwezinkathi ezingu-20: IoU = 0.76

Njengoba sibona, singathola ukuthuthuka okuphawulekayo, ukusuka ku-0.72 kuya ku-0.76 ku-IoU, ngenxa yedathasethi eyenziwe ngezithombe ezikhiqizwe i-Blender.

Ukuhlola Imodeli

Kunoma ngubani ozimisele ukuzama le modeli kukhompyutha yakhe, ngingeze neskripthi ku-GitHub, ukuze isebenze ngesikhathi sangempela kokuphakelayo kwekhamera yewebhu.

Njengoba ngiqeqeshe imodeli encane uma kuqhathaniswa (i-MobileNetV3 Large 100), amakhompyutha aphathekayo amaningi esimanje kufanele akwazi ukusebenzisa le khodi ngempumelelo.

Masisonge lesi sihloko ngezinto ezimbalwa ezibalulekile esingazithatha:

• I-Blender iyithuluzi elihle elikuvumela ukuthi ukhiqize izithombe ezingokoqobo ngaphansi kwezimo ezihlukene: ukukhanya, indawo yekhamera, ukuguqulwa, njll.
• I-Leveraging Blender ukukhiqiza idatha yokwenziwa ingase idinge isikhathi, kodwa ingenziwa ngokuzenzakalelayo kusetshenziswa i-Python API.
• Ukusebenzisa idatha ekhiqiziwe kuthuthukise ukusebenza kwemodeli yomsebenzi wokuhlukanisa i-semantic: kuthuthukise i-IoU isuka ku-0.72 yafika ku-0.76
• Ukuze uthole idathasethi ehluke kakhulu, kungenzeka ukwenza lokho ngamamodeli esandla e-Blender amaningi: umumo wezandla owengeziwe, ukwakheka okwengeziwe kungasiza imodeli yokuhlukanisa ukuthi ifane nakakhulu.

Okokugcina, uma ukwazi ukuba nemodeli esebenzayo futhi ungathanda ukuthola isu elingcono kakhulu lokuyisebenzisa, ungabheka lo mhlahlandlela:

Njengenothi eseceleni, ngenkathi lesi sihloko sigxile ekuhlukaniseni kwe-semantic, le ndlela ivumelana neminye imisebenzi yombono wekhompyutha, okuhlanganisa ukuhlukanisa ngokwesibonelo, ukuhlukanisa, nokubikezela okuyingqopha-mlando. Ngingathanda ukuzwa okunye ukusetshenziswa okungenzeka kwe-Blender okungenzeka ngiphuthelwe.

Nazi ezinye izinkomba, noma sezivele zishiwo ngaphakathi kwe-athikili: