Idatha yokususa idatha isusa amaphutha, igcwalisa imininingwane engekho, futhi imivikelo idatha yokusiza ama-algorithms athole amaphethini angempela esikhundleni sokudideka noma imisindo noma ukungahambelani.
Noma yimuphi i-algorithm idinga idatha ehlelwe kahle ehlelwe ngamafomethi ahlelekile ngaphambi kokufunda kusuka kwidatha. Inqubo yokufunda yomshini idinga idatha eyenziwe ngaphambi kwesikhathi njengesinyathelo sayo esiyisisekelo sokuqinisekisa amamodeli agcina ukunemba nokusebenza kwawo ngenkathi eqinisekisa ukuncipha.
Ikhwalithi yomsebenzi wokufaka phambili iguqula ukuqoqwa kwedatha eyisisekelo ibe ukuqonda okubalulekile eceleni kwemiphumela ethembekile yazo zonke izinhlelo zokufunda ngomshini. Le ndatshana ikuhambela ngezinyathelo ezisemqoka zedatha yokufunda komshini, kusuka ekuhlanzeni nasekuguquleni idatha kumathuluzi angokoqobo emhlabeni, izinselelo namathiphu wokukhulisa ukusebenza kwemodeli.
Ukuqonda idatha eluhlaza
Idatha eluhlaza iphuzu lokuqalisa kwanoma iyiphi iphrojekthi yokufunda ngomshini, futhi ulwazi lwemvelo yalo lubalulekile.
Inqubo yokubhekana nedatha eluhlaza ingalinganiswa kwesinye isikhathi. Imvamisa iza nomsindo, okufakiwe okungafanele noma okudukisayo okungaphazamisa imiphumela.
Amanani alahlekile angenye inkinga, ikakhulukazi lapho izinzwa zehluleka noma okokufaka kweqa. Amafomethi angahambisani nawo avela kaningi: Izinkambu zosuku zingasebenzisa izitayela ezahlukahlukene, noma idatha yesigaba ingafakwa ngezindlela ezahlukahlukene (isib., “Yebo,” “”).
Ukubona futhi ukubhekana nalezi zinkinga kubalulekile ngaphambi kokondla idatha kunoma iyiphi i-algorithm yokufunda yomshini. Ukufakwa okuhlanzekile kuholela ekuphumeni kobuhlakani.
Idatha yokufaka phambili ekufundeni komshini wokumama kwedatha vs
Ngenkathi zombili lezi zimayini kanye nokufunda komshini zincike ekusetshenzisweni kokulungiswa kokulungiselela idatha ukuze ihlaziywe, izinhloso nezinqubo zazo ziyehluka.
Kumayini yedatha, ukuvela kwangaphambili kugxile ekwenzeni ama-datasets amakhulu, angahleliwe asebenziseka ukuthola iphethini nokufingqa. Lokhu kufaka ukuhlanza, ukuhlanganiswa, kanye noguquko, kanye nemininingwane yokufometha yombuzo, ukuhlanganisa, noma ukubumbana kwezimayini, imisebenzi engadingi njalo ukuqeqeshwa okuyimodeli.
Ngokungafani nokufunda komshini, lapho kuvame ukuba khona kwangaphambili izikhungo zokwenza ngcono ukunemba kwemodeli nokunciphisa ukweqisa, idatha yezimayini yedatha ihlose ukutolizwa kanye nokuqonda okuchazayo. Isici sobunjiniyela sincane ngokubikezela nokuthola amathrendi anengqondo.
Ngokwengeziwe, ukugeleza kokusebenza kwezimayini zezimayini kungafaka phakathi ukuhlukaniswa kanye ne-binung njalo, ikakhulukazi ngokuhlukanisa okuguqukayo okuqhubekayo. Ngenkathi i-ML PreProcessing ingama uma idatha yokuqeqeshwa isilungiselelwe, i-Data Mining ingahle ibuyele ekuhlolweni kwayo.
Ngakho-ke, izinhloso zokuqalisa izinto: Ukukhishwa kwe-Insight Version Ukusebenza okuqagelayo, setha ithoni yokuthi idatha yakhiwe kanjani ensimini ngayinye. Ngokungafani nokufunda komshini, lapho kuvame ukuba khona kwangaphambili izikhungo zokwenza ngcono ukunemba kwemodeli nokunciphisa ukweqisa, idatha yezimayini yedatha ihlose ukutolizwa kanye nokuqonda okuchazayo.
Isici sobunjiniyela sincane ngokubikezela nokuthola amathrendi anengqondo.
Ngokwengeziwe, ukugeleza kokusebenza kwezimayini zezimayini kungafaka phakathi ukuhlukaniswa kanye ne-binung njalo, ikakhulukazi ngokuhlukanisa okuguqukayo okuqhubekayo. Ngenkathi i-ML PreProcessing ingama uma idatha yokuqeqeshwa isilungiselelwe, i-Data Mining ingahle ibuyele ekuhlolweni kwayo.
Izinyathelo Eziyisisekelo ku-DATE PreaPocesing
1. Ukuhlanza idatha
Idatha yangempela yomhlaba ivame ukuza namanani alahlekile, izikhala ku-spreadsheet yakho edinga ukugcwaliswa noma isuswe ngokucophelela.
Lapho-ke kukhona okuphindwe kabili, okungangenisa ngokungafanele imiphumela yakho. Futhi ungakhohlwa izindinganiso ezingekho emthethweni ezingadonsela imodeli yakho endaweni engafanele uma kushiywe kungalawulwa.
Lokhu kungaphonsa imodeli yakho, ngakho-ke ungadinga ukudonsa, ukuguqula, noma ubakhiphe.
2. Inguquko yedatha
Lapho idatha ihlanzwe, udinga ukufometha. Uma amanani akho ehluke kakhulu ebangeni, ukujwayelekile noma okujwayelekile kusiza ukuzikhulisa ngokungaguquki.
Idatha yedatha- njengegama lezwe noma izinhlobo zomkhiqizo- zidinga ukuguqulwa zibe yizinombolo ngokufaka ikhodi.
Futhi ukuthola eminye imininingwane, kuyasiza ukuhlanganisa amanani afanayo emgqonyeni wokunciphisa umsindo nokugqamisa amaphethini.
3. Ukuhlanganiswa kwedatha
Imvamisa, idatha yakho izoqhamuka kumafayili ehlukene, imininingwane, noma amathuluzi eku-inthanethi. Ukuhlanganisa konke kungakhohlisa, ikakhulukazi uma ucezu lolwazi olufanayo lubukeka luhlukile emthonjeni ngamunye.
I-Schema iyaxabana, lapho ikholomu efanayo inamagama noma amafomethi ahlukile, ajwayelekile futhi adinga ukulungiswa ngokucophelela.
4. Ukuncishiswa kwedatha
Idatha enkulu ingakwazi amamodeli we-Over'l'Lhelm futhi yandise isikhathi sokucubungula. Ngokukhetha kuphela izici eziwusizo kakhulu noma ukunciphisa ubukhulu besebenzisa amasu anjenge-PCA noma isampuli yenza imodeli yakho isheshe futhi inembe kakhulu.
Amathuluzi nemitapo yolwazi yokufaka phambili
- I-Skikit-Funda ilungele imisebenzi eyisisekelo ebalulekile. Inemisebenzi eyakhelwe ngaphakathi ukugcwalisa amanani alahlekile, izici zesilinganiso, izigaba ze-encode, bese ukhetha izici ezibalulekile. Kuwumtapo wezincwadi oqinile, oqalayo onobungane nakho konke okudingayo ukuze uqale.
- I-Pandas ingenye umtapo wezincwadi obalulekile. Kuyasiza kakhulu ekuhlolweni nasekukhohliseni idatha.
- Ukuqinisekiswa kwedatha ye-tensorflow kungasiza uma usebenza ngamaphrojekthi amakhulu. Ihlola izingqinamba zedatha futhi iqinisekisa ukuthi okokufaka kwakho kulandela ukwakheka okulungile, okuthile okulula ukungakunaki.
- I-DVC (Ukulawulwa Kwedatha Yedatha) kukhulu lapho iphrojekthi yakho ikhula. Igcina ithrekhi yezinhlobo ezahlukene zedatha yakho nezinyathelo zokuqalisa ukuze ungalahli umsebenzi wakho noma udidekile izinto phezulu ngesikhathi sokubambisana.
Izinselelo Ezijwayelekile
Enye yezinselelo ezinkulu namuhla ukuphatha idatha ephezulu. Lapho unezigidi zemigqa emithonjeni ehlukene nsuku zonke, ukuhlela nokuzihlanza konke kuba umsebenzi obalulekile.
Ukubhekana nalezi zinselelo kudinga amathuluzi amahle, ukuhlela okuqinile, nokuqapha njalo.
Olunye udaba olubalulekile lushintsha amapayipi ahleliwe. Ngomqondo, kuzwakala kukuhle; Vele usethe ukugeleza ukuze uhlanze futhi ulungiselele idatha yakho ngokuzenzakalelayo.
Kepha empeleni, ama-datasets ayahlukahluka, nemithetho esebenza umuntu angahle ahlukane komunye. Usadinga iso lomuntu ukuze uhlole amacala onqenqemeni bese wenza izingcingo zokwahlulela. I-automation iyasiza, kepha akuhlali i-plug-and-play.
Noma uqala ngedatha ehlanzekile, izinto ziyashintsha, amafomethi shift, ukubuyekeza imithombo, kanye namaphutha angena ngaphakathi
Imikhuba emihle kakhulu
Nazi izindlela ezimbalwa ezingcono kakhulu ezingenza umehluko omkhulu empumelelweni yemodeli yakho. Ake siphule phansi bahlole ukuthi badlala kanjani ezimweni zezwe langempela.
1. Qala ngokuhlukaniswa kwedatha efanele
Iphutha labaqalayo abadala lenza ukwenza konke okuhle ku-dataset ephelele ngaphambi kokuhlukanisa kube ukuqeqeshwa kanye namasethi wokuhlola. Kepha le ndlela ingakwazi ukwethula ngengozi ukubandlulula.
Isibonelo, uma ukala noma ulinganise yonke imininingwane ngaphambi kokuhlukaniswa, imininingwane esethiwe yokuhlola ingahle iqhume inqubo yokuqeqeshwa, okubizwa ngokuthi ukuvuthwa kwedatha.
Njalo hlukanisa idatha yakho kuqala, bese ufaka isicelo sokulungiselela kuphela kusethi yokuqeqeshwa. Kamuva, guqula isivivinyo setha ukusebenzisa amapharamitha afanayo (njengokuphambuka okujwayelekile nokuphambuka okujwayelekile). Lokhu kugcina izinto zilunge futhi ziqinisekise ukuhlolwa kwakho kuthembekile.
2. Ukugwema ukuvuza kwedatha
Ukuvuza kwedatha kuyinto ethambile futhi enye yezindlela ezisheshayo zokubhubhisa imodeli yokufunda yomshini. Kwenzeka lapho imodeli ifunda okuthile bekungeke kube nokufinyelela esimweni sangempela somhlaba wonke.
Izimbangela ezijwayelekile zifaka kusetshenziswa amalebula okuhlosiwe ku-Engineering ye-Accice noma ukuvumela idatha yesikhathi esizayo ithonye izibikezelo zamanje. Isihluthulelo ngaso sonke isikhathi ukucabanga ukuthi yiluphi ulwazi imodeli yakho ozoba nayo ngokweqiniso ngesikhathi sokubikezela futhi uwugcine ukhawulelwe kulokho.
3. Landelela zonke izinyathelo
Njengoba udlula ngepayipi lakho lokuphamba, ukuphatha amanani alahlekile, ukuhlanganisa okungekho emthethweni, izici zokulinganisa, kanye nokugcina umkhondo wezenzo zakho akubalulekile hhayi ngememori yakho kodwa futhi ukuze futhi kube kabusha.
Ukubhala zonke izinyathelo kuqinisekisa abanye (noma ikusasa wena) bangabuyisa indlela yakho. Amathuluzi afana ne-DVC (Ukulawulwa kwe-Data Version) noma i-Jupyter Notebook elula enezichasiselo ezicacile kungenza lokhu kube lula. Lolu hlobo lokulandela ngomkhondo lusiza futhi lapho imodeli yakho isebenza ngokungazelelwe – ungabuya futhi uthole ukuthi yini engahambanga kahle.
Izibonelo Zangempela Zomhlaba
Ukuze ubone ukuthi kungakanani komehluko owenza umehluko owenzayo, cabanga ngocwaningo lwecala olubandakanya ukubikezela kwamakhasimende enkampanini ye-telecom. Ekuqaleni, i-dataset yabo eluhlaza ifaka amanani angekho, amafomethi angahambelani, nezici ezingezona. Imodeli yokuqala eqeqeshelwe kule mininingwane engcolile efinyelele ukunemba okungu-65%.
Ngemuva kokusebenzisa ukwenziwa kwangaphambi kokuhlelwa kwangaphambi kokushoda, ukugcizelela amanani angekho, ukuhlanganisa okuhlukanisayo, okujwayelekile izici, nokususa amakholomu angafanele, kudutshulwa ukunemba okungaphezulu kwama-80%. Ushintsho lwalungekho ku-algorithm kodwa ngekhwalithi yedatha.
Esinye isibonelo esihle sivela ekunakekelweni kwezempilo. Iqembu elisebenza ngokubikezela isifo senhliziyo
usebenzise idatha yomphakathi ehlanganisa izinhlobo zedatha ezixubile kanye nezinkundla ezingekho.
Basebenzise i-binning emaqenjini eminyaka, abakwaHlelo benza iziqubu besebenzisa i-robustscaler, futhi okukodwa okunamathiselwe okufakiwe okuningana okuphakathi. Ngemuva kokuthola amandla, ukunemba kwemodeli kwathuthukiswa kusuka kuma-72% kuya ku-87%, okufakazela ukuthi indlela olungiselela ngayo idatha yakho ivame ukuba khona ngaphezu kwawo i-algorithm oyikhethayo.
Ngamafuphi, ukwenziwa kwangaphambili kuyisisekelo sanoma yimuphi umsebenzi wokufunda ngomshini. Landela imikhuba emihle kakhulu, gcina izinto zisobala, futhi ungabukeli phansi umthelela wazo. Uma kwenziwa kahle, kungathatha imodeli yakho kusuka kwesilinganiso kuya kwesinye isikhathi.
Imibuzo Ebuzwa Njalo (FAQ's)
1. Ingabe ukuvela kokulungiswa kwehlukile ngokufunda okujulile?
Yebo, kepha kancane. Ukufunda okujulile kusadinga idatha ehlanzekile, izici ezimbalwa ezimbalwa.
2
Uma isusa amaphethini anenjongo noma ukulimaza ukunemba okunemodeli, kungenzeka ukuthi uyangenwa.
3. Kungahle kweqiwe nge-data eyanele?
Cha. Imininingwane engaphezulu isiza, kepha okokufaka okusezingeni eliphansi kusaholela emiphumeleni engemihle.
3. Ngabe wonke amamodeli adinga ukwedlula okufanayo?
Cha. I-algorithm ngayinye inezinzwa ezihlukile. Okusebenzayo kweyodwa kungenzeka kungafaneli enye.
4. Ingabe ukujwayelekile kuhlale kudingekile?
Ikakhulukazi, yebo. Ikakhulu kuma-algorithms asuselwa ebanga anjenge-knn noma ama-SVMS.
I-5. Ungakwazi ukwenza ngokuzenzakalela ngokugcwele?
Hhayi ngokuphelele. Usizo lwamathuluzi, kepha ukwahlulela komuntu kusadingeka kumongo kanye nokuqinisekiswa.
Kungani ulandelela izinyathelo zokuthola amandla?
Iqinisekisa ukuvela kabusha futhi isiza ukubona ukuthi yini ethuthukisa ukusebenza noma ukulimaza ukusebenza.
Ukugcina
I-DATAS Preapocessing akuyona nje isinyathelo sokuqala, futhi yi-Bedrock yokufunda okuhle komshini. Idatha ehlanzekile, engaguquki iholela kumamodeli angewona anembe kuphela kodwa futhi athembeke. Ukusuka ekususeni izimpinda zokukhetha ukufaka inkomba ekhokhwayo, izindaba ngasinye. Ukweqa noma ukuhlekisa okwenziwe kabusha kuvame ukuholela emiphumeleni enomsindo noma ukuqonda okudukisayo.
Futhi njengoba izinselelo zedatha zivela, ukuqonda okuqinile kombono namathuluzi kuba yigugu ngokwengeziwe. Izindlela eziningi zokufunda izandla namuhla, njengalezo ezitholakala kwisayensi ephelele yedatha
Uma ufuna ukwakha amakhono aqinile, okufaka phakathi amakhono e-data we-data we-data, kufaka phakathi isipiliyoni sezandla ezinamasu wokuthola izinto, cabanga ukuhlola i-Master Data Science & Machine Ukufunda kuhlelo lwePython ngokufunda okuhle. Yenzelwe ukuvala igebe phakathi kwethiyori nokuzilolonga, kukusiza ukuthi usebenzise le miqondo ngokuqiniseka kumaphrojekthi wangempela.
