Ipulatifomu-mesh, hub futhi yakhuluma, futhi ibekwe phakathi | Izinhlobo ezi-3 zeqembu ledatha
Ukuqalisa
Ku “I-landscape eshintsha ngokushesha kwedatha ne-AI“(!), Ukuqonda idatha kanye nokwakhiwa kwe-AI akukaze kubucayi ngokwengeziwe. Kodwa-ke okuthile abaholi abaningi abakubuka kubaluleke kakhulu kwesakhiwo sethimba ledatha.
Ngenkathi iningi lenu lifunda lokhu mhlawumbe likhomba njengoba le khasi Ithimba ledatha, okuthile okungaboni ukuthi kungakhawulelwa kanjani ukuthi kungenzeka kube nomkhawulo kangakanani.
Ngempela, izinhlaka zeqembu ezahlukahlukene kanye nezidingo zamakhono zithinta kakhulu amandla enhlangano okusebenzisa idatha ne-AI ukushayela imiphumela enengqondo. Ukuqonda lokhu, kuyasiza ukucabanga nge-analoory.
Cabanga ngendlu yabantu ababili. UJohn usebenza ekhaya kanti uJane uya ehhovisi. Kunenqwaba yokulawulwa kwendlu uJane ukuthembela kuJohn okufanele akwenze, okulula kakhulu njengoba eyiyo ekhaya isikhathi esiningi.
UJane noJohn banezingane nangemva kokuba sebekhulile kancane uJohn unesibindi esiphindwe kabili esizokwenza! Ngenhlanhla, izingane ziqeqeshelwe ukwenza izisekelo; Bangageza, bacoceke futhi ngezikhathi ezithile benze kancane ukuhlehlisa ngokuphoqelelwa okuthile.
Njengoba izingane zikhula, abazali bakaJohn bangena ngaphakathi. Baguga, ngakho uJohn ubanakekela, kodwa ngenhlanhla, izingane zizanele ngalesi sikhathi. Ngokuhamba kwesikhathi indima kaJohane isishintshe kancane! Kepha uhlala ekwenza kube ngumndeni owodwa ojabulisayo, wenuzi – ngenxa kaJohn noJane.
Buyela kwidatha – UJohn ufana neqembu ledatha, futhi wonke umuntu uchwepheshe wesizinda. Bathembela kuJohn, kodwa ngezindlela ezihlukile. Lokhu kushintshe okuningi ngokuhamba kwesikhathi, futhi ukube bekungenjalo kungenzeka ngabe kwakuyinhlekelele.
Kuyo yonke le ndatshana, sizohlola uhambo lukaJohn kusuka endaweni ephakathi nendawo, ngokusebenzisa i-hub-and-ekhulume nethimba ledatha yesiteji ye-Mesh.
Amaqembu aphakathi nendawo
Iqembu eliphakathi linesibopho sezinto eziningi ezizojwayela:
- Ipulatifomu ledatha eliyisisekelo kanye nokwakhiwa: izinhlaka kanye namathuluzi asetshenziselwa ukwenza lula idatha kanye nemisebenzi ye-AI.
- Idatha ne-AI Engineering: Ukuhlanganisa nokuhlanza ama-datasets; Idatha ehlelekile engahleliwe yokulayishwa kwemisebenzi ye-AI
- I-BI: Ukwakha amadeshibhodi ukubona ukubona ukuqonda
- I-AI ne-ML: Ukuqeqeshwa kanye nokuhanjiswa kwamamodeli kwimininingwane ehlanzekile ehlisiwe
- Ukugqugquzela inani ledatha nokuqeqesha abantu ukuze baqonde ukusebenzisa amathuluzi e-BI
Lona umsebenzi omningi wabantu abambalwa! Eqinisweni, akunakwenzeka ukuthi ukhala kukho konke lokhu ngasikhathi sinye. Kuhle ukugcina izinto zincane futhi zilawuleke, zigxile kumacala ambalwa okusetshenziswa ambalwa futhi kuthambekele ekutholeni amathuluzi anamandla ukuze uthole ikhanda lokuqala kusenesikhathi.
Ungahle uthole ngisho ne-nanny noma i-au pair ukusiza ngomsebenzi (kulokhu – abeluleki).
Kepha le ndlela inamaphutha. Kulula ukuwela ogibeni lwe-silo, isimo lapho iqembu eliphakathi liba ibhodlela elikhulu ledatha nezicelo ze-AI. Amaqembu edatha adinga futhi ukuthola ulwazi lwesizinda kusuka kochwepheshe besizinda ukuze baphendule ngempumelelo izicelo, nazo ezidla isikhathi futhi zinzima.
Enye indlela yokuphuma ukwandisa iqembu. Abantu abaningi basho imiphumela eminingi. Kodwa-ke, kunezindlela ezingcono zesimanje ezingenza izinto zihambe ngokushesha okukhulu.
Kepha munye kuphela uJohn. Ngakho-ke yini angayenza?
Ngokwengxenye ihlukaniswe noma i-hub futhi yakhuluma
Isethaphu ehlukaniswe ngokwengxenye iyimodeli ekhangayo yezinhlangano eziphakathi nendawo noma amancane, ubuchwepheshe-bokuqala lapho kunamakhono obuchwepheshe ngaphandle kweqembu ledatha.
Ifomu elilula kakhulu lineqembu ledatha eligcina ingqalasizinda ye-bi, kepha hhayi okuqukethwe uqobo. Lokhu kushiyelwe 'abasebenzisi bamandla' abakuthatha njengezandla zabo futhi bakha izimbotshana zabo.
Lokhu, yiqiniso, kugijima kuzo zonke izinhlobo zezingqinamba, ezifana nesicupho se-silo, ukutholwa kwedatha, ukubusa kanye nokudideka. Ukudideka kubuhlungu ikakhulukazi lapho abantu abatshelwa ukuba bazenzele bazame futhi behluleke ngenxa yokungaqondi kwedatha.
Indlela ethandwa kakhulu yizendlalelo ezengeziwe zesitaki okufanele zivulwe. Kunokunyuka konjiniyela we-analytics kanye nabahlaziyi bedatha kuya ngokuya kuthatha umthwalo wemfanelo owengeziwe. Lokhu kufaka ukusebenzisa amathuluzi, ukwenza amamodeli wedatha, amapayipi okuphela kokuphela, nokukhuthaza ebhizinisini.
Lokhu kuholele ezinkingeni ezinkulu lapho zisetshenziswa ngokungalungile. Ubungeke uvumele indodana yakho eneminyaka emihlanu ukuba inakekele ukunakekelwa kwabadala bakho futhi inakekele indlu inganakekelwa.
Ngokukhethekile, ukuntuleka kwemigomo eyisisekelo yokulinganisa idatha kanye nezinjini zokugcina zedatha kuholela ekutheni izindleko zemodeli kanye nezindleko zokufakelwa. Kunezibonelo ezimbili zakudala.
Eyokuqala lapho abantu abaningi bezama ukuchaza into efanayo, njengemali engenayo. Ukumaketha, ezezimali, kanye nomkhiqizo bonke banenguqulo ehlukile. Lokhu kuholela ekubhekeni okungenakugwenywa ekubuyekezweni kwebhizinisi lekota lapho wonke umnyango ubika ngenombolo ehlukile – ukuhlaziya ukukhubazeka.
Elinye liyabalwa. Ake sithi imali ifuna imali etholakala ngenyanga, kepha umkhiqizo ufuna ukwazi ukuthi kuyini ngokwesikhombisa ngosuku lwesikhombisa. “Kulula lokho,” kusho umhlaziyi. “Ngizovele ngidale imibono ebonakalayo ngala ma-metric kuwo”.
Njengoba noma yimuphi unjiniyela wedatha uyazi, lokhu kusebenza kwamanani okuqalisa kuyabiza kakhulu, ikakhulukazi uma ubucwebe budinga ukuba ngosuku noma ngehora, kusukela lapho udinga ikhalenda 'imodeli' ephuma 'imodeli. Ngaphambi kokuthi wazi ukuthi kukhona rolling_30_day_sales , rolling_7_day_sales , rolling_45_day_sales njalo njalo. Lawa mamodeli abiza i-oda lobukhulu ngaphezu kwalokho obekudingeka.
Mane nje ucele ubuncane obuphansi kakhulu obudingekayo (nsuku zonke), ukwenza izinto ezibonakalayo lokho, futhi ukudala ukubukwa phansi komfula kungaxazulula le nkinga kodwa kungadinga insiza ephakathi.
I-Hub yokuqala futhi ekhuluma imodeli kumele ibe nokuzijabulisa okucacile uma ngabe ulwazi olungaphandle kwethimba ledatha luncane noma lubambi.
Njengoba amaqembu ekhula, ifa, izinhlaka kuphela zekhodi ezinjenge-Apache Airlow futhi zinikeza inkinga: ukuntuleka kokubonakala. Abantu abangaphandle kweqembu ledatha abafuna ukuqonda ukuthi kwenzekani bazokwethembela kumathuluzi angeziwe ukuze uqonde okwenzeka ekugcineni – kuze kube sekupheleni kwe-metadata emithonjeni ehlukene.
Kubalulekile ukwedlula lolu lwazi kuchwepheshe wesizinda. Kukangaki lapho utshelwe ukuthi 'idatha ingabukeki kahle', kuphela ukubona ngemuva kokulandela konke ngesandla ukuthi bekuyinkinga ohlangothini lomkhiqizi wedatha?
Ngokukhulisa ukubonakala, ochwepheshe besizinda baxhumeke ngqo kubanikazi bemininingwane yomthombo noma izinqubo, ezivumela ukulungiswa kusheshe. Lokhu kususa umthwalo ongadingekile, ukushintshana komongo, namathikithi eqembu ledatha.
Hub futhi ukhulume (umsulwa)
I-hub emsulwa futhi ikhulume kancane njengokudlulisa izingane zakho osemusha ngezibopho ezithile ngaphakathi kwama-Guardrali. Awugcini nje ukuthi ubanikeze imisebenzi yokwenza njengokuthatha imigqomo futhi bahlanze igumbi labo – ucela lokho okufunayo, njengegumbi “elihlanzekile nelicocekile,” futhi wethemba ukuthi bakwenze. Izisusa zisebenza kahle lapha.
Kwi-Hub emsulwa futhi ukhulume indlela, ithimba ledatha liphatha ipulatifomu futhi livumele abanye balisebenzise. Bakha izinhlaka zokwakha nokuthumela amapayipi e-AI nedatha, futhi baphathe ukulawulwa kokufinyelela.
Ochwepheshe besizinda bangakha izinto zokugcina-ekugcineni uma zidinga. Lokhu kusho ukuthi bangahambisa idatha, imodeli, i-orchestterate ipayipi, futhi basebenze ngama-AI noma amadeshibhodi njengoba bebona kufanelekile.
Imvamisa, iqembu eliphakathi nalo lizokwenza okuncane kwalokhu. Lapho amamodeli wedatha kuwo wonke ama-Domains ayinkimbinkimbi futhi edlula khona, cishe kufanele njalo athathe ubunikazi bokuletha amamodeli wedatha ye-Core. Umsila akufanele uchithe inja.
Lokhu kuqala ukufana nomqondo womkhiqizo wedatha – Ngenkathi iqembu lezezimali lingathatha ubunikazi bokutshala imali nokuhlanza idatha, iqembu eliphakathi nendawo lingaba yimikhiqizo ebalulekile yedatha efana netafula lamakhasimende noma i-invoice table.
Lesi sakhiwo sinamandla kakhulu njengoba sisebenzisana kakhulu. Imvamisa isebenza kuphela uma amaqembu wesizinda enamazinga aphezulu obuciko obukhulu bezobuchwepheshe.
Amapulatifomu avumela ukusetshenziswa kwekhodi futhi akukho-code ndawonye kunconywa lapha, ngaphandle kwalokho ukuxhomekeka kobuchwepheshe obunzima eqenjini eliphakathi kuzohlala kukhona.
OLUNYE LOMUNYE LESIQINISO SEZINHLOKO Ingabe Ukuqeqeshwa Nokusekelwa. Iqembu eliphakathi nendawo noma iHub lizosebenzisa isikhathi esithile lisekela futhi likhuphukela opikono ukuze wakhe umsebenzi we-AI nowedatha kahle ngaphakathi ngaphakathi kwama-Guardrails.
Futhi, ukuhlinzeka ukubonakala lapha kunzima ngezinhlaka zemithi yemihla ngemihla. Amaqembu aseCentral azobe esethweswa umthwalo we-metadata ezitolo ezisesikhathini, njengekhathalogi yedatha, ngakho-ke abasebenzisi bebhizinisi bangakuqonda okwenzekayo.
Enye indlela – Ochwepheshe besizinda esisekhuphukayo sokuba nezinhlaka zokufunda zobuchwepheshe ezijulile ze-Python zobuchwepheshe ezinama-curve wokufunda iziqu, kunzima kakhulu ukudonsa.
Umkhiqizo wePlatifomu / Umkhiqizo wedatha
Isiphetho semvelo ohambweni lwethu lwendlu yezemibono lusithatha luye ku-matnating yedatha noma indlela ye-mesh yesikhulumi.
Kuleli khaya, kulindeleke ukuthi wonke umuntu azazi ukuthi imithwalo yemfanelo yabo iyini. Zonke izingane zikhulile futhi zingathembela kuzo ukugcina indlu zihlele futhi zinakekele izakhamuzi zakhona. Kukhona ukusebenzisana okuseduze futhi wonke umuntu usebenza ndawonye ngomthungo.
Kuzwakala kahle, awucabangi!?
Empeleni, akuvamile lokhu kube lula. Ukuvumela amaqembu e-satellite ukuthi asebenzise eyawo ingqalasizinda futhi akhe noma yini abayifunayo indlela eqinisekile yokuphelelwa amandla futhi wehlise izinto phansi.
Noma ngabe wawungamisa ngokujwayelekile amaqembu kuwo wonke amaqembu, izindlela ezinhle kakhulu zizohlupheka.
Ngikhulumile emaqenjini angenakubalwa ezinhlanganweni ezinkulu njengamaketanga okuthengisa noma izindiza, futhi ugweme i-mesh akuyona inketho ngoba ukwahlukana kwamabhizinisi amaningi kuncike komunye nomunye.
Lawa maqembu asebenzisa amathuluzi ahlukile. Ezinye izinzuzo ze-Airflow Izimo kanye nezinhlaka zomhlaba ezakhiwe ngabeluleki eminyakeni edlule. Abanye basebenzisa ubuchwepheshe bamuva kanye ne-Blow, Blowed Data Stack.
Bonke balwa nenkinga efanayo; Ukusebenzisana, ukuxhumana, kanye nokuqhuma kwama-orchestrating kugeleza amaqembu ahlukene.
Ukusebenzisa iplatifomu eyodwa eyengeziwe yokwakha idatha kanye nokuhamba komsebenzi we-AI lapha kungasiza. Indiza yokulawula enobunye icishe ifane ne-orchestrator yama-orchestrators, ehlanganisa i-metadata ezindaweni ezahlukene futhi ikhombisa ukuphela ukuze kuqedwe uhlu lwezinkumbulo ezizindeni zonke.
Ngokwemvelo kwenza ngendiza ephumelelayo yokulawula lapho umuntu angahlangana khona amapayipi ahlulekile, ukuxhumana, futhi alulame – konke ngaphandle kokuncika eqenjini lobunjiniyela obumaphakathi obekuzoba yi-bottleneck yedatha ephakathi.
Kune-analogies ecacile yalokhu ebunjiniyela be-software. Imvamisa, ikhodi iphumela kumalogi ahlanganiswe yithuluzi elilodwa elifana neDatadog. Lawa mapulatifomu ahlinzeka ngendawo eyodwa ukubona konke okwenzekayo (noma kungenzeki), izexwayiso, kanye nokubambisana kwesinqumo sezehlakalo.
Ukubeka kafushane
Izinhlangano zifana nemindeni. Njengoba nje sithanda umbono womndeni owodwa, omkhulu, ojabulayo, owanele, kuvame ukuba yizibopho okudingeka sizelethe zokwenza izinto zisebenze ekuqaleni.
Njengoba bekhula, amalungu asondela ekuzimeleni, njengezingane zikaJohn. Abanye bathola indawo yabo njengababambiqhaza abathembekile kodwa abathembekile, njengabazali bakaJohane.
Izinhlangano azifani. Amaqembu wedatha avuthwa kude ne-Do-Ers emaqenjini ahlanganisiwe abenzile ku-HUB futhi axoxe ngokwakha. Ekugcineni, izinhlangano eziningi zizoba yinye inqwaba uma kungenjalo amakhulu abantu abayimininingwane ephayona kanye nokusebenza kwe-AI kwizikhulumi zazo.
Lapho lokhu kwenzeka, kungenzeka ukuthi idatha ne-AILLE ne-Agele kusetshenziswe kanjani ngobunzima bamabhizinisi amakhulu lapho ukusebenzisana kanye nokuhlela amaqembu ahlukahlukene kungavinjelwanga.
Ukuqonda lapho izinhlangano zihlobene nalezi zindlela zibalulekile. Izama ukuphoqa ingqondo–mindset yomkhiqizo enkampanini engavuthiwe, noma ukunamathela eqenjini elikhulu eliphakathi nenhlangano enkulu nevuthiwe kuzoholela enhlekeleleni.
Inhlanhla 🍀
