Ithimba le-Netflix AI Lisika Ingxenye Ebanzi Yokufunda Ukubambezeleka ukusuka kumasekhondi kuye kuma-Millisecond ngokuhlukanisa izingxenye ze-Cassandra nge-ID ngayinye

Ithimba lonjiniyela leNetflix lishicilele indlela yokuphatha ama-partitions abanzi e-Apache Cassandra. Umsebenzi wocwaningo uqondise ku-TimeSeries Abstraction ye-Netflix, inkundla yedatha yomcimbi wesikhashana.

I-TL;DR

• Ukwahlukanisa okunamandla kuhlukanisa izingxenye ezibanzi ze-Cassandra nge-TimeSeries ID ngayinye, ngokuhambisanayo nangokusobala, ngaphandle kwezinguquko zohlelo lokusebenza.
• Ukuthola kugijima endleleni efundwayo ngokubala ibhayithi kanye nomcimbi we-Kafka; ukuhlukanisa okuhlosiwe kuma-partitions angaguquleki kuqala.
• Izihlungi ze-Bloom (ama-microsecond enedijithi eyodwa) kanye nenqolobane wide_row indlela yokubheka imethadatha ifundeka kuma-partitions ezingane ezincane.
• Amasheke, ama-partitions agciniwe agciniwe, amasheke e-Data Bridge Spark, nokunemba konogada bokuqhathaniswa kwethunzi.
• Ukufunda kuthuthukisiwe ukusuka kumasekhondi ukuya kuma-milliseconds anedijithi ekabili ephansi; ukubambezeleka komsila kwehle kwafika ku-~200 ms; Ama-partitions angu-500MB+ ahlale etholakala.

Iyini i-Dynamic Repartitioning?

I-TimeSeries Abstraction ye-Netflix ingenisa futhi ibuza ama-petabyte edatha yomcimbi wesikhashana nge-millisecond latency. Isebenzisa i-Apache Cassandra 4.x njengesitoreji esiyisisekelo. I-Cassandra yakhethelwa ukusebenza, ukubambezeleka, izindleko, nokuvuthwa kokusebenza.

Idatha yochungechunge lwesikhathi ihlelwa yaba izingxenye eziqoqa imicimbi ngesihlonzi kanye nebanga lesikhathi. Lezi zingxenye zingakhula 'zibanzi' njengoba imicimbi iqoqwa ngokuhamba kwesikhathi. Ukuhlukanisa kabusha okunamandla kuhlukanisa ukwahlukanisa okukhulu kube izingxenye ezincane zezingane ngokulandelana. Izinhlelo zokusebenza zilokhu zibuza ngendlela efanayo ukwahlukanisa okunengqondo kuyilapho isakhiwo sokulondoloza sishintsha ngokusobala.

Kungani Ukuhlukaniswa Okubanzi Kulimaza Kufundwa

Kumadathasethi amaningi, ukubambezeleka kokufunda okumaphakathi kuhlala kuma-millisecond enedijithi eyodwa. Uma ama-partitions ekhula abe banzi kakhulu, ukubambezeleka kokufundwa komsila kuyenyuka kube imizuzwana. Lokhu kufunda kancane kungaveza ukuphela kwesikhathi sokufunda. Ezimweni ezimbi kakhulu, amaqoqo abona ukumiswa kokuqoqwa kukadoti, ukusetshenziswa okuphezulu kwe-CPU, kanye nomugqa wochungechunge.

Amaseva e-TimeSeries aphinde aphathe ukufundwa okuphakeme kakhulu, okuhlanganisa inkinga. Ukukhuphula iqoqo leCassandra kuhlale kuyinketho. Kepha ithimba leNetflix belifuna ezinye izindlela ezihlakaniphile kunokuphonsa imali eningi enkingeni.

Isu Lokuhlukanisa Ngemuva kwe-TimeSeries

I-TimeSeries ihlukanisa amasethi edatha abe izingcezu zesikhathi ezihlukene: Izingcezu zesikhathi, amabhakede esikhathi, namabhakede omcimbi. Lokhu kuvumela ithimba le-Netflix ukuthi libuze futhi lilahle idatha ngesikhathi ngaphandle kokudala amatshe amathuna. Uma indawo yegama (idathasethi) idaliwe, abasebenzisi bacacisa izici zomthwalo womsebenzi okulindelekile. Ipayipi lokuhlinzeka lisebenzisa ukulingisa kwe-Monte Carlo ukuze kukhethwe ingqalasizinda kanye nokuhlelwa kokuhlukanisa.

Le ndlela yokubheka phambili iyafinyela ezimeni ezintathu. Umthwalo womsebenzi ungaziwa noma ulinganiselwe ngokungalungile ekuqaleni kwephrojekthi. Umthwalo womsebenzi ungase uguquke njengoba izidingo zethrafikhi nemikhiqizo zishintsha. Okokugcina, ama-data outliers akhona, lapho ama-ID ambalwa athola imicimbi eminingi kakhulu.

I-Discrete Time Slices inikeza i-hatch yemvelo yokuphunyuka kwezimo ezimbili zokuqala. I-Time Slice ngayinye entsha ingasebenzisa isu elihlukile lokuhlukanisa. Kodwa ukulungisa mathupha izinkulungwane zamasethi edatha akuzinzile, ngakho-ke i-automation iyadingeka.

Isixazululo 1: Ukuhlukaniswa kabusha Kwesiqephu Sesikhathi

I-Cassandra iveza ama-API wokuzihlola njenge nodetool tablehistograms ngamaphesenti osayizi wokuhlukanisa. Isisebenzi sangemuva siqapha lawa mahistogram futhi siwaveze ngethebula elibonakalayo le-Cassandra. Ibala isici sokulungisa lapho osayizi bokuhlukanisa begeja ukuminyana okumisiwe. Lokho kuminyana kuvame ukusethwa phakathi kuka-2 MiB no-10 MiB, kuye ngomthwalo womsebenzi.

Isibonelo, ukunikeza kanye okukhethiwe amabhakede esikhathi semizuzwana engu-60, okukhiqiza ama-partitions angaphansi kuka-10 KB. Lokho kuhlukaniswa ngokweqile kubangele ukukhuliswa kokufunda okuphezulu kanye nomugqa wentambo. Isisebenzi siyabuyekeza esizayo Izingcezu zesikhathi ezinesu elilungisiwe:

DynamicTimeSliceConfigWorker:
namespace: my_dataset_1
Observed: TimeSlices have p99 partitions below configured target of 10MB.
Proposed: time_bucket interval: 60s -> 604800s

Lokhu kunciphise ukubambezeleka kokufunda kanye nokuvala isikhathi okubangelwa umugqa wochungechunge. Kodwa kusiza kuphela uma iningi letafula ligunyaza ukuhlukaniswa kabusha. Akusizi uma kuphela amaphesenti omazisi ebanzi. Kulawo macala ayingxenye, Ithimba leNetflix linikeza izinketho ezintathu:

• Ungenzi lutho: indlela elungile lapho amamethrikhi ezinga eliphezulu engabonisi umthelela.
• Ukubuyisela Ingxenye: ichitha isicelo sokundiza esiphula i-SLO yokubambezeleka, ibuyisela idatha eqoqwe kuze kube manje.
• Vimba ama-ID: isinyathelo esidlulele sokuhlola noma ama-ID kagaxekile aphazamisa isistimu.

Ama-ID okuvimbela asetshenziswa ngokulungiselela, afaka kuhlu ama-ID e-TimeSeries ahlukumezayo:

dgwts.config..block.Ids: ", , "

Lezi zinketho azisizi lapho evumelekile, omazisi ababalulekile bakhula banzi. Labo abafonayo basadinga wonke umcimbi, kulapho Isixazululo 2 sisebenza khona.

Isixazululo 2: I-Dynamic Partitioning nge-ID ngayinye

I-Dynamic partitioning iyipayipi elingavumelanisi elihlukanisa izingxenye ezibanzi nge-TimeSeries ID ngayinye. Isebenza ezingeni le-ID, hhayi ileveli yethebula. Inezigaba ezintathu: Ukutholwa, Ukuhlela & Ukuhlukanisa, kanye Nokufundwayo Kokuhlinzeka.

Ukutholwa kwenzeka endleleni efundwayo. Wonke amathrekhi amabhayithi afundiwe afundiwe ukuze ahlukaniswe. Uma amabhayithi edlula umkhawulo omisiwe, iseva ikhipha umcimbi ku-Kafka:

{
  "time_slice": "data_20260328",
  "time_series_id": "profileId:123",
  "time_bucket": 7,
  "event_bucket": 2,
  "immutable": true,
  "version": "0"
}

Lapha, immutable imaka ingxenye engasatholi imibhalo. I version inkambu igcinelwe ukusetshenziswa esikhathini esizayo, njengokungavumelekile. Ithimba le-Netflix lithola ekufundeni, hhayi ekubhaleni, ngoba idatha eminingi ayidingi ukuhlukaniswa. Okunye okufundwayo kuma-partitions abanzi kuhlala kuhamba kancane imizuzwana kuze kube yilapho ipayipi lifinyelela phezulu. Ukuqaliswa kokuqala kuhloswe ukwahlukanisa okungaguquki ukuze kuncishiswe ubunkimbinkimbi.

Ukuhlela ufunda sonke isahlukaniso kanye ukuze abale uhlelo olunembile lokuhlukanisa. Ukuhlola kuvumela ukuhlela okuhlulekile ukuthi kufundeke endaweni yokugcina elondoloziwe. I wide_row Izitolo zethebula lemethadatha zihlukanisa izifunda, izindawo zokuhlola, nolwazi lomgudu.

Ukuhlukanisa izithunywa zesu elifana nalokhu EventBucketPartitionSplitStrategy. Inikeza amabhakede amaningi omcimbi ebhakedeni lesikhathi esifanayo. Ngokuhlukaniswa okubanzi kakhulu, ivala amabhakede omcimbi ukulawula ukukhulisa ukufundwa. Ukusabalala kuzo zonke izingxenye kusabalalisa ukufundwa phezu kwe-Cassandra replicas.

Iyaqinisekisa iqhathanisa i-checksum ehlukaniswe ngaphambilini ne-checksum yangemva kokuhlukaniswa. Ukuhlukana kumakiwe COMPLETED kuphela uma womabili amasheke ehambisana. I-Netflix iphinde ilandelele osayizi bokuhlukanisa bangaphambi nangemuva kokuqinisekisa ukuthi ama-split alinganiswe kahle.

Amaseva e-TimeSeries ngezikhathi ezithile alayisha okhiye bokuhlukanisa abaqediwe abaqediwe kuzihlungi ze-Bloom zenkumbulo. Konke okufundwayo kuhlola isihlungi se-Bloom, esiphendula ngamasekhondi amancane anedijithi eyodwa. Lelo sheke lincane ngokwanele ukuba lingabonakali kwabafonayo. Lapho ishaya, iseva iyafunda wide_row imethadatha yokuhambisa umbuzo:

{
  "pre_split_data": {
    "time_slice": "data_20260328",
    "time_series_id": "6313825",
    "time_bucket": 0,
    "event_bucket": 2
  },
  "post_split_data": {
    "time_slice": "wide_data_20260328_0",
    "event_bucket_partition_strategy": {
      "target_event_buckets": 2,
      "start_event_bucket": 32
    }
  }
}

Lokho kufundwa kwemethadatha kusekelwa inqolobane yokufunda. Okukhona PartitionReader bese ihlinzeka ngokufundwa kusuka kuma-partitions amancane ahlukene. Ukusebenzisa kabusha i-schema esifanayo kunciphisa izinguquko zekhodi, futhi imiphumela iyahlanganiswa ngaphambi kokubuya. Ukwahlukanisa okubanzi koqobo akulokothi kususwe, okuhlinzeka ngokubuyela emuva okuphephile kokwehluleka okuyingxenye kanye nokuvumelana ekugcineni.

Ithimba le-Netflix liphinde laqinisekisa ukuhlukaniswa okungaxhunyiwe ku-inthanethi lisebenzisa imisebenzi yeDatha Bridge Spark. Ukukhishwa okunezigaba kuthuthuke ngamamodi okufunda njengoba ukuzethemba kukhula. Isigaba sayo sokuqhathanisa siqhathanise amabhayithi anikezwa izindlela zokufunda ezindala nezintsha kumodi yesithunzi.

Isixazululo 1 vs Isixazululo 2: Ukuqhathanisa

Sebenzisa Amacala anezibonelo

• Amalogi omsebenzi wabasebenzisi aphile isikhathi eside: A profileId:123 iminyaka enqwabelanayo yemicimbi yokudlala iba banzi. Isixazululo sesi-2 siyisabalalisa kuwo wonke amabhakede omcimbi, ngakho-ke ukuqagela kuhlala kutholakala. I-Netflix ifake ama-partitions angu-500MB+ ngenkathi isatholakala isebenzisa le ndlela.
• Idivayisi noma i-IoT telemetry: Isethi encane yamadivayisi ezingxoxo abusa umthamo womcimbi. Isixazululo 2 sihlukanisa lawo ma-ID ashisayo ngaphandle kokuhlukanisa kabusha ithebula lonke.
• Idathasethi entsha ehlinzekwe ngaphezulu: Iqembu liqagela amabhakede amasekhondi angu-60 futhi lithola ama-partitions angaphansi kwe-10 KB. Isixazululo 1 sinweba isikhawu sezingcezu zesikhathi esizayo, ukusika ukukhulisa ukufundwa.
• Amadeshibhodi azwelayo ukubambezeleka: Oshaya ucingo udinga impendulo esheshayo kunedatha ephelele. Ingxenye Yokubuyisela ibeka ukubambezeleka ku-SLO futhi ibuyisela okuqoqiwe.

Izame: Idemo Yezihlukanisi Ezisebenzayo

Isifanisi esishumekiwe esingezansi simodela indlela efundwayo ekupheleni iye ekupheleni. Setha usayizi wokuhlukanisa, umkhawulo wokuthola, kanye nenani lebhakede lomcimbi oqondiwe. Ukuqalisa ukufunda kuhamba ngokutholwa, ukuhlela, ukuhlukanisa, ukuqinisekiswa, kanye nokunikeza. Ibonisa umcimbi wokuthola i-Kafka kanye nokwehla okulinganiselwe kwe-latency; izinombolo ziyimifanekiso, azilinganiswa.

Imiphumela

Ukubambezeleka kokufunda okumaphakathi kwama-partitions abanzi kwehlile ukusuka kumasekhondi ukuya kuma-milliseconds anedijithi ekabili ephansi. Ukubambezeleka komsila kwehle kusuka kumasekhondi ambalwa kuya cishe ku-200 ms noma okungcono. Ukuphela kwesikhathi sokufunda kwehlile, futhi ithimba le-Netflix libike ukusetshenziswa okuphansi kwe-CPU kanye nomugqa omncane wochungechunge. Sekukonke, amaqoqo eCassandra azinzile.

Ngemigqa ebanzi ngokwedlulele, isevisi ingase ifake iphethini futhi ibuze ngama-partitions angu-500MB+ ngenkathi isatholakala. Izahlukaniso ezinjalo ngaphambilini zibangele ukuphela kwesikhathi njalo kanye nokungatholakali kwama-blips. Umbuzo ophethiwe ubuya ngempumelelo, uhweba ukubambezeleka okuphakeme ukuze kutholakale:

{
  "next_page_token": "...",
  "records": [ { "...": "..." } ],
  "response_context": [
    {
      "namespace": "...",
      "time_taken": "41.072410142s"
    }
  ]
}

Ithimba leNetflix libala umsebenzi wesikhathi esizayo, okuhlanganisa nokuhlukanisa eguquguqukayo ukwahlukanisa okubanzi nokucubungula kabusha ukuhlukaniswa okuhlulekile ngaphambilini. Izifundo ezimbili ziyagqama. Ukunciphisa indawo engaphezulu yenguquko eyinkimbinkimbi kanye nokuphakela ngokunyuka kuyakhokha ngokusebenza. Ukutshala imali ezindleleni zokwakha ukuzethemba kuvunyelwa ubunkimbinkimbi besici, irediyasi yokuqhuma, nomthelela.

Hlola Imininingwane yobuchwepheshe lapha. Futhi, zizwe ukhululekile ukusilandela Twitter futhi ungakhohlwa ukujoyina wethu 150k+ML SubReddit futhi Bhalisela ku Iphephandaba lethu. Linda! ukutelegram? manje ungasijoyina kuthelegramu futhi.

Udinga ukusebenzisana nathi ekuthuthukiseni i-GitHub Repo yakho NOMA Ikhasi Lobuso Lokugona NOMA Ukukhishwa Komkhiqizo NOMA I-Webinar njll.? Xhuma nathi