Ifundwa Kanjani I-Monotonicity kanye Nokuqina Kweziguquko Kumodeli Yokulinganisa usebenzisa iPython

Ngithanda ukubonga wonke umuntu ozinike isikhathi sokufunda nokuzibandakanya nesihloko sami. Ukwesekwa kwakho nempendulo yazisa ngempela.

Ungakwazi ukukhiqiza kabusha ukuhlaziya endaweni yami yokugcina ye-GitHub: Ukuthola Isikweletu ngePython.

akukhona nje ukuqeqesha i-algorithm yokufunda komshini nokuhlola ukusebenza kwayo nge-AUC noma i-Gini coefficient.

Abaningi abasaqalayo ekumodeleni baphuthuma ekuqeqesheni amamodeli, beqa izinyathelo ezibalulekile ezinquma ukuthi imodeli iqinile ngempela futhi iyatolika. Lo mdlandla, othatha imizuzu embalwa kuphela – isikhathi eside ngokwanele ukuze amamethrikhi okusebenza avele esikrinini – ngokuvamile asitha umsebenzi ojule futhi oqinile owandulela lesi sigaba.

Engozini yekhredithi, ikhwalithi yemodeli incike kakhulu ezintweni eziguquguqukayo ezisebenzisayo. Okuhlukile okubonakala kubikezela kudathasethi yokuqeqeshwa kungase kusebenze ngokungahambisani phakathi nesikhathi noma kubantu abahlukahlukene. Uma singakunaki lokhu, sizifaka engozini yokwakha imodeli eyenza kahle ekuthuthukisweni kodwa ehluleka ekukhiqizeni.

Lokhu kuphakamisa imibuzo emithathu eyisisekelo. Ingabe okuguquguqukayo okukhethiwe kubonisa ingozi yekhredithi eqhubekayo ngokuhamba kwesikhathi? Ingabe ukuthambekela kwale ngozi kuhlala kuzinzile unyaka nonyaka? Ingabe ukusatshalaliswa kwalokhu kuhlukahluka kuhlala kuqhathaniswa kuwo wonke amasethi edatha okuqeqesha, okuhlola, nangaphandle kwesikhathi?

– Ngiqale ngichaze imiqondo ye-monotonicity nokuzinza ekutholeni amaphuzu esikweletu.

– Bese ngisebenzisa le mibono eziguquguqukayo eziyisikhombisa ezikhethiwe kokuthunyelwe kwami ​​​​kwangaphambilini.

– Okokugcina, ngihlola ukuzinza kwedathasethi ngisebenzisa I-Population Stability Index (PSI) phakathi neminyaka nakuzo zonke izitimela, ukuhlolwa, namasethi edatha aphelelwe yisikhathi.

Kwethulwa Idatha

Eposini lami langaphambilini, ngethule indlela elula ehlanganisa ukuhlaziywa kobudlelwane obuguquguqukayo nokuqinisekiswa okuphambene ukuze ukhethe ngokuqinile okuguquguqukayo kwemodeli yokufaka amaphuzu. Le ndlela iqondakala kalula, kulula ukuyisebenzisa, futhi inamandla, ikakhulukazi uma ihlanganiswe nokuhlehla kwezinto, okuhlala kuyimodeli yereferensi ekutholeni amaphuzu esikweletu.

Ngigcine okuguquguqukayo okuyisikhombisa ngemuva kwenqubo yokukhetha:

Izinombolo ezinhlanu eyodwa [person_income, person_age, person_emp_length, loan_int_rate, and loan_percent_income]

futhi ezimbili ngokwezigaba eyodwa [person_home_ownership and cb_person_default_on_file].

Umbuzo engiwubuzayo manje ukuthi ingabe lezi ziguquko zibalulekile ngempela ekulinganiseni amapharamitha emodeli yokugcina yamaphuzu, nokuthi ngingasihumusha kanjani inkombandlela yengozi yokuguquguquka ngakunye.

Ukuchaza i-Monotonicity kanye nokuzinza

I-Monotonicity ibhekisela ekuhlaziyweni kwesiqondiso sobungozi bokuguquguquka okukhethwe kusengaphambili. Kokuguquguquka okuqhubekayo, kuphendula umbuzo olandelayo: uma inani lokuguquguquka likhuphuka noma lehla, ingabe ingozi yekhredithi iyanda noma yehla ngokufanele?

Isibonelo, esimweni sebhizinisi, silindele ukuthi uma imali engenayo yenkampani inyuka, isimo sayo sezezimali siba ngcono. Ngokuphambene, lapho imali engenayo yehla, isimo sayo sezezimali siba sibi. Lesi yisiqondiso sobungozi.

Ukuzinza iqhubekela phambili. Iphendula umbuzo: ingabe lesi siqondiso sengozi sihlonishwa ngokuqhubekayo phakathi neminyaka eminingi, noma sibona ukuguquguquka kwengozi? Ukuguqulwa kwengozi kwenzeka lapho, naphezu kokwenyuka kwemali engenayo, isimo sezezimali siwohloka – noma okuphambene nalokho. Ukuzinza kunikeza umbono wesikhathi eside wokuziphatha okuguquguqukayo futhi kusekela ukuthathwa kwezinqumo unolwazi.

Ekutholeni amaphuzu esikweletu, sifunda kokubili i-monotonicity yezinto eziguquguqukayo kanye nokuzinza kwazo ngokuhamba kwesikhathi. Siphinde sifunde ukuzinza kokusabalalisa okuhlukile phakathi kweminyaka elandelanayo naphakathi kwesitimela, ukuhlolwa, namasethi edatha aphelelwe yisikhathi.

I-Monotonicity kanye nokuzinza kwezinguquko

Lokhu kuhlaziya kusebenza njengesinyathelo sokukhetha kusengaphambili. Uma okuguquguqukayo kubonisa ukuguqulwa kwengozi ngokuhamba kwesikhathi, kufanele sikuphathe noma sikususe kumodeli. Ngokuhlukahluka okuqhubekayo, ukwelashwa ngokuvamile kuhlanganisa ukuhlukanisa okuhlukile nokuhlanganisa imigqomo yakhona. Ngokuguquguquka kwezigaba, singahlanganisa ngokuqondile izigaba ezithile.

Ukuchaza i-Risk Direction

Isinyathelo sokuqala siwukunikeza isiqondiso sengozi kokuguquguquka ngakunye.

Okwe ukuguquguquka okuqhubekayosabela uphawu oluthi “+” uma silindele ukuthi ukwanda kokuguquguquka kuholela ekwenyukeni kwengozi yekhredithi. Sinikeza uphawu oluthi “-” uma silindele ukuthi ukunyuka kuholela ekwehleni kwengozi yekhredithi.

Okwe okuguquguqukayo kwesigaba kanambambilisinikeza uphawu oluthi “+” uma ukusuka endaweni enobungozi kakhulu ukuya esigabeni esiyingozi kakhulu kwandisa ubungozi. Sinikeza uphawu “−” uma lwehlisa ubungozi.

Okwe ukuhlukahluka kwezigaba eziningiasabele uphawu olunambambili. Esikhundleni salokho, silinganisa izigaba ukusuka kweyingozi encane ukuya kweyingozi kakhulu ngokusekelwe kuzinga lazo lokuzenzakalelayo elisebenzayo. Isigaba esinezinga eliphansi lokuzenzakalelayo yisona esiyingozi kakhulu; lowo ophakeme kakhulu uyingozi kakhulu. Sibe sesiqinisekisa lesi simo ngochwepheshe bebhizinisi.

Ithebula elingezansi lifingqa inkomba yengozi elindelekile yenguquko ngayinye eqhubekayo efundwayo. U-“+” usho ukuthi ukunyuka kokuhluka kulindeleke ukuthi kukhuphule ingcuphe yekhredithi futhi ngaleyo ndlela kube namathuba okubaliwe okuzenzakalelayo. Igama elithi “−” lisho okuphambene.

Ngenza ukuphawula okubili okuqondile lapha. Ngoba ubudala_bomuntuiminyaka yobudala iwukuguquguquka okubucayi okungase kubandlulule abalingani. Silindele womabili amaqembu amancane kakhulu namadala kakhulu ukuthi athwale ubungozi obuphezulu, okwenza kube nzima ukunikeza isiqondiso esisodwa. Ngakho-ke sivumela idatha iveze iphethini yobungozi. Ngoba ubunikazi_bekhaya_bomuntuokuguquguqukayo kunezigaba eziningi, okwenza kube nzima ngokulinganayo ukwabela inkombandlela kanambambili i-priori. Silindele ukuthi isigaba se-RENT sithwale ubungozi obukhulu kakhulu, kulandele i-MORTGAGE, bese kuba ngu-OWN, kanti isigaba ESINYE sithwebula ozakwabo ezimeni ezididayo zezindlu. Sivumela idatha iqinisekise lokhu ku-oda.

Indlela Ewusizo

Empeleni, sihlola izinga lokuzenzakalelayo elisebenzayo ngokuhamba kwesikhathi ngamavelu achaziwe wezinto eziguquguqukayo ezichazayo. Ngamanani esiwachaza njengayingozi, silindele amanani azenzakalelayo aphezulu. Ngamanani esiwachaza njenganobungozi obuncane, silindele amanani aphansi azenzakalelayo.

Ngoba okuguquguqukayo okuqhubekayosibahlukanisa sisebenzisa ama-quantiles. Sisebenzisa ama-terciles — Q1, Q2, kanye ne-Q3 — sibala isilinganiso esimisiwe somgqomo ngamunye unyaka ngamunye. Uma okuguquguqukayo kunophawu oluthi “+”, isilinganiso esimisiwe ku-Q1 kufanele sibe ngaphansi kune-Q2, okufanele sibe ngaphansi kuka-Q3, ngaso sonke isikhathi. Ngokwesithombe, ijika le-Q3 lihlala ngaphezu kwejika le-Q2, elihlala ngaphezu kwejika le-Q1.

Ngoba okuguquguqukayo ngokwezigabasibala isilinganiso esimisiwe sesigaba ngasinye senkathi ngayinye. Ijika lesigaba esiyingozi kakhulu kufanele lihlale ngaphezu kwamajika azo zonke ezinye izigaba.

Isicelo: I-Monotonicity kanye Nokuqina Kweziguquko Eziyisikhombisa

Sisebenzisa lolu hlaka ezintweni eziyisikhombisa ezikhethwe ngaphambilini. Ukusatshalaliswa kokuguquguquka “okuzenzakalelayo” ngonyaka kusethi yokuqeqeshwa kungokulandelayo:

Izinguquko Eziqhubekayo

Sihlukanisa okuguquguqukayo okuqhubekayo kuma-terciles kusethi yokuqeqesha.

Iholo Lomuntu Ubungozi be-monotonicity buhlonishwa kuzo zonke izikhathi. Amaqembu aphikisayo anemali ephansi kakhulu abonisa amanani azenzakalelayo aphezulu kuyo yonke iminyaka. Asiboni ukuguqulwa kwengozi.

Iminyaka Yomuntu Ingozi ye-monotonicity ayihlonishwa. Sibona ukuguqulwa kwengozi, futhi i-Q2 ayikho kuyo yonke iminyaka. Lokhu okuguquguqukayo akunawo amandla okubikezela okuhlukanisa phakathi kozakwabo abahle nabahle kakhulu. Angikufaki ekuqhubekeni nokumodela.

Ubude bomsebenzi Ubungozi be-monotonicity buhlonishwa emhlabeni wonke kuyo yonke iminyaka.

Izinga Lenzalo Ubungozi be-monotonicity buhlonishwa iminyaka yonke.

Imali Engenayo Yephesenti Lokubolekwa Ubungozi be-monotonicity buhlonishwa umhlaba wonke kuyo yonke iminyaka ngalokhu kuguquguquka.

Okuguquguqukayo ngokwezigaba

Okuzenzakalelayo Komlando (cb_person_default_on_file) Ingozi ye-monotonicity iyahlonishwa. Amaqembu aphikisayo anomlando wokuzenzakalelayo abonisa amanani azenzakalelayo aphezulu kuzo zonke izikhathi. Lo mphumela uhambisana ngokuphelele.

Ubunikazi Bekhaya (ubunikazi_bomuntu_wekhaya) Ubungozi be-monotonicity buhlonishwa ezingeni lomhlaba wonke kodwa hhayi ezingeni lonyaka ka-2016, 2017, kanye no-2018.

Kulesi simo, sinezinketho eziningana. Ngikhetha ukuhlanganisa kabusha okuguquguqukayo kube izigaba ezintathu: OWN, MORTGAGE, kanye (RENT + OTHER). Ngemuva kokuhlanganisa kabusha, ubungozi be-monotonicity buhlonishwa umhlaba wonke.

Isifinyezo

Lokhu kuhlaziywa kwe-monotonicity kungiholela ekubeni ngikhiphe i-variable person_age, ukuzinza kwayo engozini okungahlonishwa. Ngigcina okuguquguqukayo okuyisithupha okusele ngesinyathelo esilandelayo.

Ukuzinza kwesethi yedatha

Manje ngifunda ukuzinza kokusabalalisa okuguquguqukayo. Umgomo uwukuqinisekisa ukuthi ukusatshalaliswa kokuhluka ngakunye kuhlala kufana phakathi neminyaka yonke naphakathi kwesitimela, ukuhlolwa, namasethi edatha aphelelwe yisikhathi.

I-Population Stability Index (PSI)

Sisebenzisa i-PSI – inkomba esebenzayo esetshenziswa kakhulu ekutholeni amaphuzu esikweletu – ukukala amashifu okusabalalisa. Kusebenza ngokuqondile eziguquguqukayo zezigaba. Ngokuhlukahluka okuqhubekayo, siyabahlukanisa kuqala. Kulesi sihloko, ngisebenzisa ama-terciles ngokuguquguquka okuqhubekayo.

Kokuhluka ngakunye, sibala ingxenye yokuqaphela emgqomeni ngamunye noma esigabeni sawo womabili amadathasethi. I-PSI ibe isiqhathanisa, umgqomo ngomgqomo, izilinganiso ezibonwa kudathasethi yereferensi nedathasethi eqondiwe, isebenzisa ifomula elandelayo ye-logarithmic:

$PSI = sum_{i=1}^{k} (p_i – q_i) cdot lnleft(frac{p_i}{q_i}right)$

Lapha, pᵢ futhi qᵢ isho izilinganiso ku-bin i wereferensi namadathasethi okuqondiwe, ngokulandelanayo. Kulesi sihloko, ngichaza ngokucacile indlela yokusebenzisa le nkomba. Uma ingaphansi kuka-10%, okuguquguqukayo kuthathwa njengokuzinzile. Uma ingaphansi kwama-25%, akukho shintsho oluphawulekayo olubonwayo.

Unyaka Nonyaka Ukuzinza

Ngiyahlola ukuthi ingabe ukusatshalaliswa kokuguquguquka ngakunye kushintshile ukusuka onyakeni owodwa kuya komunye.

Konke okuguquguqukayo kuzinzile ngokuhamba kwesikhathi – akukho ukwephulwa kwe-threshold okubukwayo (i-PSI ngaphansi kuka-10%).

Ukuzinza kwesethi yedatha

Ngihlola ukuzinza kokusabalalisa okuguquguqukayo kuwo wonke amadathasethi amathathu, ngihlola izimo ezintathu:

• Isitimela vs Test,
• Isitimela vs Ukuphuma Kwesikhathi,
• Futhi Ukuhlola vs Ukuphuma Kwesikhathi.

Akukho ukwephulwa kwe-threshold okubukwayo kuzo zonke izimo, okuqinisekisa ukuthi abashayeli bezingcuphe abakhethiwe bazinzile phakathi kokulinganisa namasethi okuhlola.

Isiphetho

Kulesi sihloko, ngethule uhlaka oluqinile lokufunda i-monotonicity nokuzinza kumodeli wokushaya amaphuzu. Ngabonisa indlela yokwabela isiqondiso sengozi kokuguquguquka ngakunye, indlela yokuqinisekisa lesi siqondiso phakathi neminyaka, kanye nendlela yokubona amashifu okusabalalisa kusetshenziswa i-PSI. Lesi sinyathelo – esivame ukweqiwa ekusebenzeni – sibalulekile ekuqinisekiseni ukuthi imodeli engiyakhayo ayisebenzi nje kuphela, kodwa futhi iqinile, iyachazeka, futhi ithembekile ngokuhamba kwesikhathi.

Eposini lami elilandelayo, ngizokwethula isilinganiso semodeli yokugcina amaphuzu ngisebenzisa okuguquguqukayo okuyisithupha okugciniwe.

Amakhredithi Esithombe

Zonke izithombe nokubonwayo kulesi sihloko kudalwe umlobi esebenzisa i-Python (i-pandas, i-matplotlib, i-seaborn, ne-plotly) kanye ne-excel, ngaphandle uma kushiwo ngenye indlela.

Izithenjwa

[1] U-Lorenzo Beretta no-Alessandro Santaniello.
Ama-algorithms Aseduze Wokuphoqelela Komakhelwane: Ukuhlola Okubalulekile.
Ilabhulali Kazwelonke Yezokwelapha, 2016.

[2] I-Nexialog Consulting.
Traitement des données maquantes dans le milieu bancaire.
Iphepha lokusebenza, 2022.

[3] U-John T. Hancock no-Taghi M. Khoshgoftaar.
Inhlolovo Ngedatha Yesigaba Yenethiwekhi Yezinzwa.
Ijenali Yedatha Enkulu, 7(28), 2020.

[4] UMelissa J. Azur, Elizabeth A. Stuart, Constantine Frangakis, noPhilip J. Leaf.
I-Imputation Multiple by Chained Equations: Iyini Futhi Isebenza Kanjani?
Ijenali Yamazwe Ngamazwe Yezindlela Zocwaningo Lwezengqondo, 2011.

[5] UMajid Sarmad.
Ukuhlaziywa Kwedatha Okuqinile Kwemiklamo Yokuhlola Yezimboni: Izindlela Ezithuthukisiwe Nesofthiwe.
Umnyango Wesayensi Yezibalo, University of Durham, England, 2006.

[6] UDaniel J. Stekhoven noPeter Bühlmann.
I-MissForest—I-Non-Parametric Missing Value Imputation ye-Mixed-Type Data.I-Bioinformatics, 2011.

[7] Supriyanto Wibisono, Anwar, and Amin.
I-Multivariate Weather Anomaly Detection Ukusebenzisa i-DBSCAN Clustering Algorithm.
Ijenali yeFiziksi: Uchungechunge Lwenkomfa, 2021.

[8] Laborda, J., & Ryoo, S. (2021). Ukukhethwa kwesici kumodeli yamaphuzu esikweletu. Izibalo, 9(7), 746.

Idatha Nelayisensi

Idathasethi esetshenziswe kulesi sihloko ilayisensi ngaphansi kwe I-Creative Commons Attribution 4.0 International (CC BY 4.0) ilayisense.

Le layisensi ivumela noma ubani ukuthi abelane futhi ajwayelane nedathasethi nganoma iyiphi injongo, okuhlanganisa ukusetshenziselwa ukuthengisa, inqobo nje uma ukuchasisa okufanele kunikezwe umthombo.

Ukuze uthole imininingwane eyengeziwe, bona umbhalo welayisense osemthethweni: CC0: Isizinda Esisesidlangalaleni.

Umshwana wokuzihlangula

Noma yimaphi amaphutha asele noma okungalungile kuwumthwalo wombhali. Impendulo nezilungiso zamukelekile.