Ungayibona kanjani futhi uvikele ukugeleza okumodeli ngaphambi kokuthi kuthinte ibhizinisi lakho

Naphezu kwe-ai hype, izinkampani eziningi zobuchwepheshe zisathembela kakhulu ekufundeni komshini ezihlelweni ezibucayi, ezivela ezincomeni ezenzelwe wena zokuthola inkohliso.

Ngizibonele mathupha ukuthi ama-drift angatholwa angaholela kanjani ezindlekweni ezibalulekile – ukutholwa kwenkohliso okuphuthelwe, imali elahlekile, kanye nemiphumela yebhizinisi le-suffimal, ukubiza nje okumbalwa. Ngakho-ke, kubalulekile ukuba nokuqapha okuqinile endaweni uma inkampani yakho ihanjisiwe noma ihlelele amamodeli wokufunda womshini ekukhiqizweni.

Imodeli engabonakali yemodeli ingaholela ekulahlekelweni okubalulekile kwezezimali, ukungasebenzi kahle kwezezimali, futhi kulimaze nedumela lenkampani. Ukunciphisa lezi zingozi, kubalulekile ukuba nokuqapha amamodeli asebenzayo, okubandakanya:

• Ukulandelela ukusebenza kwemodeli
• Ukuqashelwa kokusatshalaliswa kwesici
• Ukuthola zombili izingane ze-UNIVARERE NOKUXHUMANA

Uhlelo lokuqapha olusetshenziswe kahle lungasiza ekuboneni izindaba kusenesikhathi, ukonga isikhathi eside, imali kanye nezinsizakusebenza.

Kulesi siqondisi esiphelele, ngizohlinzeka ngohlaka lokuthi ungacabanga kanjani futhi usebenzise ukuqapha okusebenzayo kwemodeli, kukusiza ukuthi uhlale ngaphambi kwezinkinga ezingaba khona futhi uqinisekise ukuqina nokwethenjwa kwamamodeli akho ekukhiqizeni.

Uyini umehluko phakathi kwesici sokushayela futhi uthole amaphuzu?

I-Score Drift ibhekisa ekushintsheni kancane kancane ekusatshalalisweni kwezikolo eziyimodeli. Uma kushiywe kungalawulwa, lokhu kungaholela ku yehla ukusebenza kwemodeliukwenza imodeli ibethe inembe kakhulu ngokuhamba kwesikhathi.

Ngakolunye uhlangothi, isici sokushayela senzeka lapho isici esisodwa noma ngaphezulu isipiliyoni sokusatshalaliswa. Lezi zinguquko kumanani esici zingathinta ubudlelwano obungaphansi kwemodeli elifunde ngalo, futhi ekugcineni kuholele ekubikezelweni kwemodeli engalungile.

Ukulingisa amashifu amaphuzu

Ukuze ngithole izinselelo zokuthola inkohliso yangempela, ngadala i-dataset yokwenziwa ngezici ezinhlanu zokuthenga ngokwezezimali.

Le khasi I-Reference Dataset imele ukusatshalaliswa kwasekuqaleni, ngenkathi Idatha yokukhiqiza wethula amashifu ukulingiswa ukwanda kwe ukuthengiselana okuphezulu kakhulu ngaphandle kokuqinisekiswa kwe-PIN kuma-akhawunti amasha, okubonisa ukwanda kokukhwabanisa.

Isici ngasinye sinokusatshalaliswa okuhlukile okungaphansi:

• Inani lokuthenga: Ukusatshalaliswa okujwayelekile okujwayelekile (kufakwe kwesokudla ngomsila omude)
• Iminyaka ye-akhawunti (izinyanga): Ukusatshalaliswa okujwayelekile okujwayelekile phakathi kuka-0 kuye ku-60 (kuthatha inkampani eneminyaka engu-5)
• Isikhathi selokhu okwenziwayo kokugcina: Ukusatshalaliswa kwe-Exponential
• Ukubala kokuthenga: Ukusatshalaliswa kwe-Poisson
• Kungene Pin: Ukusatshalaliswa kwe-binomial.

Ukulinganiselwa kwezikolo ezilinganayo, ngibelwe izinsimbi ezingemihlelwa ngezikhathi kulezi zici futhi ngisebenzise umsebenzi we-sigmoid ukuze ngicindezelwe izibikezelo phakathi kuka-0 kuye ku-1. Lokhu kulingisa indlela imodeli yenkohliso yokuhlehlisa.

Njengoba kukhonjisiwe esakhiweni esingezansi:

• Izici ezikhukhuliwe: Inani lokuthenga, ubudala be-akhawunti, ukubalwa kokuthenga, kanye ne-PIN efakiwe wonke amashifu anolwazi ekusatshalalisweni, esikalini, noma ebudlelwaneni.

• Isici esizinzile: Isikhathi selokhu okwenziwayo kokugcina wahlala engashintshiwe.

• Izikolo ezikhukhuliwe: Ngenxa yezici ezikhukhuliwe, ukusatshalaliswa kwezikolo eziyimodeli sekushintshile.

Lokhu kusetha kusivumela ukuthi sihlaziye ukuthi izikolo ze-Drift zifaka kanjani izikolo zemodeli ekukhiqizeni.

Ukuthola imodeli Score Drift usebenzisa i-PSI

Ukuqapha izikolo zemodeli, ngasebenzisa i-Capeal Steability Index (PSI) ukukala ukuthi kuguqulwe amamodeli angakanani ngokuhamba kwesikhathi.

I-PSI isebenza ngamaphuzu wemodeli eqhubekayo eqhubekayo nokuqhathanisa ingxenye yezikolo kubha ngamunye phakathi kwesethenjwa kanye nemininingwane yokukhiqiza. Kuqhathanisa umehluko ngezilinganiso ezilingana kanye nezilinganiso zazo ze-logarithmic ukufaka izibalo ezifingqiwe ezizokwengeza ukudonsa.

Ukuqaliswa kwePython:

# Define function to calculate PSI given two datasets
def calculate_psi(reference, production, bins=10):
  # Discretize scores into bins
  min_val, max_val = 0, 1
  bin_edges = np.linspace(min_val, max_val, bins + 1)

  # Calculate proportions in each bin
  ref_counts, _ = np.histogram(reference, bins=bin_edges)
  prod_counts, _ = np.histogram(production, bins=bin_edges)

  ref_proportions = ref_counts / len(reference)
  prod_proportions = prod_counts / len(production)
  
  # Avoid division by zero
  ref_proportions = np.clip(ref_proportions, 1e-8, 1)
  prod_proportions = np.clip(prod_proportions, 1e-8, 1)

  # Calculate PSI for each bin
  psi = np.sum((ref_proportions - prod_proportions) * np.log(ref_proportions / prod_proportions))

  return psi
  
# Calculate PSI
psi_value = calculate_psi(ref_data['model_score'], prod_data['model_score'], bins=10)
print(f"PSI Value: {psi_value}")

Ngezansi kwesifinyezo sendlela yokuhumusha amanani e-PSI:

• I-PSI <0.1: AKUKHO DRIFT, noma ukushayela okuncane kakhulu (ukusatshalaliswa okucishe kufane).
• 0.1 ≤ PSI <0.25: Eminye edonsa. Ukusatshalaliswa kuhlukile ngandlela thile.
• 0.25 ≤ PSI <0.5: I-Drift elinganiselayo. Ukushintshwa okubonakalayo phakathi kwesethenjwa nokusatshalaliswa kokukhiqiza.
• Psi ≥ 0.5: Drift ebalulekile. Kunokushintsha okukhulu, okubonisa ukuthi ukusatshalaliswa kokukhiqizwa kuguqukile kakhulu kusuka kudatha yereferensi.

Le khasi Inani le-PSI ye-0.6374 kuphakamisa ukuhamba okubalulekile phakathi kwenkomba yethu kanye nemininingwane yokukhiqiza. Lokhu kuqondanisa ne-histogram yamamodeli amaphuzu ukusatshalaliswa, okuqinisekisa ukushintshwa kokushintshwa kwezikolo eziphakeme ekukhiqizeni – okubonisa ukwanda kokuthengiselana okuyingozi.

Ukuthola Isici Drift

I-kolmogorov-smirnov test for izinombolo zezinombolo

Ukuhlolwa kweKolmororov-Smirnov (KS) kuyindlela yami engiyithandayo yokuthola ukudonsa ezicini zezinombolo, ngoba kunjalo non-parametric, okusho ukuthi akukuthathi ukusatshalaliswa okujwayelekile.

Isivivinyo siqhathanisa ukusatshalaliswa kwesici ku-Reference and Detation Datasets ngokulinganisa umehluko omkhulu phakathi kwemisebenzi yokusatshalaliswa kwe-Empirical Cumulative (ECDFS). Amabanga we-KS Statisting Sticing kusuka ku-0 kuye ku-1:

• 0 Ibonisa Umehluko phakathi kokusatshalaliswa kokubili.
• Amanani asondelene no-1 aphakamise ukuhamba okukhulu.

Ukuqaliswa kwePython:

# Create an empty dataframe
ks_results = pd.DataFrame(columns=['Feature', 'KS Statistic', 'p-value', 'Drift Detected'])

# Loop through all features and perform the K-S test
for col in numeric_cols:
    ks_stat, p_value = ks_2samp(ref_data[col], prod_data[col])
    drift_detected = p_value < 0.05
		
		# Store results in the dataframe
    ks_results = pd.concat([
        ks_results,
        pd.DataFrame({
            'Feature': [col],
            'KS Statistic': [ks_stat],
            'p-value': [p_value],
            'Drift Detected': [drift_detected]
        })
    ], ignore_index=True)

Ngezansi amashadi e-ECDF wezinto ezine ezinezinombolo kudatha yethuthi yethu:

Ake sibheke isici se-akhawunti yobudala I-Dataset yedatha yokukhiqiza ibheke kuma-akhawunti amasha, ngoba inengxenye enkulu yokubukwa ineminyaka ephansi ye-akhawunti.

Ukuhlolwa kwesikwele kwezimpawu zezigaba

Ukuthola amashifu ezicini ezihlukanisiwe ne-Boolean, ngithanda ukusebenzisa isivivinyo se-chi-skwele.

Lokhu kuhlolwa kuqhathanisa ukusatshalaliswa kwemvamisa kwesici sesigaba ku-Reference and Defence Datets, bese kubuyisa amanani amabili:

• Isibalo se-chi-skwele: Inani eliphakeme libonisa ukuhamba okukhulu phakathi kwesethenjwa kanye nemininingwane yokukhiqiza.
• I-p-value: Inani le-P elingezansi kwe-0.05 liphakamisa ukuthi umehluko phakathi kwesethenjwa kanye nemininingwane yokukhiqiza kubalulekile ngokwezibalo, okukhombisa ukuthi kungahle kube khona isici esidonsayo.

Ukuqaliswa kwePython:

# Create empty dataframe with corresponding column names
chi2_results = pd.DataFrame(columns=['Feature', 'Chi-Square Statistic', 'p-value', 'Drift Detected'])

for col in categorical_cols:
    # Get normalized value counts for both reference and production datasets
    ref_counts = ref_data[col].value_counts(normalize=True)
    prod_counts = prod_data[col].value_counts(normalize=True)

    # Ensure all categories are represented in both
    all_categories = set(ref_counts.index).union(set(prod_counts.index))
    ref_counts = ref_counts.reindex(all_categories, fill_value=0)
    prod_counts = prod_counts.reindex(all_categories, fill_value=0)

    # Create contingency table
    contingency_table = np.array([ref_counts * len(ref_data), prod_counts * len(prod_data)])

    # Perform Chi-Square test
    chi2_stat, p_value, _, _ = chi2_contingency(contingency_table)
    drift_detected = p_value < 0.05

    # Store results in chi2_results dataframe
    chi2_results = pd.concat([
        chi2_results,
        pd.DataFrame({
            'Feature': [col],
            'Chi-Square Statistic': [chi2_stat],
            'p-value': [p_value],
            'Drift Detected': [drift_detected]
        })
    ], ignore_index=True)

Isimo se-Chi-Square sika-57.31 ngenani le-P-Vate of 3.72E-14 liqinisekisa ukuhamba okukhulu esicini sethu sesigaba, Entered PIN. Lokhu kuthola ukuqondanisa nge-histogram ngezansi, okubonisa ukubonakala okubonakalayo:

Ukuthola amashifu e-multivariate

Ukuxhumeka kwama-Spearman kwama-Shifts ekusebenzeni okubili

Ngaphezu kokuqapha amashifu esici ngasinye, kubalulekile ukulandelela amashifu ebudlelwaneni noma ukusebenzisana phakathi kwezicieyaziwa ngokuthi amashifu e-multivariate. Noma ukusatshalaliswa kwezici ngazinye zihlala zizinzile, amashifu ahlukahlukene angabonisa umehluko onenjongo kwidatha.

Ngokuzenzakalelayo, ama-pandas ' .corr() Umsebenzi ubala ukuxhumeka kwePearson, okuthatha kuphela ubudlelwano obuqondile phakathi kwezinto eziguqukayo. Noma kunjalo, Ubudlelwano phakathi kwezici bavame ukungabi nalutho nokho usalandela umkhuba ongaguquguquki.

Ukuthwebula lokhu, siyasebenzisa Ukuxhumeka kweSpearman ukukala ubudlelwane be-monotonic phakathi kwezici. Ibamba noma ngabe izici Shintsha ndawonye Ngendlela engaguquki, noma ngabe ubudlelwano babo bungaqondile.

Ukuhlola amashifu ebudlelwaneni besici, siqhathanisa:

• Ukulungiswa kwereferensi (ref_corr): Ithwebula ubudlelwano besici somlando kudathabhethi yezethenjwa.
• Ukuhlobana Kwemikhiqizo (prod_corr): Ithwebula ubudlelwano obusha besici ekukhiqizeni.
• Umehluko ngokuphelele ekuhlolisweni: Izinyathelo zingakanani ubudlelwano besici buye bashintsha phakathi kwesethenjwa kanye nemininingwane yokukhiqiza. Amanani aphezulu akhombisa amashifu aphezulu.

Ukuqaliswa kwePython:

# Calculate correlation matrices
ref_corr = ref_data.corr(method='spearman')
prod_corr = prod_data.corr(method='spearman')

# Calculate correlation difference
corr_diff = abs(ref_corr - prod_corr)

Isibonelo: Shintsha ngokuhlobana

Manje, ake sibheke ukuxhumeka phakathi transaction_amount na- account_age_in_months:

• Phakathi kwa- ref_corrukuhlangana kungu-0.00095, okubonisa ubudlelwano obubuthakathaka phakathi kwalezi zici ezimbili.
• Phakathi kwa- prod_corrukuhlangana kungu -0.0325, okubonisa ukuhlangana okungalungile okubuthakathaka.
• Umehluko ngokuphelele ekuxhumekeni kwe-Spearman ngu-0.0335, okuwukushintsha okuncane kodwa okubonakalayo.

Umehluko ngokuphelele ngokuhlangana ukhombisa a gudlula ebudlelwaneni phakathi transaction_amount na- account_age_in_months.

Kwakungenakuba khona ubudlelwano phakathi kwalezi zici ezimbili, kepha i-dataset yokukhiqiza ikhombisa ukuthi manje sekunokuxhumana okungalungile okubuthakathaka, okusho ukuthi ama-akhawunti amasha anama-akhawunti aphezulu okuthengiselana. Lokhu kusobala!

I-AutoEncoder yamashifu ajwayelekile, anezici eziphakeme ze-multivariate

Ngaphezu kokuqapha ukusebenzisana kokubhala, singabheka ama-shiffs ngaphesheya kwemininingwane eminingi kwidatha.

Ama-autoencoders amathuluzi anamandla okuthola amashifu aphezulu aphezulu we-multivariatelapho izici eziningi zishintsha ngokuhlangene ngezindlela ezingabonakala ngokubheka ukusatshalaliswa kwezici ngazinye noma ukuxhumeka okubonakalayo.

I-autoencoder iyinethiwekhi ye-neural efunda ukumelwa okucindezelwe kwedatha ngezakhi ezimbili:

• I-Encoder: Icindezela idatha yokufaka esifanekisweni esilinganiselwe.
• Idicoder: Yenza kabusha okokufaka kwasekuqaleni kusuka ekumeleleni okucindezelwe.

Ukuthola amashifu, siqhathanisa Ukukhishwa kabusha ku okokufaka kwangempela futhi bahlanganise Ukulahleka kokwakhiwa kabusha.

• Ukulahleka kabusha kokwakhiwa kabusha okuphansi → I-autoencoder ihlela ngempumelelo imininingwane, okusho ukuthi ukubonwa okusha kuyafana nakho sekubonile futhi kwafunda.
• Ukulahleka kokwakhiwa kabusha okuphezulu → Idatha yokukhiqiza iyaphambuka kakhulu emaphethini afundile, ekhombisa okungahle kube khona.

Ngokungafani namamethrikhi we-Drift Drift agxile kuwo izici ngazinye noma ubudlelwane bobabiliukuthwebula kwama-autoencoders izinhlungu eziyinkimbinkimbi, ezingezona ezilayini ngaphesheya kwezinto eziningi ngasikhathi sinye.

Ukuqaliswa kwePython:

ref_features = ref_data[numeric_cols + categorical_cols]
prod_features = prod_data[numeric_cols + categorical_cols]

# Normalize the data
scaler = StandardScaler()
ref_scaled = scaler.fit_transform(ref_features)
prod_scaled = scaler.transform(prod_features)

# Split reference data into train and validation
np.random.shuffle(ref_scaled)
train_size = int(0.8 * len(ref_scaled))
train_data = ref_scaled[:train_size]
val_data = ref_scaled[train_size:]

# Build autoencoder
input_dim = ref_features.shape[1]
encoding_dim = 3 
# Input layer
input_layer = Input(shape=(input_dim, ))
# Encoder
encoded = Dense(8, activation="relu")(input_layer)
encoded = Dense(encoding_dim, activation="relu")(encoded)
# Decoder
decoded = Dense(8, activation="relu")(encoded)
decoded = Dense(input_dim, activation="linear")(decoded)
# Autoencoder
autoencoder = Model(input_layer, decoded)
autoencoder.compile(optimizer="adam", loss="mse")

# Train autoencoder
history = autoencoder.fit(
    train_data, train_data,
    epochs=50,
    batch_size=64,
    shuffle=True,
    validation_data=(val_data, val_data),
    verbose=0
)

# Calculate reconstruction error
ref_pred = autoencoder.predict(ref_scaled, verbose=0)
prod_pred = autoencoder.predict(prod_scaled, verbose=0)

ref_mse = np.mean(np.power(ref_scaled - ref_pred, 2), axis=1)
prod_mse = np.mean(np.power(prod_scaled - prod_pred, 2), axis=1)

Amashadi angezansi akhombisa ukusatshalaliswa kokulahleka kokwakhiwa kabusha phakathi kwazo zombili imininingwane.

Idatha yokukhiqiza inephutha eliphakeme elinobuqili eliphakeme kunaleso sedatha yezethenjwa, okukhombisa ukushintshwa kwemininingwane ephelele. Lokhu kuqondanisa noshintsho kwidatha yokukhiqiza ngenani eliphakeme lama-akhawunti amasha anokuthengiselana okuphezulu.

Bikazela

Ukuqapha amamodeli kuyinto ebalulekile, kepha evame ukunganakwa, umthwalo wemfanelo wososayensi wedatha kanye nonjiniyela bokufunda bemishini.

Zonke izindlela zezibalo ziholele esiphethweni esifanayo, esivumelana namashifu abonwe kwidatha: Batholakele umkhuba wokukhiqiza ubheke kuma-akhawunti amasha enza ukuthengiselana okuphezulu. Lokhu kuguqulwa kuholele ezingeni eliphakeme lemodeli, kusayina ukwanda kokukhwabanisa okungenzeka.

Kulokhu okuthunyelwe, ngamboza amasu wokuthola ukudonsa emazingeni amathathu ahlukene:

• Imodeli Score Drift: Ukusebenzisa Inkomba yokuqina yabantu (PSI)
• Isici Somuntu Esidliwayo: Ukusebenzisa Kolmogorov-smirnov test ezicini zezinombolo futhi Ukuhlolwa kwesikwele ngezinto ezihlukile
• I-multivariate drift: Ukusebenzisa Ukuxhumeka kweSpearman ngokuxhumana okubili futhi Ama-autoencoders Okwezinhlangothi eziphezulu, amashifu ahlukahlukene.

Lezi ngamasu ambalwa nje engithembela kukho ukuze aqaphe okuphelele – kunenqwaba yezinye izindlela ezivumelekile zezibalo ezivumelekile ezingathola ne-Drift ngempumelelo.

Amashifu atholakele avame ukukhomba izingqinamba ezingaphansi eziqinisekisa ukuqhubeka nophenyo. Imbangela yezimpande ingaba sína njenge-bug iqoqo ledatha, noma elincane njengenguquko yesikhathi efana nokulungiswa kwesikhathi semini.

Kukhona namaphakheji we-python amahle kakhulu, afana ngokusobala.i, lokho kushintsha okuningi kwalokhu kuqhathaniswa. Kodwa-ke, ngikholwa ukuthi kukhona inani elibalulekile ekuqondeni okujulile amasu wezibalo ngemuva kokutholwa kwe-DRITT, kunokuba ancike kumathuluzi kuphela.

Yini inqubo yokuqapha imodeli efana ezindaweni ozisebenzele?

---

Ufuna ukwakha amakhono akho e-AI?

👉🏻 Ngigijima I-AI Weekender na- Bhala okuthunyelwe kwebhulogi yamasonto onke kwisayensi yedatha, amaphrojekthi we-AI ngempelasonto ye-AI, izeluleko zomsebenzi zochwepheshe kwidatha.

---

Izinsizakusebenza