Eda esidlangalaleni (Ingxenye 1): Ukuhlanza nokuhlola idatha yokuthengisa ngama-pandas

! Uyemukelwa ekuqaleni kohambo olukhulu lwedatha engibiza khona “u-Eda esidlangalaleni.” Kulabo abangaziyo, ngikholwa yindlela engcono kakhulu yokufunda noma yini ukubhekana nenkinga yomhlaba wangempela bese wabelana ngenqubo yonke engcolile – kufaka phakathi amaphutha, ukunqoba kanye nakho konke okuphakathi. Uma ubukade ufuna ukukhuphula ama-panda akho namakhono okuhlaziya idatha, lolu uchungechunge lwakho.

Sizosebenza njengabahlaziyi bedatha yenkampani ye-e-commerce ephakathi engaphakathi Noveshop. Basinikeza ukuthengisa okuluhlaza, okungcolile kwe-CSV futhi babuza umbuzo olula: “Senza kanjani?”

Umgomo we Ingxenye 1 Kuyisisekelo: Sizohlanza le datha le-e-commerce eliyinselele, sihlole isakhiwo salo esiyisisekelo, futhi sikwazi amakhono amakhono akwa-EDA kuma-pandas ukuthi wonke usosayensi wedatha esebenzisa nsuku zonke. Lolu chungechunge luhlelwe ukukuthatha kusuka ku saqala (Ingxenye 1) ku -phambili Umhlaziyi wedatha (Ingxenye 3), ngakho-ke zizwe ukhululekile ukugxumela lapho ukhona.

Ngaphambi kokuthi singene kwikhodi, ake sibethe phansi inkuthazo yethu. NgeNovashop, kudingeka siphendule imibuzo elula, kodwa enamandla: Yimiphi imikhiqizo eshayela imali engenayo kakhulu? Yimaphi amazwe akhiqiza ukuthengisa okuningi? Ake sibheke.

Ukubuka konke kwedatha: Khipha idatha yokuthengisa

Ukuqala ukuhlaziya kwethu kwe-novashop, sizobe sisebenzisa Idatha yokuthengisa ye-UCI online. Le yidatha enhle, enengqondo kakhulu, engalungiswa kahle edonsa konke ukuthengiselana kwenkampani yokuthengisa engeke ese-inthanethi e-Ek-online esitolo phakathi kuka-2010 nokufika kuka-2011.

Le datha inamalayisense ngaphansi kwelayisense le-Creative Commons Attribution eli-4.0 (CC ngu-4,0).

Lokhu kuvumela ukwaba kanye nokuzivumelanisa nemininingwane yanoma iyiphi injongo, inqobo nje uma kunikezwe isikweletu esifanele.

Idathasethi iqukethe izintambo ezingaphezu kwesigamu sesigidi, futhi igcwele izinhlobo zama-anomalies ohlangana nawo emhlabeni wangempela – amanani angekho, izinombolo ezingezinhle, nokufomatha kombhalo okungahambisani. Lokhu yilokho kanye esikufunayo!

Nawa amakholomu ayisishiyagalombili asemqoka sizobe sisebenza nalokho, nokuthi sisitshelani ngokombono webhizinisi:

• I-invoiceno: Yilokhu Inombolo ye-invoice. Inombolo ehlukile enamadijithi ayi-6 eyabelwe ukuthengiselana ngakunye. Uma ikhodi iqala ngo-'C ', kukhombisa a ukukhansela (ukubuya).
• Isitoko / ikhodi yomkhiqizo: Ikhodi enamadijithi ayi-5 ahlukile abelwe umkhiqizo ngamunye ohlukile.
• Incazelo: Igama lento. Lokhu kuvame ukudinga ukuhlanza (izikhala ezingeziwe, amacala angahambisani).
• Inani: Inani lezinto ezithengiwe. Mangaki amayunithi ahilelekile ngokuthengiselana ngakunye? Kungaba njalo -phikisayo ngembuyiselo.
• I-Endericate: Usuku nesikhathi sokuthengiselana. Lokhu kubalulekile ekuhlaziyweni kwe-Time-Series ngokuhamba kwesikhathi.
• Intengo yokukodwa: Intengo yomkhiqizo ngeyunithi ngayinye ku-sterling (GBP). Intengo yento eyodwa. Ngezinye izikhathi kungaba ngu-0 noma kubi ngenxa yamaphutha / izinto zamahhala.
• I-Coursed: Inombolo eyingqayizivele enamadijithi ayi-5 anikezwe ikhasimende ngalinye elibhalisiwe. Ngokusobala, lokhu kuvame ukulahleka, okusho ukuthi sinokuthengiselana okuningi kusuka kubavakashi.
• Izwe: Igama lezwe lapho ikhasimende lihlala khona. Lokhu kungaba kuhle ekuhlukaniseni ukuthengisa kwamazwe omhlaba.

Ake sithathe i-peek esheshayo emigqeni embalwa yokuqala ukubona ukuthi yini esibhekana nayo. Lokhu kungumphumela we df.head():

Masingenisa i-dataset zibe ama-pandas futhi zibone ukuthi mangaki imigqa esibhekene nayo.

import pandas as pd
import numpy as np
df = pd.read_csv(‘Online Retail.csv’)
df.shape

Okukhipha:

(541909, 8)

Yimigqa eminingi impela. Kungenzeka ukuthi ngihlaze imigqa kancane.

Ukulayishwa kwedatha nokusika: Ukubhekana neVolumu

Ingxenye 1, sizothatha a I-10% isampula engahleliwe yedatha ephelele. Lokhu kusinika usayizi olawuleke ngokwengeziwe – cishe imigqa engu-54,000 – ngenkathi kugcinwa izici zedatha yasekuqaleni. Le yindlela ejwayelekile futhi esebenzayo lapho ubhekene nezindawo ezinkulu zedatha noma lapho ugxile ekuhlolweni.

# Data Loading and Slicing
FILE_PATH = ‘Online Retail.csv’
SAMPLE_FRACTION = 0.1 # We will sample 10% of the data
full_df = pd.read_csv(FILE_PATH, encoding=’unicode_escape’)

# Take a random 10% sample for faster processing in Part 1
df = full_df.sample(frac=SAMPLE_FRACTION, random_state=42).reset_index(drop=True)

Manje, ake sihlole ukwakheka futhi kusetshenziswa df.shape

Okukhipha:

(54191, 8)

Phezulu! Manje sesingaqala.

Ukuhlolwa kwedatha yokuqala: Ukuthola Izandla Zethu Zingcolile

Isinyathelo sami sokuqala kuzoba ukuthola lokho esikuhlanzayo. Ngidinga ukuphendula le mibuzo

• Ngabe i-dataset iqukethe ini?
• Yiziphi izinhlobo zedatha namanani angewona ama-null?
• Ngabe izinombolo zibukeka kanjani?

Ukuhlola Okubonakalayo: Df.head () ne-DF.tail ()

Uma ngibheka imigqa yokuqala neyokugcina, ngingaqinisekisa izinto ezimbalwa:

• Ukukhanselwa nokubuya: Ngabona ukuthi ngibona abanye InvoiceNo amanani aqala nge 'C'kanye nokuhambelana Quantity ayilungile. Lokhu kuqinisekisa ukubuya kufakiwe, kanye nokuhlaziywa kwemali, ngizodinga ukuhlukanisa noma ukuhlukanisa.
• Ukulahleka: Ngingabona ngibona NaN amanani ku CustomerID Ikholamu, eqinisekisa inqwaba yokukhokhelwa kwezivakashi.
• Umbhalo ongahambisani: Ngihlole ikholamu yencazelo yama-whitespase. Ngokusekelwe kwisampula elincane engilitholile, angikwazanga ukusho ngempela. Kepha akulimazi ukubhekana nakho lapho ngihlanza idatha. Ngingahle futhi ngigcine imali eguquguqukayo. Kuhlale kuyindlela engcono kakhulu yokwenza strip konke okuholayo / okulandela umkhondo Kuzo zonke izintambo amakholomu ukuvikela amaphutha acashile lapho amaqembu.
• Ngiphinde ngabona okunye ukungahambisani kwemali enkulu kwikholamu yezwe. Ngabona izwe laqamba ama-Erire. Cishe kusho i-Ireland, kungadinga ukukushintsha lokho.

Yiziphi izinhlobo zedatha namanani angewona ama-null? (.info()Isihlehlukene

Isinyathelo esilandelayo esibalulekile ukubheka ukwakheka usebenzisa .info() indlela. Lokhu kungitshela uhlobo lwedatha yekholomu ngayinye futhi, okubaluleke kakhulu, mangaki amanani angewona ama-non-null (angashoda) esinawo.

Ukutholwa Okusemqoka

• Amanani Alahlekile: Ngemuva kokuthatha isampula yethu eyi-10%, ngabona igebe elikhulu ngaphakathi CustomerID. Cishe ama-25% wama-ID wamakhasimende alahlekile. Ngiphinde ngabona amanani angamakhulu ambalwa alahlekile ngaphakathi Description ukuthi ngizodinga ukubhekana.
• Izinhlobo zedatha: Le khasi InvoiceDate ikholomu isabalalwe njenge object (String) uhlobo. Kufanele uguqule lokhu ku-panda efanelekile datetime into.CustomerID futhi okwamanje a thwala amaphikocishe ngoba iqukethe lezo NaN Amanani! Lokhu kunemininingwane emincane ngizodinga ukukhumbula uma ngike ngafuna ukuyisebenzisa njenge-ID ye-Integer eyiqiniso.

Ngabe izinombolo zibukeka kanjani? (.describe()Isihlehlukene

Okulandelayo, ngasebenzisa .describe() ukuthola isifinyezo esisheshayo sezibalo sazo zonke izinombolo zamakholomu (Quantity na- UnitPrice).

• Ubuningi (QuantityIsihlehlukene: Ubuncane obuncane -2472. Lokhu kufakazela ukuthi ukubuya kukhona, kepha isilinganiselo salawo maphuzu aphansi siphakamisa ukuthengiselana okwedlulele. Ngokuhlaziywa okuyisisekelo kokuthengisa, ngingahle ngidinge ukuhlunga lezi zinombolo ezingezinhle futhi ngingaba khona abathengisi abahle kakhulu nababi.
• Intengo yokukodwa (UnitPriceIsihlehlukene: Intengo ephansi 0. Lokhu kusho ukuthi eminye imikhiqizo yanikezwa mahhala noma ingabe okufakiwe kwendawo. Njengoba umkhiqizo kufanele ube nentengo enhle ekuthengisweni okujwayelekile, ukuhlunga noma yimiphi imigqa lapho UnitPrice Ingabe zero noma ngaphansi kuhlale kuyindlela engcono kakhulu ukubala imali ngokunemba.

Ngokusekelwe kulokhu kuhlola okusheshayo okwenziwe. Le datha ayikude nokuhlanzekile. Sinokukhohlakala okukhulu, izinhlobo zedatha ezingalungile, amanani amabi anezinkinga ezingezinhle kakhulu kumakholomu ethu ayisisekelo, kanye nemibhalo yokungahambelani.

Ukuphatha amanani alahlekile

Manje njengoba sesazi ukuthi idatha yethu ilahlekile kuphi, singaqala ukuxoxa ngamasu okuphatha amanani angenayo. Ngikhathazekile kakhulu nge Description na- CustomerID.

Izincazelo ezingekho

Le khasi Description Usitshela ukuthi yimuphi umkhiqizo othengisiwe, ngakho-ke ukulahlekelwa kunikeza ukuthengiselana okungenanjongo. Kusampula yethu, ngaphansi kwe-1% yemigqa inezincazelo ezingekho. Ukuphonsa leyo migqa kunengqondo ngokuphelele, ngoba ayinamsebenzi yokuhlaziywa ezingeni lomkhiqizo.

# Drop rows where the Description is missing
df.dropna(subset=['Description'], inplace=True)
# checking for null counts
df['Description'].isnull().sum()

Okukhipha:

np.int64(0)

Kulungile, uphelele! Onke amanani angenalutho ahambile.

Ama-ID amakhasimende alahlekile

Okulahlekile CustomerIDs (cishe ama-25% wesampula yethu) ayisivumelwano esikhulu kakhulu. Uma ngibashiya bonke, ngizolahlekelwa cishe ikota yedatha yethu yokuthengisa, ezonikeza i-novashop umbono ohlelwe okuyingozi wemali engenayo ephelele.

Ngemali engenayo kanye nokuhlaziywa komkhiqizo (inhloso yengxenye 1), angiyidingi ngempela CustomerID. Singaqhubeka nokuhlaziywa kuyo yonke imigqa Noma ngaphandle i-id. Sidinga umazisi kuphela uma sihlela ukwenza ukuhlukaniswa kwamakhasimende (njengokuhlaziywa kwe-RFM), engizosindisa ingxenye 2!

Ukuguqulwa kohlobo lwedatha kanye nokususwa okuphindwe kabili

Manje njengoba sesixazulule izincazelo ezingekho, izindaba ezimbili ezilandelayo ezilandelayo zibhekene nemigqa ephindwe kabili futhi zilungisa okubalulekile kwethu InvoiceDate ikholomu.

Ukulungisa uhlobo lwedethi yosuku lwe-invoice

Khumbula Kanjani .info() kukhonjisiwe InvoiceDate njengentambo (object)? Sidinga ukulungisa lokhu ngokushesha ngakho-ke ama-pandas ayazi ukuthi ungahlela kanjani futhi ahlanganise idatha yethu ngezikhathi ezithile.

# Convert InvoiceDate from object (string) to datetime
df[‘InvoiceDate’] = pd.to_datetime(df[‘InvoiceDate’])

Ukususa izimpinda

Uma imigqa emibili ifana ngaphesheya -nke Amakholomu, aphindaphindwa futhi azokwenza kahle izinombolo zethu zokuthengisa. Ake sizihlole futhi uzisuse.

# Remove Duplicates
num_duplicates = df.duplicated().sum()
print(f”Found {num_duplicates} fully duplicated rows.”)

Okukhipha:

Found 62 fully duplicated rows

Asibadedele

# Remove duplicates, keeping the first instance
df.drop_duplicates(inplace=True)
print(f”Dataframe size after removing duplicates: {len(df)} rows.”)

Okukhipha:

Dataframe size after removing duplicates: 53980 rows.

Ukuhlunga ukubuyisa namaphutha

-Kwethu .describe() Indlela isitshengise amafulegi abomvu amabi: amanani amabi (ukubuyisa) kanye namanani entengo e-zero / amabi (amaphutha / izinto zamahhala). Engxenyeni 1, sifuna ukubala Imali engenayo evela ekuthengisweningakho-ke kufanele sihlungele lo msindo.

Ukuphatha amanani amabi namanani

Sizogcina ukuthengiselana kuphela lapho:

• Quantity iqinile kune-zero (ukuhlunga konke ukubuyisa kanye nokukhanselwa).
• UnitPrice ayiphathi ngokuphelele kune-zero (ukuhlunga amaphutha nezinto zamahhala).

# Filter: Keep only transactions where Quantity is positive (i.e., sales, not returns)
df = df[df[‘Quantity’] > 0]

# Filter: Keep only transactions where UnitPrice is positive (i.e., not free or an error)
df = df[df[‘UnitPrice’] > 0]
print(f”Dataframe size after filtering: {len(df)} rows.”)

Okukhishwayo

Dataframe size after filtering: 52933 rows.

Ukuhlanza Amakholomu Umbhalo

Okokugcina, ake sibhekane nalezo zinkinga ezijwayelekile ezibonakalayo sabona ngokubona, njenge EIRE Ikhodi yezwe kanye nanoma yikuphi okufihliwe okucashile ku Description.

• I-Strip Whitespace: Sisebenzisa .str.strip() Ukususa izikhala eziholayo / ezilandelanayo kuzo zombili Description na- Country.
• Izulu Lizwe: Sisebenzisa imephu engahambisani ne- 'Erire' to 'Ireland'.

# Cleaning Text Columns
# Clean Description and Country columns
df[‘Description’] = df[‘Description’].str.strip()
df[‘Country’] = df[‘Country’].str.strip()

# Handle specific country name inconsistencies
# EIRE is a common inconsistency in this dataset for Ireland
df[‘Country’].replace(‘EIRE’, ‘Ireland’, inplace=True)
print(“Text columns cleaned and standardized.”)

Isici sobunjiniyela nokuqonda okokuqala

Idatha manje ihlanzekile. Manje ekugcineni singaqala ukubuza imibuzo emnandi. I-metric eyodwa ebaluleke kakhulu kunoma iyiphi idatha yokuthengisa iyimali engenayo. Kusukela idatha yethu yangempela inayo kuphela Quantity na- UnitPricesidinga unjiniyela Imali engenayo kithi ngokwethu.

Isici sobunjiniyela: ukudala Revenue Insika

Imali engenayo yokuthengiselana kumane nje inani lezinto ezithengisiwe ngentengo yento ngayinye.

df[‘Revenue’] = df[‘Quantity’] * df[‘UnitPrice’]

Ukuqonda kokuqala: Yimaphi amazwe ashayela imali engenayo?

Masisebenzise idatha yethu ehlanzekile ukuphendula umbuzo we-novashop: “Yimaphi amazwe aqhuba ukuthengisa kwethu?”
Sizosebenzisa inhlanganisela enamandla yezinyathelo ezintathu:

• Isixhobo ngu Country ikholomu.
• -Qonde eceleni (sum) the Revenue ngaphakathi kweqembu ngalinye.
• Khetha imiphumela evela kwemali engenayo ephansi kakhulu.

# Group by Country, sum the Revenue, and sort for the top 10
top_countries = df.groupby(‘Country’)[‘Revenue’].sum().sort_values(ascending=False).head(10)
print(“n — — Top 10 Countries by Revenue (GBP) — -”)
print(top_countries)

Okukhipha:

--- Top 10 Countries by Revenue (GBP) ---
Country
United Kingdom 941268.661
Netherlands 27435.830
EIRE 26066.000
France 23645.330
Germany 22389.510
Australia 12429.990
Spain 5600.900
Switzerland 5483.890
Hong Kong 3597.850
Belgium 3593.510
Name: Revenue, dtype: float64

Ukukhishwa kwalokhu kukhombisa ukubusa okukhulu yi i-United Kingdom. Lokhu kulindeleke kusukela enkampanini kusekelwe e-UK. Amazwe amaningana alandelayo asinika umgwaqo osheshayo we-NoveAshop's International Focus agxilwe kuyo.

Ukugcina

Sakwenza. Sithathe i-dataset eluhlaza, yesigidi-a-million, silindile ngokulawulwa, silwa namanani alahlekile, izinhlobo zedatha ezihleliwe, zifakwe amaphutha ethu, futhi kubalwa ama-metric wethu wokuqala webhizinisi. Umsebenzi onzima, wesisekelo wenziwa.

Nakhu ukubuyiselwa okusheshayo kwalokho esikunqobile engxenyeni 1:

• I-Data Vetting: Sasebenzisa .head(), .info()futhi.describe() Ukuhlonza izingqinamba ezibalulekile njengamanani amabi / amanani amabi, ama-ID amakhasimende alahlekile, nefomethi ye-datetime engalungile.
• Ukuhlanza Idatha: Sisuse ngokuhlelekile ama-nulls nezimpinda, ziguqulwe InvoiceDate ukuze kufanele datetime into, futhi yahlunga ukuthengiselana okungathengisi (ukubuya nezinto zamahhala).
• Isici sobunjiniyela: Sidale ubucayi Revenue ikholomu.
• Ukuqonda okokuqala: Sikhiqize amazwe aphezulu ayi-10 ngemali engenayo ye-NOVANASHOP, abanikeze indawo yabo yokuqala yedatha kusuka kusethi ehlanzekile.

Idathasethi ehlanzekile manje isibelwe ukuhlaziywa okuyinkimbinkimbi ngokwengeziwe. Engxenyeni 2, sizongena ekujuleni kwenhliziyo Ukuhlaziywa komkhiqizo kanye nesikhathi sokuhunyushwa. Sizothola ukuthi yiziphi izinto abenzi bangempela bemali futhi bahlaziya ukuthi ivolumu yokuthengisa ishintsha kanjani ihora nehora nenyanga ngenyanga.

Ngijabule ngokuqhubeka! Uma ukujabulele le ndatshana. Zizwe ukhululekile ukungazisa nganoma iyiphi yalezi ziteshi. Impendulo yakho izosho lukhulu kimi.

Okuphakathi

I-LinkedIn

Uhlobo lwenkani

I-YouTube