# Introduction
All analysis starts the same way: you load a dataset and try to figure out what exactly is in it. How many lines? Which columns are numbers? How much money is missing? Is something crooked? Many of us answer those questions by copying and pasting the same df.describe(), df.isna().sum()again df.groupby(...).agg(...) snippets we've typed a thousand times, and reformat the output by hand when it's time to put it into a report.
That is a waste of effort. The Python ecosystem now has tools to take it from a raw DataFrame to a formatted summary table, shared on one or two lines – and some are specifically designed to produce the kind of “Table 1” you see so often in research papers. This tutorial walks you through 7 steps to building a recurring pipeline rather than a bunch of one-off snippets. We will use the Palmer Penguins database throughout. It's small, open, and has a realistic mix of numeric and categorical columns, real missing values, and natural group diversity (species). So let's get started.
# 1. Setting Your Location and Loading Data
Install the packages we will use in this tutorial:
pip install pandas seaborn skimpy tableone great-tables fg-data-profilingAn important note on the last one: the popular library has been renamed more than once. From the beginning pandas-profilinghe was ydata-profiling in 2023, it was renamed again fg-data-profiling in April 2026. Elder ydata-profiling the package still installs and works, but it no longer receives updates, so new projects should be preferred fg-data-profiling. We will include both import styles in step 5.
Now load the data. Seaborn has a built-in dataset for penguins, which we save for download:
import pandas as pd
import seaborn as sns
df = sns.load_dataset("penguins")
print(df.shape)
print(df.dtypes)
print(df.isna().sum())Output:
(344, 7)
species object
island object
bill_length_mm float64
bill_depth_mm float64
flipper_length_mm float64
body_mass_g float64
sex object
dtype: object
species 0
island 0
bill_length_mm 2
bill_depth_mm 2
flipper_length_mm 2
body_mass_g 2
sex 11
dtype: int64You will see seven columns: in three categories (species, island, sex) and four numerical values (bill_length_mm, bill_depth_mm, flipper_length_mm, body_mass_g). The measurements have 2 missing values each, too sex has 11. Stick with these details — we'll see how each tool reports this missing data.
# 2. Finding the Base with df.describe()
Built-in pandas describe() it's an obvious start, and for good reason. It is fast and does not require additional installation:
Output:
bill_length_mm bill_depth_mm flipper_length_mm body_mass_g
count 342.00 342.00 342.00 342.00
mean 43.92 17.15 200.92 4201.75
std 5.46 1.97 14.06 801.95
min 32.10 13.10 172.00 2700.00
25% 39.22 15.60 190.00 3550.00
50% 44.45 17.30 197.00 4050.00
75% 48.50 18.70 213.00 4750.00
max 59.60 21.50 231.00 6300.00This is really useful, but be aware of its blind spots. By default it ignores section columns entirely. It gives you a calculation non-null values but never tells you the missing percentages directly. And it boils down to five numbers – no skewness, no kurtosis, no sense of distribution. A quick gut check is fine, but as a basis for a report it leaves gaps. The next step is closing some of them without leaving the Pandas.
# 3. Pushing Pandas Further include, .agg()again groupby
Before getting external packages, it's worth knowing how far Pandas alone can take you – because for most everyday work, this is enough.
First, repeat the columns of the sections to form the same summary with include="all":
df.describe(include="all").round(2)Output:
species island bill_length_mm bill_depth_mm flipper_length_mm body_mass_g sex
count 344 344 342.00 342.00 342.00 342.00 333
unique 3 3 NaN NaN NaN NaN 2
top Adelie Biscoe NaN NaN NaN NaN Male
freq 152 168 NaN NaN NaN NaN 168
mean NaN NaN 43.92 17.15 200.92 4201.75 NaN
std NaN NaN 5.46 1.97 14.06 801.95 NaN
min NaN NaN 32.10 13.10 172.00 2700.00 NaN
25% NaN NaN 39.22 15.60 190.00 3550.00 NaN
50% NaN NaN 44.45 17.30 197.00 4050.00 NaN
75% NaN NaN 48.50 18.70 213.00 4750.00 NaN
max NaN NaN 59.60 21.50 231.00 6300.00 NaNNow you get it unique, topagain freq in text columns corresponding to numeric values (with NaN to fill in cells where calculations do not work). One table, per column.
Second, create a custom shortcut with .agg() so you control exactly which figures appear – including those describe() left, such as skewness and kurtosis – and bolt on the percentage of missing data:
numeric = df.select_dtypes("number")
summary = numeric.agg(["mean", "median", "std", "skew", "kurt"]).T
summary["missing_pct"] = df[numeric.columns].isna().mean().mul(100).round(1)
summary.round(2)Output:
mean median std skew kurt missing_pct
bill_length_mm 43.92 44.45 5.46 0.05 -0.88 0.6
bill_depth_mm 17.15 17.30 1.97 -0.14 -0.91 0.6
flipper_length_mm 200.92 197.00 14.06 0.35 -0.98 0.6
body_mass_g 4201.75 4050.00 801.95 0.47 -0.72 0.6Third – and this is the step people forget – the chain groupby() before the describe() to get a stratified summary in one line:
df.groupby("species")["body_mass_g"].describe().round(1)Output:
count mean std min 25% 50% 75% max
species
Adelie 151.0 3700.7 458.6 2850.0 3350.0 3700.0 4000.0 4775.0
Chinstrap 68.0 3733.1 384.3 2700.0 3487.5 3700.0 3950.0 4800.0
Gentoo 123.0 5076.0 504.1 3950.0 4700.0 5000.0 5500.0 6300.0Pattern to be made inside: select_dtypes selecting columns, .agg([...]) choosing math, groupby choosing strata. These three handle a surprising share of the actual reporting needs independently. The remaining steps are about saving time, adding polish, and handling cases where “not good enough” is wrong.
# 4. Getting the Rich Console Summary with skimpy
When you want more than that describe() but don't want to put it together by hand, skimpy the right choice. It's like a supercharge describe() that runs on your console or notebook and handles all column types at once. It works with both Pandas and Polars DataFrames.
from skimpy import skim
skim(df)One call prints a structured report: a summary of the data (row and column counts), a breakdown by data type, a table of numbers with mean, standard deviation, absolute percent spread, and an ASCII histogram in a row for each column, and a separate table of string columns showing things like character counts and most/least common values. Missing data is reported as both a count and a percentage, where you would expect it.
Output:
╭──────────────────────────────────────────────── skimpy summary ─────────────────────────────────────────────────╮
│ Data Summary Data Types │
│ ┏━━━━━━━━━━━━━━━━━━━┳━━━━━━━━┓ ┏━━━━━━━━━━━━━┳━━━━━━━┓ │
│ ┃ Dataframe ┃ Values ┃ ┃ Column Type ┃ Count ┃ │
│ ┡━━━━━━━━━━━━━━━━━━━╇━━━━━━━━┩ ┡━━━━━━━━━━━━━╇━━━━━━━┩ │
│ │ Number of rows │ 344 │ │ float64 │ 4 │ │
│ │ Number of columns │ 7 │ │ string │ 3 │ │
│ └───────────────────┴────────┘ └─────────────┴───────┘ │
│ number │
│ ┏━━━━━━━━━━━━━━━━━━━━┳━━━━┳━━━━━━━━━━━━━━━━━━━━┳━━━━━━━┳━━━━━━━┳━━━━━━┳━━━━━━━┳━━━━━━━┳━━━━━━┳━━━━━━┳━━━━━━━━┓ │
│ ┃ column ┃ NA ┃ NA % ┃ mean ┃ sd ┃ p0 ┃ p25 ┃ p50 ┃ p75 ┃ p100 ┃ hist ┃ │
│ ┡━━━━━━━━━━━━━━━━━━━━╇━━━━╇━━━━━━━━━━━━━━━━━━━━╇━━━━━━━╇━━━━━━━╇━━━━━━╇━━━━━━━╇━━━━━━━╇━━━━━━╇━━━━━━╇━━━━━━━━┩ │
│ │ bill_length_mm │ 2 │ 0.5813953488372093 │ 43.92 │ 5.46 │ 32.1 │ 39.23 │ 44.45 │ 48.5 │ 59.6 │ ▃█▆█▃ │ │
│ │ bill_depth_mm │ 2 │ 0.5813953488372093 │ 17.15 │ 1.975 │ 13.1 │ 15.6 │ 17.3 │ 18.7 │ 21.5 │ ▄▅▆█▆▂ │ │
│ │ flipper_length_mm │ 2 │ 0.5813953488372093 │ 200.9 │ 14.06 │ 172 │ 190 │ 197 │ 213 │ 231 │ ▂██▄▆▃ │ │
│ │ body_mass_g │ 2 │ 0.5813953488372093 │ 4202 │ 802 │ 2700 │ 3550 │ 4050 │ 4750 │ 6300 │ ▂█▆▄▃▁ │ │
│ └────────────────────┴────┴────────────────────┴───────┴───────┴──────┴───────┴───────┴──────┴──────┴────────┘ │
│ string │
│ ┏━━━━━━━━━┳━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━┳━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━┳━━━━━━━━━━━━┓ │
│ ┃ column ┃ NA ┃ NA % ┃ shortest ┃ longest ┃ min ┃ max ┃ chars/row ┃ words/row ┃ total words┃ │
│ ┡━━━━━━━━━╇━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━╇━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━╇━━━━━━━━━━━━┩ │
│ │ species │ 0 │ 0 │ Adelie │ Chinstrap │ Adelie │ Gentoo │ 6.59 │ 1.00 │ 344 │ │
│ │ island │ 0 │ 0 │ Dream │ Torgersen │ Biscoe │ Torgersen │ 6.09 │ 1.00 │ 344 │ │
│ │ sex │ 11 │ 3.19767442 │ Male │ Female │ Female │ Male │ 4.99 │ 0.97 │ 333 │ │
│ └─────────┴────┴────────────┴──────────┴───────────┴────────┴───────────┴────────────┴───────────┴────────────┘ │
╰──────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯Line histograms are an outstanding feature. You get to learn about the status of each distribution without opening the programming library. For interactive testing, skimpy is a good middle ground: it has a lot more information than describe()It's much simpler than a full HTML report.
# 5. Generating a Full Interactive Report on Profiling
When you need the complete picture — distribution, correlation, correlation, duplicate detection, and data quality warnings — is better to produce a full profile report. This is the one line that has taken the place of the afternoon plot of many analysts.
For a stored package:
from data_profiling import ProfileReport # fg-data-profiling
profile = ProfileReport(df, title="Penguins Profiling Report", explorative=True)
profile.to_file("penguins_report.html")If you are a legacy package, only the import line changes:
import ydata_profiling # legacy ydata-profilingThe output is a self-contained, interactive HTML file with overview categories (size, memory, duplicate rows), individual variable details (descriptive statistics and histograms), correlations across several coefficients, missing values analysis, and automatic warning flags, high cardinality, fixed columns, and the like.
A section of the generated profile report
One tradeoff is speed: a cluttered report covers a lot, and slows down for large datasets. Two arguments fix that. Use it minimal=True to turn off the expensive calculation, and replace the sample profile with the whole frame if you just need to hear about the data:
profile = ProfileReport(df.sample(frac=0.5), minimal=True)There is also a .compare() how to put the two data sets in each other – they are important for detecting drift between the training set and the production data, or between two periods.
# 6. Creating a Real “Table 1” with tableone
Everything so far is yours you – testing services. tableone for your students. Produces a separate table of baseline characteristics that opens almost every clinical and quantitative research paper (hence “Table 1”), with formatting and statistics expected by reviewers.
from tableone import TableOne
data = df.dropna(subset=["sex"])
columns = ["bill_length_mm", "bill_depth_mm",
"flipper_length_mm", "body_mass_g", "island", "sex"]
categorical = ["island", "sex"]
nonnormal = ["body_mass_g"] # summarize with median [IQR] instead of mean (SD)
table1 = TableOne(
data,
columns=columns,
categorical=categorical,
nonnormal=nonnormal,
groupby="species",
pval=True,
smd=True,
)
print(table1.tabulate(tablefmt="github"))The result is a well-formatted table: continuous variables like mean (SD)sections like n (%)anything you flagged as unusual median [Q1,Q3] – divided between your group variables, with missing data column, p-values, and standard mean difference (SMD) between groups:
| | | Missing | Overall | Adelie | Chinstrap | Gentoo | SMD (Adelie,Chinstrap) | SMD (Adelie,Gentoo) | SMD (Chinstrap,Gentoo) | P-Value |
|------------------------------|-----------|-----------|------------------------|------------------------|------------------------|------------------------|--------------------------|-----------------------|--------------------------|-----------|
| n | | | 333 | 146 | 68 | 119 | | | | |
| bill_length_mm, mean (SD) | | 0 | 44.0 (5.5) | 38.8 (2.7) | 48.8 (3.3) | 47.6 (3.1) | 3.315 | 3.023 | -0.393 | <0.001 |
| bill_depth_mm, mean (SD) | | 0 | 17.2 (2.0) | 18.3 (1.2) | 18.4 (1.1) | 15.0 (1.0) | 0.062 | -3.022 | -3.220 | <0.001 |
| flipper_length_mm, mean (SD) | | 0 | 201.0 (14.0) | 190.1 (6.5) | 195.8 (7.1) | 217.2 (6.6) | 0.837 | 4.140 | 3.119 | <0.001 |
| body_mass_g, median [Q1,Q3] | | 0 | 4050.0 [3550.0,4775.0] | 3700.0 [3362.5,4000.0] | 3700.0 [3487.5,3950.0] | 5050.0 [4700.0,5500.0] | 0.064 | 2.885 | 3.043 | <0.001 |
| island, n (%) | Biscoe | | 163 (48.9) | 44 (30.1) | 0 (0.0) | 119 (100.0) | 1.819 | 2.153 | nan | <0.001 |
| | Dream | | 123 (36.9) | 55 (37.7) | 68 (100.0) | 0 (0.0) | | | | |
| | Torgersen | | 47 (14.1) | 47 (32.2) | 0 (0.0) | 0 (0.0) | | | | |
| sex, n (%) | Female | | 165 (49.5) | 73 (50.0) | 34 (50.0) | 58 (48.7) | <0.001 | 0.025 | 0.025 | 0.976 |
| | Male | | 168 (50.5) | 73 (50.0) | 34 (50.0) | 61 (51.3) | | | | |The real advantage is exporting. A TableOne item goes directly to the format you need for your manuscript – LaTeX (pasted into Overleaf), HTML, Markdown, or CSV:
table1.to_latex("table1.tex")
table1.to_html("table1.html")
table1.to_csv("table1.csv")One important thing that package authors emphasize, and so would I: automatic calculations do not replace judgment. Choosing which tests to run, whether variations are really normal, and how to handle the lack of all human review before anything goes to publication. tableone removes tedium, not burden.
# 7. Polishing it into a Publishing Quality Table with Large Tables
tableone formats research tables directly. Because or any other summary — business report, slide, or README — Large Tables turns an empty DataFrame into a styled table, ready for presentation, in the same way gt package do in R. It takes a Pandas or Polars frame and renders it in HTML or an image.
Take the custom shortcut we created back in step 3 and dress it up:
from great_tables import GT, md
numeric = df.select_dtypes("number")
stats = (numeric.agg(["mean", "median", "std", "min", "max"]).T
.rename_axis("measurement").reset_index())
stats["missing_pct"] = df[numeric.columns].isna().mean().mul(100).values
table = (
GT(stats, rowname_col="measurement")
.tab_header(title="Penguin Body Measurements",
subtitle="Descriptive statistics, Palmer Archipelago")
.fmt_number(columns=["mean", "median", "std", "min", "max"], decimals=1)
.fmt_percent(columns="missing_pct", decimals=1, scale_values=False)
.data_color(columns="std", palette="Blues")
.tab_source_note(md("Source: *palmerpenguins* dataset (Horst et al.)."))
)Few things happen here. fmt_number again fmt_percent hold the display formatting so you don't have to rotate manually. data_color applies a color gradient to std column, which draws the eye to highly variable rates. tab_header again tab_source_note add a title and attribute that makes the table look complete. There's a lot more – column panels, conditional styling, even inline glossy lines – but even this produces something you can happily put in front of stakeholders.
To use the result, pass in the HTML string (works anywhere, no additional dependencies):
html = table.as_raw_html()
with open("summary_table.html", "w") as f:
f.write(html)
A stylish outlet called Great Tables
# Putting It Together: One Job, Every Time
The whole point of automation is repetition. Wrap the pipeline so that the following dataset is a single call:
from great_tables import GT, md
def descriptive_report(df, decimals=1):
numeric = df.select_dtypes("number")
stats = (numeric.agg(["count", "mean", "median", "std", "min", "max"]).T
.rename_axis("variable").reset_index())
stats["missing_%"] = df[numeric.columns].isna().mean().mul(100).values
return (
GT(stats, rowname_col="variable")
.tab_header(title="Descriptive Statistics",
subtitle=f"{len(df):,} rows x {df.shape[1]} columns")
.fmt_integer(columns="count")
.fmt_number(columns=["mean", "median", "std", "min", "max"], decimals=decimals)
.fmt_percent(columns="missing_%", decimals=1, scale_values=False)
.data_color(columns="std", palette="Blues")
.tab_source_note(md("Generated automatically with pandas + Great Tables."))
)
descriptive_report(df) # point it at any DataFrameThat's the difference between snippets and a tool: you write them once and reuse them for every dataset that sits on your desk.
# Wrapping up
Descriptive equations don't have to be a function that you reuse across projects. The ladder we ride has an article for every situation:
- Pandas
describe()again.agg(): Zero dependencies, perfect for quick tests and custom snapshots. - skimpy: A rich console summary with histograms and percentages of missing data, in one call.
- fg-data-profiling: A complete interactive HTML report if you need a full picture of experimental data analysis (EDA).
- table one: Sorted “Table 1” with p-values and SMDs of the research papers, sent in one line in LaTeX, HTML, and CSV.
- Main Tables: A polished, publication-quality style for any summary you've produced.
Choose a simple tool that answers your question. For a five-second check, describe() you win. With the manuscript, the tableone and the Great Tables achieved their own preservation. And once you wrap your favorite combination at work, you stop to do descriptive statistics and start running them – which is where you want to be so you can spend your time on analysis that really requires your brain.
Kanwal Mehreen is a machine learning engineer and technical writer with a deep passion for data science and the intersection of AI and medicine. He co-authored the ebook “Increasing Productivity with ChatGPT”. As a Google Generation Scholar 2022 for APAC, he strives for diversity and academic excellence. He has also been recognized as a Teradata Diversity in Tech Scholar, a Mitacs Globalink Research Scholar, and a Harvard WeCode Scholar. Kanwal is a passionate advocate for change, having founded FEMCodes to empower women in STEM fields.
