From NetCDF to Data: A Practical Pipeline for Urban Climate Risk Analysis

research has shifted to handling large data sets. Large-scale Earth System Models (ESMs) and reanalysis products such as CMIP6 and ERA5 are no longer just repositories of scientific data but petabyte-sized spatial datasets that require extensive data engineering before they can be used for analysis.

From machine learning, to architecture data, the process of turning climate science into policy is like a classic pipeline: raw data acquisition, feature engineering, deterministic modeling, and production of the final product. However, unlike conventional machine learning from tabular data, computational climatology raises issues such as non-standard spatial temporal scales, non-specific climate limitations, and the obligation to maintain a more complex physical interpretation.

This article presents a lightweight and efficient pipeline that bridges the gap between raw climate data processing and impact modeling, converting NetCDF datasets into understandable, city-level hazard information.

Issue: From Raw Tensors to Decision-Ready Insight

While there has been an unprecedented release of high-resolution climate data around the world, converting it into location-specific and actionable information remains a non-trivial task. Most of the time, the problem is not that there is no data; it's a data format quirk.

Climate data is typically stored in the Network Common Data Form (NetCDF). These files:

• It consists of large multidimensional arrays (tensors usually with shape time × latitude × longitude × variables).
• A highly spatial mask, temporal integration, and alignment of the coordinate reference system (CRS) are required even before statistical analysis.
• They are not inherently understandable in the tabular structures (eg, SQL databases or Pandas DataFrames) commonly used by planners and economists.

This kind of disruption in the structure creates an interpretation gap: the original real data is there, but the socio-economic details, which should be taken decisively, are not.

Primary Data Sources

One of the characteristics of a strong pipeline is that it can combine traditional fundamentals with forward-looking assumptions:

• ERA5 Reanalysis: Brings past climate data (1991-2020) such as temperature and humidity
• CMIP6 Projections: Provides possible future climate scenarios based on various emission scenarios

With these data sources one can get the spatial availability of anomalies instead of relying only on global measurements.

Local Straight Lines: Defining Extreme Heat

An important issue in climate analysis is deciding how to define “extreme” conditions. A fixed global threshold (for example, 35°C) is not sufficient as local adaptation varies greatly from region to region.

Therefore, we characterize extreme heat with a threshold based on percentages obtained from historical data:

import numpy as np
import xarray as xr

def compute_local_threshold(tmax_series: xr.DataArray, percentile: int = 95) -> float:
    return np.percentile(tmax_series, percentile)

T_threshold = compute_local_threshold(Tmax_historical_baseline)

This approach ensures that extreme events are defined in accordance with local climate conditions, making the analysis more accurate and meaningful.

Thermodynamic Feature Engineering: Wet-Bulb Temperature

Temperature alone is not enough to accurately determine a person's heat stress. Humidity, which affects the body's evaporative cooling mechanism, is also a big factor. Wet-bulb temperature (WBT), which is a combination of temperature and humidity, is a good indicator of body stress. Here is the formula we use based on Stull's (2011) equation, which is easy and quick to calculate:

import numpy as np

def compute_wet_bulb_temperature(T: float, RH: float) -> float:
    wbt = (
        T * np.arctan(0.151977 * np.sqrt(RH + 8.313659))
        + np.arctan(T + RH)
        - np.arctan(RH - 1.676331)
        + 0.00391838 * RH**1.5 * np.arctan(0.023101 * RH)
        - 4.686035
    )
    return wbt

Wet bulb temperatures above 31–35°C approach the limits of human survival, making this an important factor in risk modeling.

Interpreting Climate Data for Human Impact

To move beyond physiological variables, we translate climate exposures into human impacts using a simple epidemiological framework.

def estimate_heat_mortality(population, base_death_rate, exposure_days, AF):
    return population * base_death_rate * exposure_days * AF

In this scenario, mortality is modeled as a function of population size, baseline mortality rate, duration of exposure, and the resulting risk factor.

Although simplified, this formulation enables the translation of adverse temperature conditions into interpretable impact metrics such as relative mortality.

Economic Impact Model

Climate change also affects economic productivity. Empirical research suggests a non-linear relationship between temperature and economic output, with productivity decreasing at higher temperatures.
We estimate this using a simple polynomial function:

def compute_economic_loss(temp_anomaly):
    return 0.0127 * (temp_anomaly - 13)**2

Although simplified, this captures the important insight that economic losses accelerate as temperatures drop in ideal conditions.

Case study: Comparing Climates

To illustrate the pipeline, we consider two different cities:

• Jacobabad (Pakistan): A city with extreme basic heat
• Yakutsk (Russia): A city with a cold climate

Despite using the same pipe, the results vary greatly based on the local climate. This highlights the importance of context-aware modeling.

Pipeline Architecture: From Data to Insight

The complete pipeline follows a structured workflow:

import xarray as xr
import numpy as np

ds = xr.open_dataset("cmip6_climate_data.nc")

tmax = ds["tasmax"].sel(lat=28.27, lon=68.43, method="nearest")

threshold = np.percentile(tmax.sel(time=slice("1991", "2020")), 95)

future_tmax = tmax.sel(time=slice("2030", "2050"))
heat_days_mask = future_tmax > threshold

This approach can be broken down into a series of steps that reflect a typical data science workflow. It starts with data import, which involves loading the raw NetCDF files into the computer setup. Next, spatial feature extraction is performed, where relevant variables such as maximum temperature are identified by spatial coordinates. The next step is a baseline calculation, using historical data to determine a percentage-based threshold that reflects worst-case scenarios.

When the baseline is fixed, anomaly detection detects future intervals when temperatures break the threshold, literally identifying thermal events. Finally, these observed events are transferred to impact models that turn them into comprehensible results such as accounts of death and economic damage.

When properly configured, this sequence of operations allows large meteorological datasets to be efficiently processed, turning complex multi-dimensional data into systematic and interpretable results.

Limitations and assumptions

Like any analysis pipeline, this one also depends on a set of simplifying assumptions, which should be considered while interpreting the results. Mortality rates are based on the assumption of vulnerability of the same population, which does not reflect differences in age distribution, social conditions or the availability of infrastructure such as cooling systems, etc. Economic impact assessment at the same time describes the worst case scenario and completely ignores the sensitivity of different sectors in certain sectors and operational strategies in a certain area. Additionally, there is inherent uncertainty in the climate projections themselves from the variety of climate model and future output scenarios. Finally, the spatial resolution of global datasets can mitigate the impact of local areas such as urban heat islands, thus causing an underestimation of the potential risk in densely populated urban areas.

Overall, these limitations point to the fact that the results of this pipeline should not be taken as they are as precise predictions but rather as experimental measurements that can provide a guiding understanding.

Important Ideas

This pipeline shows some important insights at the intersection of climate science and data science. First, the main difficulty in climate studies is not the modeling complexity but rather the massive data engineering effort required to process raw, high-dimensional data sets into usable formats. Second, the integration of multi-domain models to integrate climate data with epidemiological and economic frameworks often provides practical value, rather than just developing one component on its own. In addition, transparency and interpretability become key design principles, as a well-organized and easily traceable workflow allows for greater validation, trust, and acceptance among scholars and decision-makers.

The conclusion

Climate datasets are rich but complex. Unless planned pipelines are created, their value will remain hidden from decision makers.

Using data engineering principles and integrating domain-specific models, one can transform raw NetCDF data into actionable, city-level climate projections. The same approach serves as an illustration of how data science can be useful in bridging the gap between climate scientists and decision makers.

A simple implementation of this pipeline can be found here for reference:

References

• [1] Gasparrini A., Temperature-related mortality (2017), Lancet Planetary Health
• [2] Burke M., Temperature and economic productivity (2018), Nature
• [3] Stull R., Wet-bulb temperature (2011), Journal of Applied Meteorology
• [4] Hersbach H., ERA5 reanalysis (2020), ECMWF