4. The Machine-Learning Workflow

Day 1 · From Problem to Prediction

The Machine-Learning Workflow

Follow the main stages used to design, train, evaluate and operate a machine-learning system for weather and climate applications.

Learning objectives

By the end of this chapter, you should be able to:

  • Describe the main stages of a machine-learning workflow.
  • Connect an operational problem with appropriate data and model outputs.
  • Explain why data preparation and independent evaluation are essential.
  • Recognise common workflow risks, including data leakage and overfitting.
  • Explain why operational monitoring must continue after deployment.

Start with the problem, not the model

A successful machine-learning project begins with a clearly defined weather or climate problem, the people who will use the information, and the decision that the system is expected to support. The model should be selected only after these requirements are understood.

The six-stage workflow

1
Define the problem Identify the prediction target, intended users, required lead time, geographical scale and operational decision.
2
Collect the data Obtain relevant observations, satellite products, reanalysis, forecast data, climate-model output and supporting information.
3
Prepare the data Check quality, handle missing values, align dates and locations, standardise units and create model-ready inputs.
4
Train the model Use historical examples to estimate relationships between the inputs and the target variable.
5
Evaluate the model Compare predictions with observations using appropriate metrics, independent data and relevant baseline methods.
6
Deploy and monitor Integrate the model into an operational workflow and continue checking its reliability, data inputs and performance.

Explore each workflow stage

1. Define the operational problem

Clearly state what should be predicted and how the prediction will be used. A vague goal such as “use AI to improve forecasting” is not sufficient.

A stronger definition would be: “predict whether daily maximum temperature will exceed 40°C at a specific location five days in advance to support heat-preparedness decisions.”

2. Identify and collect the required data

The required data depend on the problem. Possible sources include station observations, satellite products, reanalysis, numerical forecasts, climate-model output, elevation and calendar information.

Data should cover a sufficiently long period and represent the locations, seasons and extreme events where the model will be used.

3. Prepare and explore the data

Data preparation may include checking missing values, removing or flagging invalid observations, converting units, matching forecast lead times and aligning grids with station locations.

Basic exploration should examine distributions, seasonal patterns, extreme values, relationships between variables and geographical differences.

4. Select and train the model

Select a model that is appropriate for the prediction task, data volume, available computing resources and required interpretability.

Begin with a simple baseline before testing more complex methods. Model complexity should be justified by meaningful improvements in performance or operational value.

5. Evaluate performance

Evaluation should use data that were not used for model training. The model should be compared with observations and with relevant baseline forecasts.

Evaluation should consider average performance, extreme events, geographical differences, uncertainty and whether improvements are useful to operational users.

6. Deploy, communicate and monitor

Deployment involves more than saving a trained model. The system needs reliable data feeds, automated processing, quality checks, documentation and clear responsibilities.

Performance should be monitored because data sources, instruments, forecasting systems, user needs and climate conditions can change.

Worked example: improving a temperature forecast

  1. Define: Improve daily maximum-temperature forecasts for a local station at a five-day lead time.
  2. Collect: Obtain historical model forecasts, station observations, lead-time information and seasonal variables.
  3. Prepare: Match forecasts and observations by date, location and valid time; check units and missing values.
  4. Train: Fit a model using historical forecast inputs and observed maximum temperature.
  5. Evaluate: Compare corrected and raw forecasts on dates not used during training.
  6. Operate: Generate daily corrected forecasts and monitor performance, particularly during extreme heat.

Common workflow risks

Poor problem definition The model output may not support a real user decision.
Data leakage Information from the test period accidentally enters model training.
Overfitting The model memorises training data but performs poorly on new cases.
Unrepresentative data Important locations, seasons or extreme events are missing.
Inappropriate metrics Evaluation does not reflect the intended operational use.
No monitoring Changes in inputs or performance remain undetected after deployment.

Check your understanding

Which workflow stage is involved?

Converting rainfall measurements from millimetres to consistent units

This belongs to data preparation.

Deciding that the system must predict dangerous heat five days ahead

This belongs to problem definition.

Comparing predictions with station observations from an unseen year

This belongs to model evaluation.

Checking whether a daily satellite-data feed has stopped updating

This belongs to deployment and operational monitoring.

Institutional reflection

Think about a possible AI application in your institution. Which workflow stage would be the most difficult to complete successfully?

Consider data availability, technical skills, computing infrastructure, evaluation, operational integration and long-term maintenance.