Day 2 – Climate Data

7. Responsible AI

Day 1 · Responsible and Inclusive Use

Responsible AI

Explore how AI systems can be designed and used in ways that are scientifically credible, transparent, inclusive, secure, accountable and relevant to African weather and climate services.

Learning objectives

By the end of this chapter, you should be able to:

Explain why responsible AI is important in weather and climate services.
Identify principles of transparency, fairness, inclusion and accountability.
Recognise risks related to data ownership, privacy and external dependence.
Explain why human oversight remains essential.
Use a responsible AI checklist to assess a proposed application.

Responsible AI begins before model development

Responsible AI is not only about checking a finished model. It begins with deciding which problem should be addressed, who should participate, what data may be used, how performance will be evaluated and who will remain accountable for decisions.

Principles of responsible AI

Local relevance The system should address a clearly defined operational or community need rather than introducing technology without a practical purpose.

Transparency Data sources, assumptions, methods, limitations and intended uses should be documented and communicated clearly.

Fairness Performance should be examined across locations, communities and user groups so that benefits and risks are not distributed unfairly.

Inclusion Forecasters, data providers, decision-makers and affected communities should participate in defining needs and evaluating outputs.

Data stewardship Data ownership, access rights, privacy, quality and appropriate reuse should be respected.

Accountability Clear responsibilities should be established for model approval, operational decisions, warning communication and corrective action.

Reliability and safety The system should be tested under normal and extreme conditions, with procedures for identifying failures and preventing harmful use.

Capacity and sustainability Local institutions should be able to understand, operate, maintain and adapt the system over time.

Why human oversight remains essential

AI models learn from historical data and may fail during unusual events, changing conditions or situations that were poorly represented during training. Human experts provide physical understanding, local knowledge and awareness of operational consequences.

Review AI output alongside observations and numerical forecasts.
Identify physically unrealistic or inconsistent predictions.
Consider local conditions not represented in the model.
Interpret uncertainty and conflicting guidance.
Decide whether and how warnings should be issued.
Remain accountable for operational communication and decisions.

Explainability and communication

Users do not always need to understand every mathematical detail of a model, but they should understand what information was used, what the output means, how reliable it is and when it should not be trusted.

Forecast products should communicate uncertainty, known limitations and the role of expert review rather than presenting predictions as guaranteed outcomes.

Data ownership and stewardship

Who owns and controls the data?

National meteorological services, communities and partner organisations may have different ownership and access conditions. These conditions should be agreed before data are shared or reused.

Is personal or sensitive information involved?

Impact-based services may use population, health, mobility or vulnerability information. Appropriate safeguards and access controls may be required.

Is the data suitable for the intended purpose?

Data collected for one purpose may not be appropriate for another. Quality, coverage, limitations and permitted uses should be assessed.

Will local institutions retain access?

Projects should avoid arrangements in which local institutions provide data but cannot access, evaluate or benefit from the resulting models and products.

Risk of external dependence

A system may appear successful during a funded project but become unusable when external technical support, cloud computing or software access ends.

Sustainability planning should include documentation, source-code access, training, maintenance responsibilities, computing costs, data continuity and procedures for updating the model.

Responsible AI checklist

Is the operational problem clearly defined?
Have intended users participated in defining the system?
Are the data sources, quality and ownership documented?
Has performance been tested across different locations and conditions?
Are uncertainty and limitations communicated clearly?
Can affected users question or challenge the system’s outputs?
Can the institution operate and maintain the system locally?
Is there a procedure for detecting and correcting failures?
Is a qualified person responsible for the final decision?

Scenario: an externally developed flood-warning system

An external organisation develops an AI flood-warning system. The national meteorological service cannot inspect the training data, modify the model or operate it without the external provider.

What concerns should be discussed before operational use?

Transparency of the model and training data.
Independent validation under local conditions.
Ownership and access to data and model outputs.
Long-term dependence on the external provider.
Maintenance, continuity and operating costs.
Responsibility when an alert is incorrect or missed.
Whether forecasters can review and override the output.

Check your understanding

Which responsible AI principle is involved?

Model performance is reported separately for well-observed and data-sparse regions.

This supports fairness, transparency and geographical evaluation.

Forecasters can reject an output that conflicts with observations and physical understanding.

This is human oversight and accountability.

Communities participate in deciding how heat-warning categories should be communicated.

This supports inclusion, local relevance and participatory design.

The project documents who owns the observations and who may reuse them.

This is data stewardship and governance.

Institutional reflection

Consider one AI application that could be introduced in your institution. Which responsible AI principle would require the most attention?

Think about local relevance, data ownership, transparency, fairness, inclusion, capacity, sustainability and human accountability.