Challenge 11 - Interactive map-based dashboard for flood forecasting

[DiegoDomenici]

Stream 1 - Data Visualization

Mentors

• Juan Colonese
• James Varndell
• Dimitar Tasev

Skill Required

Frontend Visualization (Intermediate to Advanced)

• React/TypeScript, Deck.gl/MapLibre GL, Plotly.js, Jupyter ecosystem (ipyleaflet, ipywidgets, Voila)
• HTML/CSS/JavaScript fundamentals
• UX/UI design principles
• Docker containerization

Geospatial Technologies (Intermediate)

• Vector data formats (GeoJSON, Parquet, Shapefiles)
• Web mapping libraries and protocols
• Coordinate systems and projections

Python Development:

• FastAPI backend development
• Xarray/Zarr data manipulation
• GeoPandas or similar for spatial data processing
• Async/await patterns for performance

Goal

Develop a modern, responsive, and highly interactive web-based visualization dashboard that enables users (mostly modellers and analysts) to explore flood forecast data from Zarr datastores efficiently, with intuitive map-based navigation and comprehensive station-level analytics.

Key Outcomes:
• Enable interactive exploration of 10,000+ stations globally, with zoom-level specific visualisation features, like clustering, simpification, overviews and tiling can be used.
• Provide rich contextual visualizations combining forecast data, river network and catchments (vector tile visualisation), and alert thresholds (dynamic styling).
• Support multiple use cases/deployments: operational forecasting, retrospective analysis, and experiments comparisons.
• Modular and reusable visualisation components.
• Be ready for increased scalability and performance, but the user experience is more important in this project.

The challenge is focused on the visualisation and user-facing application but depending how the implementation of the solution progresses some developments may be required in the data provision infra-structure. Therefore, close collaboration and communication with the mentors will be key to achieve the desired outcome.

Description of the Challenge

Current problem/limitation
The existing Voila-based interface, while functional, has significant UX limitations:

• Slow response times: Kernel restarts and notebook cell execution create delays
• Basic interactions: Limited to simple click events on map markers
• Rigid layout: Difficult to customise views or compare multiple stations
• The current system focuses primarily on point data (stations)
• River networks and catchment boundaries are not visualised
• No spatial context for understanding upstream/downstream relationships
• No animation or time-stepping through forecasts
• Comparisons between multiple forecast runs are only possible for one station at a time.

Data/system
Forecasts and some other initial datasets will be served from an existing REST API connected to a zarr datastore. Alternatively, a serverless bucket-like datastore may be provided. Some geospatial data may be served via public WMS services.

Proposed solution
The proposed solution will be a modern and responsive web-based data visualisation app, accessible only for internal users (authentication or user handling should not be part of the solution), for flood forecasting model results.

Evaluation criteria

• Feasibility
• Easy to maintain / Future-proof approach
• Transferability
• Comprehensibility

[geacomputing]

Hi there,

thank you for sharing this opportunity: it looks very interesting, and I am seriously considering applying.
I’ve put together a list of technical questions below to better understand the scope and requirements.
Thanks in advance for your support.

Technical Questions for ECMWF Mentors — Challenge 11 (Flood Dashboard)

1. Preferred framework or platform

Is there a preferred or recommended framework stack for the project?
I am considering using Django + GeoDjango for the backend because this simplifies geospatial handling and offers built-in support for modular endpoints, multilingual interfaces, and role-based access (even if auth is out of scope).
Is Django acceptable, or is ECMWF expecting FastAPI to be used exclusively?

---

2. Backend resources during development

To run the prototype efficiently, the project will require compute and storage resources (database space, geospatial layers, Zarr access, etc.).
Will ECMWF provide:

• a development API environment,
• a server/VM,
• or backend storage resources?

Alternatively, would it be acceptable to host the prototype during development on a developer sandbox platform such as Heroku, Fly.io, or similar, with the intention of later migrating it to ECMWF infrastructure?
Or should everything be kept strictly local until a later integration phase?

---

3. Asynchronous processing and task orchestration

Given that tasks such as extracting subsets from Zarr, generating tiles, or computing upstream/downstream relationships can be computationally heavy, would ECMWF support the use of an asynchronous task queue (e.g., Celery, RQ, or similar)?
This would allow the application to spin up a temporary “active session” where a worker handles:

• data extraction,
• filtering,
• map layer preparation (e.g., Deck.gl),
• and multi-run comparisons.

Is this approach aligned with expectations, or should the prototype remain synchronous for now?

---

4. Data samples and data model/architecture

Could ECMWF provide example datasets (or documentation) for:

• river network layers,
• catchment boundaries,
• station metadata,
• and any hierarchical encoding (e.g., Pfafstetter indices)?

Clarifications needed:

• Does the river network dataset include Pfafstetter hierarchy or an equivalent connectivity scheme?
• What formats are used (GeoJSON, shapefiles, Parquet, vector tiles, WMS, etc.)?
• What is the approximate size and spatial resolution of these datasets?
• Are these layers delivered through the REST/Zarr API, or via external WMS/WFS services?

This information is essential to design upstream/downstream tracing and station contextualization.

---

5. Expected “on-the-fly analytics”

Could you clarify what types of real-time analytics the dashboard is expected to support?
For example:

• forecast summary statistics (RMSE, bias, exceedance probability),
• comparison between model runs,
• threshold exceedance detection,
• animated time-step exploration,
• hydrological indicators (lag, rise time, cumulative flow).

A minimal required list would help define both the API endpoints and the client-side rendering strategy.

Thanks again for any input you might be willing to give.
Kind regards from sunny Cyprus
Marco

[colonesej]

Dear @geacomputing, thank you for your interest in this challenge. I'll try to address your questions below, but this challenge has an exploratory feel, so feel free to include your ideas of how to better tackle the main goal.

1. We are not attached to a particular framework, but we are looking for a lightweight application and a focus in the user-facing side of it. Of course, the data handling and the processing of some data can affect the overall user experience. So, some development may be required in the data provision, backend and APIs, even though these should not be the focus. We will keep that in mind when assessing the proposals and outcomes.
2. If needed, we will provide access to existing data or infra-structure to test and deploy your solution. This will probably be self-hosted in the European Weather Cloud. Your requirements should be part of your proposal.
3. You are right that some data processing may be computationally heavy, but we do not expect heavy loads in the application, so performance is not a priority at the moment. With that said, any component that you feel is important for your proposal, if based in open-source software, can be provided.
4. This is an example of the river network and catchments dataset that we will want to integrate https://zenodo.org/records/7629908. There are available WMS services for it, but a local copy may be needed for a better experience and customisation in the application. Station metadata are simple tabular data; could be served as GeoJSON or parquet or any other fitting format. Although it can be derived, I'm not sure about topological information at the station level. There are other ways to compute what needs to be computed without it (mainly catchment area boundaries). You can assume this will be made available to you and no heavy river network-based computation should be expected in this application.
5. You can expect them to be dataset that you can retrieve somehow. This project should focus in how to visualise and interact with such dataset. For instance, when we say "model comparison" is about selecting two model "versions" and visualising them side by side, maybe with synched map viewers/windows. "model version" would be a different parameter to an API request that will return some geoJSON of points or a timeseries for a location. Our datasets for this application are simple at the moment, indexed point information mainly.

Hope this clarifies your questions and looking forward to hearing from you again.

[JuanGuerrero09]

Dear @colonesej,

I’m really interested in this challenge, especially because I’ve previously worked with React and Leaflet to build a map-based application for a tourism app in Madrid, and I’ve been using FastAPI for some of my university projects lately.

I have a few questions about the project’s direction. First, will the final outcomes be open-source, or is this intended strictly for internal usage? Regarding the spatial scale, I’m curious if the dashboard is expected to visualize actual inundation extents within the catchments, or if the focus remains primarily on the station points and the river network.

Also, since there is an existing REST API, do you expect us to build directly onto that infrastructure, or should we implement a new middleware API from scratch to handle the communication between the dashboard and the Zarr datastore? Finally, given the requirement for 10,000+ stations, what is the preferred approach for the global view—like vector tiling or clustering—to ensure the page doesn’t crash from the rendering load but is still fast enough to handle all the data requests?