Visualising MUSE2 data as an OLAP cube

[ahawkes]

Is your feature request related to a problem? Please describe

MUSE has a lot of input and output data. It would be good to be able to see it easily.

Describe the solution you'd like

Let's say all results are stored in one file (e.g. an HDF), and a viewer is created to view the contents of that file. A typical view would be time on the horizontal axis, and e.g. process capacity on the vertical axis (for a set of selected processes). Another typical view would be aggregate commodity flow (from all processes) on the vertical (for a selected set of commodities).

User could select variables/granularity to display on each axis. e.g. on the horizontal axis user could select to view time slice level values and/or annual values. If they don't select time slice level, then it displays the aggregate of the time slices. Or another example, they could see capacity of a process type for the whole system, or could select to see it at regional level (where whole system level is aggregate of the regions, so the region dimension is being collapsed).

I think in general the horizontal axis would be time, though I guess it's not an absolute requirement. Also, some data only exists at annual level (e.g. capacity), which would need to be catered for.

Vertical axis could be virtually anything. Capacity, opex, capex, objective values, commodity flows etc etc etc.

I've heard a similar solution in VEDA 2.0 referred to as a "data cube".

Also a similar thing could be used to view input data. Once input data is read in, it would be good to be able to see it through such a viewer instead of trawling through a CSV? Also serves to reassure modellers about what data the model is actually using.

Describe alternatives you've considered

Output variables to CSV, perhaps across multiple files.

Additional context

Add any other context about the feature request here.

[alexdewar]

I've had a quick look into this and it seems the generic term is OLAP cube: https://en.wikipedia.org/wiki/OLAP_cube

There seem to be various open software tools out there for visualising data in this way, but it's probably worth looking into what data formats they support and making sure our data is in a form they can consume (or scripting something ourselves). Of course, if you have a solution that works for you (i.e. VEDA) that works too!

[ahawkes]

Thanks! Yes OLAP look about right. VEDA is not a solution here as it is not open source (indeed, it's quite expensive). What are the open software tool out there that can do this?

[dalonsoa]

There's another VEDA which is open source, but it might be very domain specific and possibly a bit of an overkill to setup.

[alexdewar]

@ahawkes There seem to be a bunch of Python-based solutions for this. E.g., this one caught my eye: https://pythonhosted.org/cubes/index.html

I haven't tested any of them though, so they might not be any good! But certainly worth a look.

[alexdewar]

I think this is more of an idea than an issue, so I'll convert to discussion.