DH-17242: PartitionedTable display widget does not render properly when there are non-key, non-constituent columns

[rcaudy]

Description

When a PartitionedTable is created that has extra columns (not just key columns and the constituent columns), the display widget fails to render the partition filter widget or any table data. A box at the bottom of the component informs the user that it is "Reloading partition".

Steps to reproduce

You can reproduce with this script in a Groovy console:

tt = timeTable("PT00:00:01") // Make a TimeTable
it = tt.update("A=ii%2==0", "B=k%10") // Add some columns
pt = it.partitionBy("A", "B") // Partition by those columns
ptt = pt.table() // Grab the underlying table
pttu = ptt.update("C=A ? B : -B") // Derive a new column
pt2 = io.deephaven.engine.table.PartitionedTableFactory.of(pttu, List.of("A","B"), true, "__CONSTITUENT__", pt.constituentDefinition(), false) // Make a new PartitionedTable with the same key columns

Another way to reproduce is with a partitionedAggBy:

pt = it.partitionedAggBy([AggLast("Last=Timestamp")], false, null, "A", "B")

In both cases an extra column in the underlying table seems to be confusing the UI. This is an important, permitted use case for PartitionedTables.

Expected results

I expect a partition filtering widget to show as normal. Ideally, though, we'd have some better way to display this, since showing the extra columns of the partitioned table would be valuable. I'm not certain that PartitionedTable and PartitionAwareSourceTable should be displayed in the same way.

Additional details and attachments

DH-17242

Versions

• Engine Version: latest
• Web UI Version: ...
• Java Version: ...
• Barrage Version: ...
• OS: ...
• Browser: ...
• Docker: ...

[dsmmcken]

Ideally, though, we'd have some better way to display this, since showing the extra columns of the partitioned table would be valuable. I'm not certain that PartitionedTable and PartitionAwareSourceTable should be displayed in the same way.

Can you elaborate on what you are thinking?

[vbabich]

There's a ticket for differentiating the UI for PartitionedTable and PartitionAwareSourceTable - #2049

[rcaudy]

Ideally, though, we'd have some better way to display this, since showing the extra columns of the partitioned table would be valuable. I'm not certain that PartitionedTable and PartitionAwareSourceTable should be displayed in the same way.

Can you elaborate on what you are thinking?

If I were designing a "perfect" PartitionedTable widget, I might want the following:

1. The underlying table() displayed by default. Color-coding or some other visual indicator identifying the key columns and the constituent columns.
2. Key column selectors and a merge() button available, as today, for selecting a single constituent to display or merging the entire partitioned table.
3. The ability to select a constituent to display from the table, rather than requiring the user to fiddle with the dropdowns.
4. The ability to filter or sort the underlying table being displayed (this might call for more of the PartitionedTable Java API to be exposed over gRPC).

[dsmmcken]

Same as example in ticket just as python, except for one line I am not sure on the equivalent

from deephaven import agg
from deephaven import time_table

tt = time_table("PT00:00:01") # Make a TimeTable
it = tt.update(["A=ii%2==0", "B=k%10"]) # Add some columns
pt = it.partition_by(["A", "B"]) # Partition by those columns
ptt = pt.table # Grab the underlying table

pttu = ptt.update("C=A ? B : -B") # Derive a new column
# unsure how make this python
# pt2 = io.deephaven.engine.table.PartitionedTableFactory.of(pttu, List.of("A","B"), true, "__CONSTITUENT__", pt.constituentDefinition(), false) // Make a new PartitionedTable with the same key columns

# aggregate it
ptagg = it.partitioned_agg_by(
    aggs=[agg.last(cols=["Last = Timestamp"])], by=["A","B"]
)