bug: `when` combines multi-output expressions (but shouldn't)

[MarcoGorelli]

df = pd.DataFrame({'a': [1, 2, 3], 'b': [0,1,6]})
print(nw.from_native(df).select(nw.when(nw.col('a', 'b')>1).then(999).otherwise(666)))

df = pl.DataFrame({'a': [1, 2, 3], 'b': [0,1,6]})
print(df.select(pl.when(pl.col('a', 'b')>1).then(999).otherwise(666)))

Narwhals returns

┌──────────────────┐
|Narwhals DataFrame|
|------------------|
|       literal    |
|    0      666    |
|    1      666    |
|    2      999    |
└──────────────────┘

but Polars expands the predicate and ends up with

polars.exceptions.DuplicateError: the name 'literal' is duplicate

It's possible that multiple expressions are returning the same default column name. If this is the case, try renaming the columns with `.alias("new_name")` to avoid duplicate column names.

As a consequence, Polars allows

pl.when(pl.all()>1).then(pl.all())

but we don't

[dangotbanned]

I have a slightly tweaked example to show an issue this expanding can cause (pola-rs/polars#25022)

import polars as pl

data = {"a": [False, True, True], "b": [False, False, True], "c": [False, True, False]}
df = pl.DataFrame(data)

df.select(pl.when(pl.col("a", "b", "c")).then(pl.col("a", "b", "c")))

shape: (3, 3)
┌──────┬──────┬──────┐
│ a    ┆ b    ┆ c    │
│ ---  ┆ ---  ┆ ---  │
│ bool ┆ bool ┆ bool │
╞══════╪══════╪══════╡
│ null ┆ null ┆ null │
│ true ┆ null ┆ true │
│ true ┆ true ┆ null │
└──────┴──────┴──────┘

df.select(pl.when("a", "b", "c").then(pl.col("a", "b", "c")))

shape: (3, 3)
┌──────┬──────┬──────┐
│ a    ┆ b    ┆ c    │
│ ---  ┆ ---  ┆ ---  │
│ bool ┆ bool ┆ bool │
╞══════╪══════╪══════╡
│ null ┆ null ┆ null │
│ null ┆ null ┆ null │
│ null ┆ null ┆ null │
└──────┴──────┴──────┘

I understand (now 😅) why these produce different results, but the behavior still feels surprising 🤔

Just thought I'd give a heads-up, since I'm trying to decide on what parts of the expansion rules to adopt in (#3233)

[MarcoGorelli]

thanks - i'm not convinced it's worth the hassle of changing this from what we've got

[dangotbanned]

thanks - i'm not convinced it's worth the hassle of changing this from what we've got

Agreed 100%, I've just finished ripping that out of (#3233) after frying my brain 😂

On the bright side, it means we can do things that polars can't 😏

import polars as pl
import polars.selectors as cs

data = {
    "a": ["a", "a", "b", "b", "b"],
    "b": [1, 2, 3, 5, 3],
    "c": [5, 4, 3, 2, 1],
    "d": [9, 1, 25, 2, 1],
    "idx1": [0, 1, 1, 0, 1],
    "idx2": [2, 2, 0, 3, 4],
}
df = pl.DataFrame(data)

df.select(
    cs.by_name("a", "d")
    .first()
    .over(cs.by_index(range(1, 4)), order_by=cs.matches(r"idx"))
)

InvalidOperationError: cannot combine selectors that produce a different number of columns (3 != 2)

But since we aren't trying to do that kind of expansion, this works 🥳:

import narwhals as nw
from narwhals._plan import selectors as ncs  # noqa: F811

schema = nw.Schema.from_polars(df.schema)

project_s(
    ncs.by_name("a", "d")
    .first()
    .over(ncs.by_index(range(1, 4)), order_by=ncs.matches(r"idx")),
    schema=schema,
)

(a=col('a').first().over(partition_by=[col('b'), col('c'), col('d')], order_by=[col('idx1'), col('idx2')]),
 d=col('d').first().over(partition_by=[col('b'), col('c'), col('d')], order_by=[col('idx1'), col('idx2')]))

This shouldn't be related to when IMO, but the same underlying expansion rule is applied to both