fix: Align division by zero behavior across all backends

[FBruzzesi]

What type of PR is this? (check all applicable)

• 💾 Refactor
• ✨ Feature
• 🐛 Bug Fix
• 🔧 Optimization
• 📝 Documentation
• ✅ Test
• 🐳 Other

Related issues

• Related [Enh]: Align floordiv behavior across different backends #2748 : solved all cases but pandas-like

Checklist

• Code follows style guide (ruff)
• Tests added
• Documented the changes

[FBruzzesi]

@MarcoGorelli what's the current status of pandas-like broadcasting?

I am having a very bad time having to distinguish between python scalar vs nw.lit(value) and deal with all the situations that might arise.

As an example, we would enter __floordiv__ in all the following cases:

frame.<with_columns|select>(
    x1 = nw.lit(value) / scalar,
    x2 = nw.lit(value) / nw.lit(another_value),
    x3 = nw.lit(value) / nw.col(name),
    x4 = nw.col(name) / scalar,
    x5 = nw.col(name) / nw.lit(value),
    x6 = nw.col(name) / nw.col(another_name),
)

and they need special attentions 😂

On top of that, there is specific logic given the dtype backend 🙈

[FBruzzesi]

@MarcoGorelli I rolled backed the "not-fully functional" idea I had for pandas.
This PR would end up aligning the behavior for pyarrow and the lazy backends, which I still think it's a good achievement.

I do see tests failing for "not so old" polars due to PanicExpection's and different results of __rfloordiv__. I need to check until which version the issue exists (and either fix it or xfail/skip it).

Finally, for the missing coverage, the #2761 (comment) above is relevant, and I would like to make sure that I am not missing anything

[MarcoGorelli]

thanks - will take a closer look, i'd like to be quite careful with this one 😄

[MarcoGorelli]

thanks @FBruzzesi !

the pandas implementation looks a bit more complex than i was hoping for, will take a look

but we could start with aligning at least the other backends if that's simpler

[MarcoGorelli]

shall we just add a comment noting what we've changed from pandas' pyarrow impl?

[MarcoGorelli]

gentle ping

[MarcoGorelli]

🤔 this looks right, but imma have to think about it, not sure about repeating the broadcasting logic here..

[MarcoGorelli]

the implementation looks quite similar between duckdb and pyspark, wonder if the implementation can be shared here?

[FBruzzesi]

Disclaimer: I didn't invest more than 3 minutes looking at this

---

Yes that's true, but there is a limitation: the new abstracted self._when does not allow for chaining with .otherwise. Understandably so, for ibis it would be complex since it's inside the ibis.cases rather than a chained ops

[MarcoGorelli]

it should be possible by putting statemnts within each other, like here

narwhals/narwhals/_sql/expr.py

Lines 515 to 528 in c22b582

	def log(self, base: float) -> Self:
	def _log(expr: NativeExprT) -> NativeExprT:
	F = self._function
	return self._when(
	expr < self._lit(0),
	self._lit(float("nan")),
	self._when(
	expr == self._lit(0),
	self._lit(float("-inf")),
	op.truediv(F("log", expr), F("log", self._lit(base))),
	),
	)
	return self._with_elementwise(_log)

[FBruzzesi]

With the latest changes in when, I am able to just write:

def __floordiv__(self, other: Self) -> Self:
    
    def func(expr: NativeExprT, other: NativeExprT) -> NativeExprT:
        return self._when(other != self._lit(0), expr // other, self._lit(None))
    
    return self._with_binary(func, other=other)

def __rfloordiv__(self, other: Self) -> Self:
    
    def func(expr: NativeExprT, other: NativeExprT) -> NativeExprT:
        return self._when(expr != self._lit(0), other // expr, self._lit(None))
     
    return self._with_binary(func, other=other).alias("literal")

at the _sql/ level. This would work for ibis and duckdb, but not for pyspark since we have a different dedicated implementation.

However, this results in all sort of typing issues. At sql level says that NativeExprT does not implement floordiv and rfloordiv. If I go add them these methods than it's even worse.

Honestly it's a 10 lines difference, but we should probably start doing some workaround for the typing.. Personally I don't think it's worth it

[MarcoGorelli]

You can use op.floordiv / op.truediv in _sql/expr, that'll resolve the typing issues

Personally I'd have preferred to implement arithmetic ops for NativeExpr and ignore Ibis' half-baked typing, but @dangotbanned seemed quite keen on typing Ibis, which seems fair enough

[MarcoGorelli]

thanks for updating! generally on-board, though still hoping for a simpler dask way, will try something out

[MarcoGorelli]

one comment

for Dask, can we just use

other_series = maybe_evaluate_expr(df, other)

instead of lines 238-244?

[MarcoGorelli]

gentle ping

[MarcoGorelli]

this is already a step forwards, we can think about pandas-like later. but i hope nobody is actually floor-dividing by zero 😄

thanks @FBruzzesi !