feat: Add support for indices and values `Series` in `Series.scatter` by FBruzzesi · Pull Request #3444 · narwhals-dev/narwhals

FBruzzesi · 2026-02-02T18:59:25Z

Description

@MarcoGorelli this might pop up in sklearn/skrub 😂

What type of PR is this? (check all applicable)

Related issues

Closes enh: scatter should accept narwhals.Series for indices #2155
Arrow refactor drive by feat(expr-ir): Add Series.scatter #3305

Checklist

Code follows style guide (ruff)
Tests added
Documented the changes

dangotbanned · 2026-02-02T19:05:55Z

narwhals/_arrow/series.py

-        # - Missing return type
-        pc_replace_with_mask: Incomplete = pc.replace_with_mask
-        return self._with_native(pc_replace_with_mask(self.native, mask, values_native))
+    def scatter(self, indices: Self, values: Self) -> Self:


This is very satisfying for (#3305) to finally pay off 🙏

Thought it might be worth splitting into a separate PR, since it would be easy to upstream to main and (possibly) avoid a numpy import

dangotbanned · 2026-02-02T19:14:18Z

@FBruzzesi would you be open to me porting over the test suite as well?

I'd need to make some tweaks, but I think I could get the diff negative 😄

(no worries if not)

FBruzzesi · 2026-02-02T19:39:39Z

Sure @dangotbanned - I will first try to get the CI green again 🥦 🥦

Fixing CI is really the broccoli's to all the currently baking cakes 🍰

#3444 (comment)

dangotbanned · 2026-02-03T13:09:52Z

tests/series_only/scatter_test.py

+@pytest.mark.filterwarnings(
+    "ignore:.*all arguments of to_dict except for the argument:FutureWarning"
+)


modin triggers this warning because it passes orient=list as positional.

So I copied over the filter from test_to_dict:

narwhals/tests/frame/to_dict_test.py

Lines 9 to 12 in 1a1d323

@pytest.mark.filterwarnings(

"ignore:.*all arguments of to_dict except for the argument:FutureWarning"

)

def test_to_dict(constructor_eager: ConstructorEager) -> None:

But I think we should either:

Handle this in narwhals

Add it to tool.pytest.ini_options.filterwarnings

dangotbanned · 2026-02-03T13:14:53Z

tests/series_only/scatter_test.py

+    assert_equal_data(df, unchanged_indexed)
+
+
+def test_scatter_pandas_index() -> None:


I renamed this to make it clearer, since indices is a parameter name for scatter.

Although, I'd prefer if we could use a less direct way of testing index preservation.
E.g. if this is important, why only pandas and not all the derivatives?

narwhals/series.py

narwhals/_polars/series.py

FBruzzesi · 2026-02-03T14:06:45Z

narwhals/series.py

+        compliant_indices = self._extract_native(indices)
+        compliant_values = self._extract_native(values)


PR unrelated: Should we rename _extract_native to _extract_compliant?

Lol, yeah I don't know when that slipped through 😂

@FBruzzesi I thought I remembered discussing this!

#2886 (comment)

Weirdly, the same thing has the wrong name on nw.Series 🤔

narwhals/narwhals/series.py

Lines 376 to 381 in 0e80656

def _extract_native(self, arg: Any) -> Any:

from narwhals.series import Series

if isinstance(arg, Series):

return arg._compliant_series

return arg

@dangotbanned which algorithm do you use to index all past comments and conversations in your brain? I am jealous 😂

@dangotbanned which algorithm do you use to index all past comments and conversations in your brain? I am jealous 😂

Deju vu + obsessive thoughts 😭

PR unrelated: Should we rename _extract_native to _extract_compliant?

@MarcoGorelli FYI, I will forget 😂

FBruzzesi · 2026-02-05T09:25:53Z

@dangotbanned let me know if you have time and manage to replicate the environment for which the tests are failing.

I am on an Apple Silicon (macOS ARM64), and the old versions of pandas (1.1.3) and numpy (1.19.3) don't have pre-built wheels for this architecture. When pip/uv tries to build them from source, it fails 😭

Update: solved in 6b23fdc

Might be nice to factor out scatter and _scatter_in_place for pandas before merging.

Update 2: b714afb

dangotbanned

Thanks @FBruzzesi!

I have a few suggestions, but overall just very happy to slowly creep #2572 into main 😉

narwhals/_pandas_like/series.py

narwhals/series.py

narwhals/_arrow/series.py

FBruzzesi · 2026-02-19T10:57:22Z

@dangotbanned if you are happy (or I would say happier since the last approval in #3444 (review)) with this, I will go ahead and merge

dangotbanned

Double approve

MarcoGorelli · 2026-02-20T12:09:54Z

thanks all, solid!

FBruzzesi added 2 commits February 1, 2026 18:52

WIP

d846808

simplify via Dan's suggestion

e75cfe6

FBruzzesi added the enhancement New feature or request label Feb 2, 2026

dangotbanned reviewed Feb 2, 2026

View reviewed changes

test: Port updated tests from (#2572)

1a1d323

#3444 (comment)

dangotbanned reviewed Feb 3, 2026

View reviewed changes

narwhals/series.py Outdated Show resolved Hide resolved

dangotbanned reviewed Feb 3, 2026

View reviewed changes

narwhals/_polars/series.py Outdated Show resolved Hide resolved

dangotbanned added tests eager-only labels Feb 3, 2026

FBruzzesi commented Feb 3, 2026

View reviewed changes

FBruzzesi and others added 2 commits February 3, 2026 15:36

aplly Dan's suggestions

8ff3927

Merge branch 'main' into feat/scatter-support-series

6319095

FBruzzesi added 3 commits February 5, 2026 10:40

try fix old pandas

6b23fdc

refactor pandas-like scatter and _scatter_in_place

b714afb

rm _scatter_in_place in favor of _scatter(..., in_place=True)

509ff1c

FBruzzesi requested a review from dangotbanned February 6, 2026 22:41

dangotbanned self-assigned this Feb 7, 2026

Merge branch 'main' into feat/scatter-support-series

129ca3e

dangotbanned removed their assignment Feb 7, 2026

dangotbanned approved these changes Feb 7, 2026

View reviewed changes

dangotbanned and others added 5 commits February 7, 2026 14:06

Merge branch 'main' into feat/scatter-support-series

117acc8

simplify top level and pandas implementations

2d19bff

use to_numpy instead

5914403

Merge branch 'main' into feat/scatter-support-series

0a593b6

Merge branch 'main' into feat/scatter-support-series

f4e78f5

apply Dan's suggestion for pandas

4c6dcd3

Merge branch 'main' into feat/scatter-support-series

6a7fba8

dangotbanned approved these changes Feb 20, 2026

View reviewed changes

dangotbanned merged commit 349ce10 into main Feb 20, 2026
36 checks passed

dangotbanned deleted the feat/scatter-support-series branch February 20, 2026 10:43

	@pytest.mark.filterwarnings(
	"ignore:.*all arguments of to_dict except for the argument:FutureWarning"
	)
	def test_to_dict(constructor_eager: ConstructorEager) -> None:

		assert_equal_data(df, unchanged_indexed)


		def test_scatter_pandas_index() -> None:

		compliant_indices = self._extract_native(indices)
		compliant_values = self._extract_native(values)

	def _extract_native(self, arg: Any) -> Any:
	from narwhals.series import Series

	if isinstance(arg, Series):
	return arg._compliant_series
	return arg

Conversation

FBruzzesi commented Feb 2, 2026 • edited by dangotbanned Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

What type of PR is this? (check all applicable)

Related issues

Checklist

Uh oh!

dangotbanned Feb 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

dangotbanned commented Feb 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

FBruzzesi commented Feb 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

FBruzzesi commented Feb 5, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

dangotbanned left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

FBruzzesi commented Feb 19, 2026

Uh oh!

dangotbanned left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

MarcoGorelli commented Feb 20, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

FBruzzesi commented Feb 2, 2026 •

edited by dangotbanned

Loading

dangotbanned Feb 2, 2026 •

edited

Loading

dangotbanned commented Feb 2, 2026 •

edited

Loading

FBruzzesi commented Feb 2, 2026 •

edited

Loading

FBruzzesi commented Feb 5, 2026 •

edited

Loading