feat: Adds private Namespace class

[dangotbanned]

What type of PR is this? (check all applicable)

• 💾 Refactor
• ✨ Feature
• 🐛 Bug Fix
• 🔧 Optimization
• 📝 Documentation
• ✅ Test
• 🐳 Other

Related issues

• Mentioned in (refactor: *(Namespace|DataFrame).from_numpy #2283)
• Related
  • chore: Add Compliant*.from_native #2315
  • refactor: Add CompliantDataFrame.from_dict #2304
  • API: io functions for v2 #2116

Checklist

• Code follows style guide (ruff)
• Tests added
• Documented the changes

If you have comments or can explain your changes, please do so below

Had this idea nagging away at me while working on (#2116).

Super high-level

This PR adds a Namespace class, that can be created via either:

Namespace.from_backend(...)
Namespace.from_native_object(...)

Supporting these new @classmethods is a new method for Implementation:

Implementation._backend_version()

Notes

I've tried to keep things isolated from the rest of narwhals - for now - but the goal would be replacing similar logic from all of:

• functions.py
• translate.py
• utils.py

Important

Not planning to make this a public class

Example

This further simplifies (#2283) to a one-liner, with the added bonus of typing from a backend string

[dangotbanned]

This could be used in place of most calls we currently do to parse_version.

Here we'd only need Implementation and use the same property for all of them.

• is_sqlframe_dataframe
• SparkLikeLazyFrame
• sqlframe._version

narwhals/narwhals/translate.py

Lines 755 to 775 in 4d2b9d5

	elif is_sqlframe_dataframe(native_object): # pragma: no cover
	from narwhals._spark_like.dataframe import SparkLikeLazyFrame
	if series_only:
	msg = "Cannot only use `series_only` with SQLFrame DataFrame"
	raise TypeError(msg)
	if eager_only or eager_or_interchange_only:
	msg = "Cannot only use `eager_only` or `eager_or_interchange_only` with SQLFrame DataFrame"
	raise TypeError(msg)
	import sqlframe._version
	backend_version = parse_version(sqlframe._version)
	return LazyFrame(
	SparkLikeLazyFrame(
	native_object,
	backend_version=backend_version,
	version=version,
	implementation=Implementation.SQLFRAME,
	),
	level="lazy",
	)

If you then combine this PR with (#2315), we can do:

from __future__ import annotations

from typing import cast

from narwhals._namespace import Namespace
import narwhals as nw

if TYPE_CHECKING:
    from narwhals._spark_like.dataframe import SQLFrameDataFrame
    
native_object = cast("SQLFrameDataFrame", "pretend im a sqlframe")
nw.LazyFrame(
    Namespace.from_native_object(native_object).compliant.from_native(native_object),
    level="lazy",
)

[dangotbanned]

For the last nw.LazyFrame part, we could even add something like this to Compliant* classes:

ToNarwhals Protocol

from __future__ import annotations

from typing import Any
from typing import Protocol
from typing import TypeVar

ToNarwhalsT_co = TypeVar("ToNarwhalsT_co", covariant=True)

class ToNarwhals(Protocol[ToNarwhalsT_co]):
    def to_narwhals(self, *args: Any, **kwds: Any) -> ToNarwhalsT_co: ...

Since they're already initialized, they have a Version - so we have everything we need:

SparkLikeLazyFrame implementation

from __future__ import annotations

from typing import TYPE_CHECKING

from narwhals.typing import CompliantLazyFrame
from narwhals.utils import Implementation
from narwhals.utils import Version

if TYPE_CHECKING:
    from narwhals._spark_like.expr import SparkLikeExpr
    from narwhals.dataframe import LazyFrame

    SQLFrameDataFrame = BaseDataFrame[Any, Any, Any, Any, Any]

class SparkLikeLazyFrame(CompliantLazyFrame["SparkLikeExpr", "SQLFrameDataFrame"]):
    _native_frame: SQLFrameDataFrame
    _implementation: Implementation
    _backend_version: tuple[int, ...]
    _version: Version

    def to_narwhals(self) -> LazyFrame[SQLFrameDataFrame]:
        if self._version is Version.MAIN:
            from narwhals.dataframe import LazyFrame

            return LazyFrame(self, level="lazy")
        from narwhals.stable.v1 import LazyFrame as LazyFrameV1

        return LazyFrameV1(self, level="lazy")

Putting it all together

from __future__ import annotations

from typing import TYPE_CHECKING
from typing import cast

from narwhals._namespace import Namespace

if TYPE_CHECKING:
    from narwhals._spark_like.dataframe import SQLFrameDataFrame


native_object = cast("SQLFrameDataFrame", "pretend im a sqlframe")
narwhals_object = (
    Namespace.from_native_object(native_object)
    .compliant.from_native(native_object)
    .to_narwhals()
)

I think this is pretty clean 🙂

[dangotbanned]

I tried using the actual types first (acb5787), but they broke the @overload(s).

These seem specific enough to match only the intended target(s)

[MarcoGorelli]

thanks @dangotbanned

[dangotbanned]

thanks @dangotbanned

thanks @MarcoGorelli!

Just going to make the replacements now in utils and functions - where this'll be used first

• feat: Adds private Namespace class #2324 (comment)
• refactor: *(Namespace|DataFrame).from_numpy #2283

[dangotbanned]

@MarcoGorelli after (#2324 (commits)) I'm gonna call it here, too easy to get carried away 😅

Hopefully you get an idea for how much of from_native this can simplify.
Especially if we add ToNarwhals later (#2324 (comment))