Skip to content

feat(deps): support pyarrow 18#10381

Merged
cpcloud merged 3 commits intoibis-project:mainfrom
cpcloud:bump-pyarrow
Oct 29, 2024
Merged

feat(deps): support pyarrow 18#10381
cpcloud merged 3 commits intoibis-project:mainfrom
cpcloud:bump-pyarrow

Conversation

@cpcloud
Copy link
Copy Markdown
Member

@cpcloud cpcloud commented Oct 28, 2024

Bump pyarrow upper bound to 18.

@cpcloud cpcloud added the dependencies Issues or PRs related to dependencies label Oct 28, 2024
@cpcloud cpcloud added this to the 10.0 milestone Oct 28, 2024
@github-actions github-actions bot added ci Continuous Integration issues or PRs nix Issues or PRs related to nix labels Oct 28, 2024
Copy link
Copy Markdown
Member

@kszucs kszucs left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The nix build is failing for some reason, otherwise it looks good to me.

@cpcloud
Copy link
Copy Markdown
Member Author

cpcloud commented Oct 28, 2024

The reason is because the MacOS SDK that ships with nix is older than what pyarrow has published wheels for (and arrow-cpp in nixpkgs is at 18.0.0 yet so it can't be built from source (but I opened a PR for that NixOS/nixpkgs#351868)).

It would be great if pyarrow continued to publish 11_0 arm64 macos wheels.

@cpcloud
Copy link
Copy Markdown
Member Author

cpcloud commented Oct 28, 2024

@kszucs Should I open an issue about the additional arm64 macos wheels?

@cpcloud
Copy link
Copy Markdown
Member Author

cpcloud commented Oct 28, 2024

We can also disable the aarch64-darwin build until NixOS/nixpkgs#346043 is merged into nixpkgs-unstable-small.

@cpcloud cpcloud force-pushed the bump-pyarrow branch 4 times, most recently from 465037a to e380ace Compare October 29, 2024 11:19
@cpcloud cpcloud merged commit 0d95336 into ibis-project:main Oct 29, 2024
@cpcloud cpcloud deleted the bump-pyarrow branch October 29, 2024 12:28
@noklam
Copy link
Copy Markdown

noklam commented Nov 29, 2024

I see this is included in 10.0.0 roadmap, any idea when will this be released? We are currently blocked by this because we need PyArrow 18 (we have tons of dependencies like spark etc, so it will only work with PyArrow 18 for Python 3.13).

We can wait a bit if it's going to be release soon, otherwise I may conditionally drop ibis support for Python 3.13

kedro-org/kedro-plugins#938

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ci Continuous Integration issues or PRs dependencies Issues or PRs related to dependencies nix Issues or PRs related to nix

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants