Update to CCCL 3.2

[bdice]

This issue tracks the progress of updating RAPIDS to support CCCL 3.2.0. The final tag is not yet available, we will use pre-release commits until the final tag is available.

Related work:

• [FEA] Support memory resources from CCCL 3.2 rmm#2011
  • This is the main tracking issue for the majority of the work still needed. Part of this is required to upgrade but not all changes will be merged prior to upgrading.
• Update to CCCL v3.2.0 rapids-cmake#955 -- the final PR, depends on all other PRs below
• CCCL 3.2 resource ref compatibility rmm#2176
• Fix arena test for CCCL 3.2 rmm#2186
• Fix injected class name bug in cccl_resource_ref constraints, use cuda::std::optional rmm#2188
• Include thrust::pair headers cudf#20708
• Move thrust::tuple usages to cuda::std::tuple cudf#20717
• Compatibility updates for CCCL 3.2 cudf#20725
• Changes needed for CCCL 3.2 compatibility cudf#20810
• Compatibility with CCCL 3.2 rapidsmpf#755
• Include <thrust/for_each.h> where it is used raft#2883
• Fix is_exhaustive, no longer constexpr raft#2888
• Include <thrust/distance.h> where it is used cugraph#5357
• Compatibility fixes for CCCL 3.2 cugraph#5363
• Proclaim return types and other fixes needed for CCCL 3.2 cugraph#5375
• Fix missing headers for thrust::pair, add trailing return types where needed NVIDIA/cuopt#732

Testing plan: Build and test each RAPIDS repo in unified devcontainers with both CCCL 3.1 and 3.2.

• ✅ rmm: built & tested on main and cccl-3.2.x branches
• ✅ ucxx: built & tested on cccl-3.2.x branch
• ✅ dask-cuda: tests pass, no changes were needed
• ✅ kvikio: built & tested on cccl-3.2.x branch
• ✅ cudf: built & tested on cccl-3.2.x branch locally
  • CI testing in progress: Test CCCL 3.2.x cudf#20723. Note: Only some jobs are expected to pass in CI.
• 🟨 rapidsmpf: built & tested on cccl-3.2.x branch locally, C++ worked but I am having a build issue with Python's CMake code not finding cuCollections that is not related to this upgrade
• ✅ raft: built & tested on cccl-3.2.x branch locally
• ✅ cuvs: built & tested on cccl-3.2.x branch locally
• ✅ cumlprims_mg: built & tested on cccl-3.2.x branch locally
• ✅ cuml: built & tested on cccl-3.2.x branch locally
• ✅ cugraph: built & tested on cccl-3.2.x branch locally
• ✅ cugraph-gnn: built & tested on cccl-3.2.x branch locally
• XGBoost - validated by @dantegd, follow-up work in Backport patch from dmlc/main to 3.1 of feedstock for CCCL 3.2 compatibility  xgboost-feedstock#117
• cuOpt
• 🟨 Spark-RAPIDS
  • Testing from @GaryShen2008 shows that builds pass but encountered a CMake error when installing. That was fixed in this commit. We are going to proceed because Spark has the ability to pin its dependencies if needed to unblock CI.
• 🟨 CCCL-RAPIDS CI
  • Generally working aside from known issues (cuGraph needs its final PR to be merged)
  • Some issues with ninja: fatal: pipe: Too many open files; I asked @trxcllnt for help fixing that
  • https://github.com/NVIDIA/cccl/actions/runs/20329428892/job/58403167622#step:6:3262

[bdice]

Followups needed before closing:

• Update to a newer CCCL 3.2.x prerelease with a fix for mdspan rapids-cmake#959
• Fix CCCL 3.2 mdspan constexpr issues raft#2911

There will be continued work on rapidsai/rmm#2011 which can be separated from this issue.

[bdice]

Closing as complete!