Skip to content

Fix retrieve_all for containers with large capacity#580

Merged
PointKernel merged 8 commits intoNVIDIA:devfrom
PointKernel:fix-large-retrieve-all
Aug 16, 2024
Merged

Fix retrieve_all for containers with large capacity#580
PointKernel merged 8 commits intoNVIDIA:devfrom
PointKernel:fix-large-retrieve-all

Conversation

@PointKernel
Copy link
Copy Markdown
Member

@PointKernel PointKernel commented Aug 15, 2024

Fix #576

This PR fixes the large input retrieve_all bug with a method similar to the streaming approach mentioned in NVIDIA/cccl#1422 (comment).

To be reverted once the CCCL fix is in place.

@PointKernel PointKernel added type: bug Something isn't working helps: rapids Helps or needed by RAPIDS labels Aug 15, 2024
@PointKernel PointKernel marked this pull request as ready for review August 16, 2024 00:36
@PointKernel PointKernel added the Needs Review Awaiting reviews before merging label Aug 16, 2024
@PointKernel PointKernel changed the title Fix retrieve_all for container with large capacity Fix retrieve_all for containers with large capacity Aug 16, 2024
@PointKernel PointKernel merged commit abc5095 into NVIDIA:dev Aug 16, 2024
@PointKernel PointKernel deleted the fix-large-retrieve-all branch August 16, 2024 19:26
jamxia155 pushed a commit to rapidsai/gqe that referenced this pull request Dec 11, 2025
…gqe!197)

The following changes are addressed in the MR

- Upgrading rapids-24.08 and cudf-24.08
   - Cudf stopped exporting `cudf::detail::initialize_with_identity`, temporarily solved by copying the function definition
- Upgrading to newer version of cucollection that temporarily fixes [cccl's retrieve_all bug](NVIDIA/cuCollections#580) 
   - They updated the key equality comparison internally so that the slot content is always on the right-hand side and the query key is on the left-hand side ([here](NVIDIA/cuCollections#479))
- Improving groupby implementation for enabling TPC-H SF1k Q18 
   - Freeing memory that is no longer used
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

helps: rapids Helps or needed by RAPIDS Needs Review Awaiting reviews before merging type: bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[BUG]: retrieve_all fails with over 1B items

2 participants