Refactor mixed_semi_join using cuco::static_set#16230
Merged
rapids-bot[bot] merged 36 commits intorapidsai:branch-24.10from Sep 18, 2024
Merged
Refactor mixed_semi_join using cuco::static_set#16230rapids-bot[bot] merged 36 commits intorapidsai:branch-24.10from
rapids-bot[bot] merged 36 commits intorapidsai:branch-24.10from
Conversation
srinivasyadav18
added
improvement
PointKernel
reviewed
Jul 9, 2024
As rapids-cmake uses the latest cuco which uses key_equal as lhs, thus build and probe table needed to swaped accordingly.
JayjeetAtGithub
previously requested changes
Jul 18, 2024
karthikeyann
reviewed
Aug 6, 2024
| experimental::row::equality::preprocessed_table::create(probe, stream); | ||
| auto const row_comparator = | ||
| cudf::experimental::row::equality::two_table_comparator{preprocessed_probe, preprocessed_build}; | ||
| cudf::experimental::row::equality::two_table_comparator{preprocessed_build, preprocessed_probe}; |
Contributor
There was a problem hiding this comment.
is there a reason for this order change build and probe?
Member
There was a problem hiding this comment.
I need to dig into why but without this change the gtests start failing. Maybe @srinivasyadav18 can provide us a quick insight.
Member
There was a problem hiding this comment.
Update: We changed this order and added swap_tables=true at mixed_join_kernels_semi.cu:+72 (tagged you there) to fix the left and right directionality in the equality in single_expression_equality::operator(). Specifically, the single_expression_equality expects indices in {build, probe} order and we have also reset the hasher and equality functors for the cuco::static_set which swaps the internal left and right directions for {build, probe} hence us using swap_table=true.
I know it's quite confusing. 😅
Member
There was a problem hiding this comment.
This is mainly due to a breaking change in NVIDIA/cuCollections#479 where we always put query keys on the left-hand side for comparison.
Contributor
Please provide benchmark showing what benefit this PR will bring to us. |
ttnghia
reviewed
Aug 15, 2024
ttnghia
reviewed
Aug 15, 2024
Member
|
/ok to test |
mhaseeb123
requested review from
a team and
mhaseeb123
and removed request for
JayjeetAtGithub,
brandon-b-miller,
galipremsagar,
jameslamb and
mhaseeb123
PointKernel
approved these changes
Sep 13, 2024
Member
PointKernel
left a comment
PointKernel
left a comment
There was a problem hiding this comment.
One minor improvement otherwise looks great to me.
@mhaseeb123 Thank you for stepping in to help resolve the issue and finalize the PR! 🔥 🔥 🔥
| experimental::row::equality::preprocessed_table::create(probe, stream); | ||
| auto const row_comparator = | ||
| cudf::experimental::row::equality::two_table_comparator{preprocessed_probe, preprocessed_build}; | ||
| cudf::experimental::row::equality::two_table_comparator{preprocessed_build, preprocessed_probe}; |
Member
There was a problem hiding this comment.
This is mainly due to a breaking change in NVIDIA/cuCollections#479 where we always put query keys on the left-hand side for comparison.
Co-authored-by: Yunsong Wang <yunsongw@nvidia.com>
mhaseeb123
added
DO NOT MERGE
Member
|
Performance results from Spark (spark-h). Thanks to @zpuller for running this. Queries of interest: q16 and q94 that use |
ttnghia
approved these changes
Sep 17, 2024
Contributor
|
Shorter for easy reading:
|
Member
|
/merge |
mhaseeb123
removed
the
4 - Needs Review
This was referenced Sep 19, 2024
This was referenced Sep 19, 2024
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
6 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
This PR refactors
mixed_semi_joinby replacing cuco legacystatic_mapwith lateststatic_set.Contributes to #12261.
Checklist