We are seeing an issue in NDS query14_part1 where we see it seemingly never finishes with cuDF versions after: #11100. Reverting this PR fixes the issue. I believe it's making slow progress, but other timeouts are kicking in (in one case after 2 hours). I can see the issue is the left_semi_join cuDF call, specifically cuco::detail::insert taking all of the time.
In scala, a pretty easy repro is:
import ai.rapids.cudf.Table
val left = Table.readParquet(new java.io.File("left_keys_1bacb1e0-70cb-4cba-b01d-b49bd09842e2544880846.parquet"))
val right = Table.readParquet(new java.io.File("right_keys_1bacb1e0-70cb-4cba-b01d-b49bd09842e21899732691.parquet"))
spark.time { left.leftSemiJoinGatherMap(right, false) }
This is 3 seconds with the new code, and a few millis (I've seen as low as 3 ms) after reverting the PR.
regerssion_cudf_samples.zip
We are seeing an issue in NDS query14_part1 where we see it seemingly never finishes with cuDF versions after: #11100. Reverting this PR fixes the issue. I believe it's making slow progress, but other timeouts are kicking in (in one case after 2 hours). I can see the issue is the
left_semi_joincuDF call, specificallycuco::detail::inserttaking all of the time.In scala, a pretty easy repro is:
This is 3 seconds with the new code, and a few millis (I've seen as low as 3 ms) after reverting the PR.
regerssion_cudf_samples.zip