[bugfix]: fix inconsistency result when replay concurrency operations#348
[bugfix]: fix inconsistency result when replay concurrency operations#348zhongkechen wants to merge 4 commits intomainfrom
Conversation
|
just for me, could you elaborate on what the fix is?
I'm having trouble understanding what this means |
Updated description of the PR. Hope it's easier to understand. |
| branches.add(childOp); | ||
| pendingQueue.add(childOp); | ||
| logger.debug("Item enqueued {}", name); | ||
| if (!skipped) { |
There was a problem hiding this comment.
why even call enqueueItem if we're going to skip it anyway. Wouldn't this future never resolve?
There was a problem hiding this comment.
branches will have the skipped items. We skip pendingQueue so that the skipped items will not be executed.
| int failed = failedCount.get(); | ||
|
|
||
| if (expectedCompletionStatus != null) { | ||
| if (succeeded + failed >= expectedCompletionStatus.completed) { |
There was a problem hiding this comment.
To make sure all succeeded/failed items/branches will succeed/fail in replay.
zhongkechen
force-pushed
the
concurrency
branch
from
6e4f56d to
26ac78a
Compare
2 participants
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
Issue Link, if available
Fixes: #344
Description
Two concurrency operation result consistent issues during replay
Branches/items that are skipped (neither succeeded nor failed ones) when checkpointing the results, could succeed or fail during replay. This occurs when a parallel operation or a map operation with large results has early termination conditions (min successful or max failure tolerant) and the items/branches are completed with an inconsistent order during replay.
Branches/items that succeeded or failed when checkpointing the results, could be skipped during replay. This occurs when a parallel operation or a map operation with large results has early termination conditions (min successful or max failure tolerant) and more branches than specified condition are completed due to concurrency.
Fixes:
Demo/Screenshots
Checklist
Testing
Unit Tests
Have unit tests been written for these changes? Yes
Integration Tests
Have integration tests been written for these changes? Yes
Examples
Has a new example been added for the change? (if applicable)