Closed
Conversation
jakelandis
force-pushed
the
ingest_back_ports
branch
2 times, most recently
from
c3e252b to
7eac0aa
Compare
* Allows repeated, non-recursive invocation of the same pipeline
…#33839) * ingest: support simulate with verbose for pipeline processor This change better supports the use of simulate?verbose with the pipeline processor. Prior to this change any pipeline processors executed with simulate?verbose would not show all intermediate processors for the inner pipelines. This changes also moves the PipelineProcess and TrackingResultProcessor classes to enable instance checks and to avoid overly public classes. As well this updates the error message for when cycles are detected in pipelines calling other pipelines.
Prior to this change when a pipeline processor called another pipeline, only the stats for the first processor were recorded. The stats for the subsequent pipelines were ignored. This change properly accounts for pipelines irregardless if they are the first or subsequently called pipelines. This change moves the state of the stats from the IngestService to the pipeline itself. Cluster updates are safe since the pipelines map is atomically swapped, and if a cluster update happens while iterating over stats (now read directly from the pipeline) a slightly stale view of stats may be shown.
jakelandis
force-pushed
the
ingest_back_ports
branch
2 times, most recently
from
a191cf1 to
4adcd77
Compare
Contributor
Author
|
3 commits ran to success both locally and via ci. Closing PR and pushing manually (don't want to squash the commits). |
Contributor
Author
|
3 commits ran to success both locally and via ci. Closing PR and pushing manually (don't want to squash the commits). |
1 similar comment
Contributor
Author
|
3 commits ran to success both locally and via ci. Closing PR and pushing manually (don't want to squash the commits). |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Ingest node 6.x back ports for:
correctly measure chained pipeline stats (#33912)
support simulate with verbose for pipeline processor (#33839)
Allow Repeated Invocation of Pipeline (#33419)
All cherry-picked with only 1 minor conflict.
PR only to run CI prior to merge.