Add logstash system index APIs#53350
Merged
williamrandolph merged 33 commits intoelastic:masterfrom Sep 14, 2020
Merged
Conversation
Collaborator
|
Pinging @elastic/es-core-infra (:Core/Infra/Core) |
jaymode
changed the title
AthenaEryma
reviewed
Mar 13, 2020
Contributor
AthenaEryma
left a comment
AthenaEryma
left a comment
There was a problem hiding this comment.
Looks good overall, left a few comments about tweaks I think we should make.
...gin/logstash/src/main/java/org/elasticsearch/xpack/logstash/action/DeletePipelineAction.java
Outdated
Show resolved
Hide resolved
...n/logstash/src/main/java/org/elasticsearch/xpack/logstash/action/DeletePipelineResponse.java
Show resolved
Hide resolved
...gstash/src/main/java/org/elasticsearch/xpack/logstash/action/TransportGetPipelineAction.java
Outdated
Show resolved
Hide resolved
...gstash/src/main/java/org/elasticsearch/xpack/logstash/action/TransportGetPipelineAction.java
Outdated
Show resolved
Hide resolved
...gstash/src/main/java/org/elasticsearch/xpack/logstash/action/TransportGetPipelineAction.java
Outdated
Show resolved
Hide resolved
| mGetResponse -> listener.onResponse( | ||
| new GetPipelineResponse( | ||
| Arrays.stream(mGetResponse.getResponses()) | ||
| .filter(itemResponse -> itemResponse.isFailed() == false) |
Contributor
There was a problem hiding this comment.
I'm not sure what to do instead, but if some of the GETs fail, do we really just want to silently swallow that? Should we at least log something here?
...gstash/src/main/java/org/elasticsearch/xpack/logstash/action/TransportPutPipelineAction.java
Outdated
Show resolved
Hide resolved
...rc/test/java/org/elasticsearch/xpack/logstash/PipelineRequestResponseSerializationTests.java
Outdated
Show resolved
Hide resolved
Contributor
|
@elasticmachine Please run elasticsearch-ci/2 |
AthenaEryma
approved these changes
Aug 27, 2020
Contributor
AthenaEryma
left a comment
AthenaEryma
left a comment
There was a problem hiding this comment.
Left a few very minor comments, but otherwise LGTM. No need for another round of review after addressing my comments here.
...ugin/logstash/src/main/java/org/elasticsearch/xpack/logstash/action/GetPipelineResponse.java
Show resolved
Hide resolved
x-pack/plugin/src/test/java/org/elasticsearch/xpack/test/rest/LogstashSystemIndexIT.java
Outdated
Show resolved
Hide resolved
...gstash/src/test/java/org/elasticsearch/xpack/logstash/action/DeletePipelineRequestTests.java
Show resolved
Hide resolved
williamrandolph
added a commit
to williamrandolph/elasticsearch
that referenced
this pull request
Sep 14, 2020
We want Logstash indices to be system indices, but the logstash service will still need to be able to manage its indices. This PR adds special system index APIs to the logstash plugin so that logstash can manage its pipelines without direct access to the underlying indices. * Add logstash module with dedicated logstash APIs * merge with x-pack plugin * add system index access allowance * Break out serialization tests into distinct classes * Log failures for partial multiget failure * Move LogstashSystemIndexIT to javaRestTest task Co-authored-by: William Brafford <william.brafford@elastic.co>
williamrandolph
added a commit
that referenced
this pull request
Sep 15, 2020
We want Logstash indices to be system indices, but the logstash service will still need to be able to manage its indices. This PR adds special system index APIs to the logstash plugin so that logstash can manage its pipelines without direct access to the underlying indices. * Add logstash module with dedicated logstash APIs * merge with x-pack plugin * add system index access allowance * Break out serialization tests into distinct classes * Log failures for partial multiget failure * Move LogstashSystemIndexIT to javaRestTest task Co-authored-by: William Brafford <william.brafford@elastic.co> Co-authored-by: Jay Modi <jaymode@users.noreply.github.com>
5 tasks
jaymode
added a commit
to jaymode/kibana
that referenced
this pull request
Oct 13, 2020
This change updates the logstash pipeline management plugin to use pipeline management APIs in Elasticsearch rather than directly accessing the .logstash index. In Elasticsearch 8.0, direct access to system indices will no longer be allowed when using standard APIs. Given this change, a new set of APIs has been created specifically for the management of Logstash pipelines and this change makes use of the APIs. Relates elastic/elasticsearch#53350
2 tasks
61 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
8 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This commit adds REST apis for logstash and its system index,
.logstash, which is used to store pipelines for central management. The module adds new endpoints that enable CRUD operationson the pipelines stored in the index.