Skip to content

Use different monitoring bulk API paths depending on ES version#11203

Merged
ycombinator merged 3 commits intoelastic:masterfrom
ycombinator:replace-monitoring-endpoint
Mar 19, 2019
Merged

Use different monitoring bulk API paths depending on ES version#11203
ycombinator merged 3 commits intoelastic:masterfrom
ycombinator:replace-monitoring-endpoint

Conversation

@ycombinator
Copy link
Copy Markdown
Contributor

@ycombinator ycombinator commented Mar 11, 2019

Resolves #9480.

Starting Elasticsearch 7.0.0, Beats should ship their monitoring data to the _monitoring/bulk Elasticsearch API endpoint. Prior to 7.0.0, _xpack/monitoring/_bulk should be used. This PR implements this version-based conditional logic.

I used Wireshark to look at the ES API endpoints being hit.

Running this PR with ES 8.0.0 or ES 7.0.0, I confirmed that the POST _monitoring/bulk endpoint was being hit:

Screen Shot 2019-03-14 at 10 55 52 AM

And running this PR with ES 6.7.0, I confirmed that the POST _xpack/monitoring/_bulk endpoint was being hit:

Screen Shot 2019-03-14 at 10 56 42 AM

@ycombinator ycombinator requested a review from a team as a code owner March 11, 2019 23:44
@elasticmachine
Copy link
Copy Markdown
Contributor

Pinging @elastic/stack-monitoring

@ycombinator
Copy link
Copy Markdown
Contributor Author

Blocked by #11204.

@ycombinator ycombinator added in progress Pull request is currently in progress. review and removed blocked in progress Pull request is currently in progress. labels Mar 11, 2019
@jakelandis
Copy link
Copy Markdown

The paths look correct, and that probably as much I am competent to review here, so (lowercase) lgtm

Copy link
Copy Markdown

@urso urso left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wish we would not have to pass the versions to MonitoringBulkWith. Actually (*Connection) stores the versions, but this means that one has to call Connect before using this method.

@ycombinator
Copy link
Copy Markdown
Contributor Author

@urso So originally (see 96a8eba) I had it so the version wasn't being passed into MonitoringBulkWith. WDYT of that approach?

@urso
Copy link
Copy Markdown

urso commented Mar 15, 2019

more like:

func (c *Connection) SendMonitoring(params map[string]string, body []interface{}) error {
    ...

    if !c.version.IsValid() {
      if err := c.Connect(); err != nil {
        return err
      }
    }

    ...
    requ, err := newMonitoringBulkRequest(c.version, conn.URL, params, enc)
	if err != nil {
		return nil, err
	}
   
    ...
} 

@ycombinator
Copy link
Copy Markdown
Contributor Author

@urso I made the change per your review suggestion. This is ready for your 👀 again. Thanks!

@ycombinator ycombinator requested a review from ph March 19, 2019 14:34
Copy link
Copy Markdown
Contributor

@ph ph left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've discussed with @urso last week and I think we will merge this one as is, we need a better way to implement different version calls in our clients.

@ycombinator ycombinator merged commit 44c40c9 into elastic:master Mar 19, 2019
@ycombinator ycombinator removed the needs_backport PR is waiting to be backported to other branches. label Mar 19, 2019
ycombinator added a commit that referenced this pull request Mar 19, 2019
…pending on ES version (#11323)

Cherry-pick of PR #11203 to 7.0 branch. Original message: 

Resolves #9480.

Starting Elasticsearch 7.0.0, Beats should ship their monitoring data to the `_monitoring/bulk` Elasticsearch API endpoint. Prior to 7.0.0, `_xpack/monitoring/_bulk` should be used. This PR implements this version-based conditional logic.

I used Wireshark to look at the ES API endpoints being hit.

Running this PR with ES 8.0.0 or ES 7.0.0, I confirmed that the `POST _monitoring/bulk` endpoint was being hit:

<img width="1436" alt="Screen Shot 2019-03-14 at 10 55 52 AM" src="https://user-images.githubusercontent.com/51061/54380101-ed567780-4647-11e9-8ed1-9b9020bb85d4.png">

And running this PR with ES 6.7.0, I confirmed that the `POST _xpack/monitoring/_bulk` endpoint was being hit:

<img width="1437" alt="Screen Shot 2019-03-14 at 10 56 42 AM" src="https://user-images.githubusercontent.com/51061/54380094-eaf41d80-4647-11e9-8658-d9a6ba14541b.png">
@ycombinator ycombinator added the needs_backport PR is waiting to be backported to other branches. label Apr 3, 2019
@ycombinator ycombinator deleted the replace-monitoring-endpoint branch December 25, 2019 11:17
@ycombinator ycombinator removed the needs_backport PR is waiting to be backported to other branches. label Jan 15, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Replace _xpack/monitoring/* with _monitoring/* ES API calls

6 participants