Skip to content

Isolate REST client single host tests#30504

Merged
jasontedor merged 1 commit intoelastic:masterfrom
jasontedor:isolate-rest-client-single-host-tests
May 10, 2018
Merged

Isolate REST client single host tests#30504
jasontedor merged 1 commit intoelastic:masterfrom
jasontedor:isolate-rest-client-single-host-tests

Conversation

@jasontedor
Copy link
Copy Markdown
Member

@jasontedor jasontedor commented May 10, 2018

These tests are sharing the same server and client for every test. Yet, we are seeing some tests fail with mysterious connection resets. It is not clear what is happening but one theory is that the tests are interfering with each other. This commit moves to use a separate server and client per test.

Relates #29558

@jasontedor
Isolate REST client single host tests
98168a8 
These tests are sharing the same server and client for every test. Yet,
we are seeing some tests fail with mysterious connection resets. It is
not clear what is happening but one theory is that the tests are
interfering with each other. This commit moves to use a separate server
and client per test.
@jasontedor jasontedor added >test Issues or PRs that are addressing/adding tests review :Clients/Java Low Level REST Client Minimal dependencies Java Client for Elasticsearch v7.0.0 v6.4.0 v6.3.1 labels May 10, 2018
@elasticmachine
Copy link
Copy Markdown
Collaborator

elasticmachine commented May 10, 2018

Pinging @elastic/es-core-infra

@jasontedor
Copy link
Copy Markdown
Member Author

jasontedor commented May 10, 2018

This does not materially change the execution time of these tests:

before:

> Task :client:rest:test
==> Test Summary: 1 suite, 8 tests
   [junit4] JVM J0:     0.32 ..     5.20 =     4.88s
   [junit4] Execution time total: 5.21 sec.
   [junit4] Tests summary: 1 suite, 8 tests

after:

> Task :client:rest:test
==> Test Summary: 1 suite, 8 tests
   [junit4] JVM J0:     0.35 ..     5.75 =     5.40s
   [junit4] Execution time total: 5.77 sec.
   [junit4] Tests summary: 1 suite, 8 tests

talevy
talevy approved these changes May 10, 2018
View reviewed changes
Copy link
Copy Markdown
Contributor

@talevy talevy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks!

@jasontedor jasontedor merged commit 596b262 into elastic:master May 10, 2018
jasontedor added a commit that referenced this pull request May 10, 2018
@jasontedor
Isolate REST client single host tests (#30504)
a6233b9 
These tests are sharing the same server and client for every test. Yet,
we are seeing some tests fail with mysterious connection resets. It is
not clear what is happening but one theory is that the tests are
interfering with each other. This commit moves to use a separate server
and client per test.
jasontedor added a commit that referenced this pull request May 10, 2018
@jasontedor
Isolate REST client single host tests (#30504)
d924ac7 
These tests are sharing the same server and client for every test. Yet,
we are seeing some tests fail with mysterious connection resets. It is
not clear what is happening but one theory is that the tests are
interfering with each other. This commit moves to use a separate server
and client per test.
dnhatn added a commit that referenced this pull request May 10, 2018
@dnhatn
Merge branch 'master' into ccr
a5be414 
* master:
  Upgrade to Lucene-7.4-snapshot-6705632810 (#30519)
  add version compatibility from 6.4.0 after backport, see #30319 (#30390)
  Security: Simplify security index listeners (#30466)
  Add proper longitude validation in geo_polygon_query (#30497)
  Remove Discovery.AckListener.onTimeout() (#30514)
  Build: move generated-resources to build (#30366)
  Reindex: Fold "with all deps" project into reindex (#30154)
  Isolate REST client single host tests (#30504)
  Solve Gradle deprecation warnings around shadowJar (#30483)
  SAML: Process only signed data (#30420)
  Remove BWC repository test (#30500)
  Build: Remove xpack specific run task (#30487)
  AwaitsFix IntegTestZipClientYamlTestSuiteIT#indices.split tests
  LLClient: Add setJsonEntity (#30447)
  Expose CommonStatsFlags directly in IndicesStatsRequest. (#30163)
  Silence IndexUpgradeIT test failures. (#30430)
  Bump Gradle heap to 1792m (#30484)
  [docs] add warning for read-write indices in force merge documentation (#28869)
  Avoid deadlocks in cache (#30461)
  Test: remove hardcoded list of unconfigured ciphers (#30367)
  mute SplitIndexIT due to #30416
  Docs: Test examples that recreate lang analyzers  (#29535)
  BulkProcessor to retry based on status code (#29329)
  Add GET Repository High Level REST API (#30362)
  add a comment explaining the need for RetryOnReplicaException on missing mappings
  Add `coordinating_only` node selector (#30313)
  Stop forking groovyc (#30471)
  Avoid setting connection request timeout (#30384)
  Use date format in `date_range` mapping before fallback to default (#29310)
  Watcher: Increase HttpClient parallel sent requests (#30130)

# Conflicts:
#	x-pack/plugin/core/src/test/java/org/elasticsearch/xpack/core/LocalStateCompositeXPackPlugin.java
dnhatn added a commit that referenced this pull request May 10, 2018
@dnhatn
Merge branch '6.x' into ccr-6.x
4785696 
* 6.x:
  Upgrade to Lucene-7.4-snapshot-6705632810 (#30519)
  Remove Discovery.AckListener.onTimeout() (#30514)
  Build: move generated-resources to build (#30366)
  Reindex: Fold "with all deps" project into reindex (#30154)
  Isolate REST client single host tests (#30504)
  Remove BWC repository test (#30500)
  Build: Remove xpack specific run task (#30487)
  AwaitsFix IntegTestZipClientYamlTestSuiteIT#indices.split tests
  LLClient: Add setJsonEntity (#30447)
  [docs] add warning for read-write indices in force merge documentation (#28869)
  Avoid deadlocks in cache (#30461)
  BulkProcessor to retry based on status code (#29329)
  Avoid setting connection request timeout (#30384)
  Test: remove hardcoded list of unconfigured ciphers (#30367)
  Add GET Repository High Level REST API (#30362)
  mute SplitIndexIT due to #30416
  Docs: Test examples that recreate lang analyzers  (#29535)
  add a comment explaining the need for RetryOnReplicaException on missing mappings
  Pass the task to broadcast actions (#29672)
  Stop forking groovyc (#30471)
  Add `coordinating_only` node selector (#30313)
  Fix accidental error in changelog
  Use date format in `date_range` mapping before fallback to default (#29310)
  Watcher: Increase HttpClient parallel sent requests (#30130)
  [Security][Tests] Azeri(Turkish) locale tripps opensaml dependency
@jasontedor jasontedor mentioned this pull request May 11, 2018
Closed
@javanna
Copy link
Copy Markdown
Contributor

javanna commented May 12, 2018

thanks @jasontedor , good idea.

martijnvg added a commit to martijnvg/elasticsearch that referenced this pull request May 14, 2018
@martijnvg
Merge remote-tracking branch 'es/ccr' into ccr_security
6f969f5 
* es/ccr: (78 commits)
  Upgrade to Lucene-7.4-snapshot-6705632810 (elastic#30519)
  add version compatibility from 6.4.0 after backport, see elastic#30319 (elastic#30390)
  Security: Simplify security index listeners (elastic#30466)
  Add proper longitude validation in geo_polygon_query (elastic#30497)
  Remove Discovery.AckListener.onTimeout() (elastic#30514)
  Build: move generated-resources to build (elastic#30366)
  Reindex: Fold "with all deps" project into reindex (elastic#30154)
  Isolate REST client single host tests (elastic#30504)
  Solve Gradle deprecation warnings around shadowJar (elastic#30483)
  SAML: Process only signed data (elastic#30420)
  Remove BWC repository test (elastic#30500)
  Build: Remove xpack specific run task (elastic#30487)
  AwaitsFix IntegTestZipClientYamlTestSuiteIT#indices.split tests
  Enable soft-deletes in v6.4
  LLClient: Add setJsonEntity (elastic#30447)
  [CCR] Read changes from Lucene instead of translog (elastic#30120)
  Expose CommonStatsFlags directly in IndicesStatsRequest. (elastic#30163)
  Silence IndexUpgradeIT test failures. (elastic#30430)
  Bump Gradle heap to 1792m (elastic#30484)
  [docs] add warning for read-write indices in force merge documentation (elastic#28869)
  ...
@bleskes bleskes added v6.3.0 and removed v6.3.1 labels May 16, 2018
@gkozyryatskyy
Copy link
Copy Markdown

gkozyryatskyy commented Jan 3, 2019 •
edited
Loading

Hello guys!

Did you understand what was the problem with this "mysterious connection resets"? Did this separation fix the problem?
If using the same client from multiple threads are "interfering with each other" is this an elasticsearch rest client bug?

Im asking this, because in my project Im also seeing some "mysterious connection resets"

P.S. my exception looks the same, as described in #29558

rg.apache.http.ConnectionClosedException: Connection closed
	at org.apache.http.nio.protocol.HttpAsyncRequestExecutor.endOfInput(HttpAsyncRequestExecutor.java:344)
	at org.apache.http.impl.nio.DefaultNHttpClientConnection.consumeInput(DefaultNHttpClientConnection.java:261)
	at org.apache.http.impl.nio.client.InternalIODispatch.onInputReady(InternalIODispatch.java:81)
	at org.apache.http.impl.nio.client.InternalIODispatch.onInputReady(InternalIODispatch.java:39)
	at org.apache.http.impl.nio.reactor.AbstractIODispatch.inputReady(AbstractIODispatch.java:114)
	at org.apache.http.impl.nio.reactor.BaseIOReactor.readable(BaseIOReactor.java:162)
	at org.apache.http.impl.nio.reactor.AbstractIOReactor.processEvent(AbstractIOReactor.java:337)
	at org.apache.http.impl.nio.reactor.AbstractIOReactor.processEvents(AbstractIOReactor.java:315)
	at org.apache.http.impl.nio.reactor.AbstractIOReactor.execute(AbstractIOReactor.java:276)
	at org.apache.http.impl.nio.reactor.BaseIOReactor.execute(BaseIOReactor.java:104)
	at org.apache.http.impl.nio.reactor.AbstractMultiworkerIOReactor$Worker.run(AbstractMultiworkerIOReactor.java:588)
	at java.lang.Thread.run(Thread.java:748)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

1 more reviewer

@talevy talevy talevy approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

:Clients/Java Low Level REST Client Minimal dependencies Java Client for Elasticsearch >test Issues or PRs that are addressing/adding tests v6.3.0 v6.4.0 v7.0.0-beta1

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

7 participants

@jasontedor @elasticmachine @javanna @gkozyryatskyy @talevy @colings86 @bleskes