Skip to content

[Backport 2.x] Update multipart download path to first write to temp files#10389

Merged
mch2 merged 1 commit intoopensearch-project:2.xfrom
andrross:backport/backport-10347-to-2.x
Oct 5, 2023
Merged

[Backport 2.x] Update multipart download path to first write to temp files#10389
mch2 merged 1 commit intoopensearch-project:2.xfrom
andrross:backport/backport-10347-to-2.x

Conversation

@andrross
Copy link
Copy Markdown
Member

@andrross andrross commented Oct 5, 2023

Backports ca0dae6 from #10347

Check List

  • New functionality includes testing.
    • All tests pass
  • New functionality has been documented.
    • New functionality has javadoc added
  • Commits are signed per the DCO using --signoff
  • Commit changes are listed out in CHANGELOG.md file (See: Changelog)
  • GitHub issue/PR created in OpenSearch documentation repo for the required public documentation changes (#[Issue/PR number])

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

@mch2 @andrross
Update multipart download path to first write to temp files (opensear…
697dd68 
…ch-project#10347)

* Update multipart download path to write to temp files.

This change updates ReadContextListener to first write parts to a temp location
until all parts have been received.

Signed-off-by: Marc Handalian <handalm@amazon.com>

* Suppress forbidden IOUtils.fsync

Signed-off-by: Marc Handalian <handalm@amazon.com>

* Remove unnecessary logging format

Signed-off-by: Marc Handalian <handalm@amazon.com>

* sync directory after file rename

Signed-off-by: Marc Handalian <handalm@amazon.com>

* Remove flaky threadpool terminate test

Signed-off-by: Marc Handalian <handalm@amazon.com>

---------

Signed-off-by: Marc Handalian <handalm@amazon.com>
(cherry picked from commit ca0dae6)
@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Oct 5, 2023

Compatibility status:

Checks if related components are compatible with change 697dd68

Incompatible components

Skipped components

Compatible components

Compatible components: [https://github.com/opensearch-project/security-analytics.git, https://github.com/opensearch-project/security.git, https://github.com/opensearch-project/custom-codecs.git, https://github.com/opensearch-project/geospatial.git, https://github.com/opensearch-project/index-management.git, https://github.com/opensearch-project/notifications.git, https://github.com/opensearch-project/sql.git, https://github.com/opensearch-project/neural-search.git, https://github.com/opensearch-project/job-scheduler.git, https://github.com/opensearch-project/observability.git, https://github.com/opensearch-project/k-nn.git, https://github.com/opensearch-project/cross-cluster-replication.git, https://github.com/opensearch-project/alerting.git, https://github.com/opensearch-project/anomaly-detection.git, https://github.com/opensearch-project/asynchronous-search.git, https://github.com/opensearch-project/common-utils.git, https://github.com/opensearch-project/ml-commons.git, https://github.com/opensearch-project/performance-analyzer.git, https://github.com/opensearch-project/reporting.git, https://github.com/opensearch-project/performance-analyzer-rca.git]

@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Oct 5, 2023

Gradle Check (Jenkins) Run Completed with:

@codecov
Copy link
Copy Markdown

codecov bot commented Oct 5, 2023

Codecov Report

Merging #10389 (697dd68) into 2.x (fc7dc20) will increase coverage by 0.03%.
The diff coverage is 76.19%.

@@             Coverage Diff              @@
##                2.x   #10389      +/-   ##
============================================
+ Coverage     70.75%   70.78%   +0.03%     
+ Complexity    58397    58359      -38     
============================================
  Files          4816     4816              
  Lines        276031   276050      +19     
  Branches      40565    40565              
============================================
+ Hits         195306   195410     +104     
+ Misses        64105    63928     -177     
- Partials      16620    16712      +92
Files Coverage Δ
...tore/stream/read/listener/ReadContextListener.java 79.48% <76.19%> (-8.65%) ⬇️

... and 467 files with indirect coverage changes

@mch2 mch2 merged commit 78c55b7 into opensearch-project:2.x Oct 5, 2023
@andrross andrross deleted the backport/backport-10347-to-2.x branch February 8, 2024 19:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@mch2 mch2 mch2 approved these changes

@reta reta Awaiting requested review from reta reta is a code owner

@anasalkouz anasalkouz Awaiting requested review from anasalkouz anasalkouz is a code owner

@Bukhtawar Bukhtawar Awaiting requested review from Bukhtawar Bukhtawar is a code owner

@CEHENKLE CEHENKLE Awaiting requested review from CEHENKLE CEHENKLE is a code owner

@dblock dblock Awaiting requested review from dblock

@gbbafna gbbafna Awaiting requested review from gbbafna gbbafna is a code owner

@setiah setiah Awaiting requested review from setiah

@kartg kartg Awaiting requested review from kartg

@kotwanikunal kotwanikunal Awaiting requested review from kotwanikunal kotwanikunal is a code owner

@nknize nknize Awaiting requested review from nknize

@owaiskazi19 owaiskazi19 Awaiting requested review from owaiskazi19 owaiskazi19 is a code owner

@peternied peternied Awaiting requested review from peternied

@Rishikesh1159 Rishikesh1159 Awaiting requested review from Rishikesh1159 Rishikesh1159 is a code owner

@ryanbogan ryanbogan Awaiting requested review from ryanbogan

@saratvemulapalli saratvemulapalli Awaiting requested review from saratvemulapalli saratvemulapalli is a code owner

@shwetathareja shwetathareja Awaiting requested review from shwetathareja shwetathareja is a code owner

@dreamer-89 dreamer-89 Awaiting requested review from dreamer-89

@VachaShah VachaShah Awaiting requested review from VachaShah

@dbwiddis dbwiddis Awaiting requested review from dbwiddis dbwiddis is a code owner

@sachinpkale sachinpkale Awaiting requested review from sachinpkale sachinpkale is a code owner

@sohami sohami Awaiting requested review from sohami sohami is a code owner

@msfroh msfroh Awaiting requested review from msfroh msfroh is a code owner

Assignees

No one assigned

Labels

skip-changelog

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@andrross @mch2