Skip to content

refactor(network): convert .crate downloads to use http_async#16902

Open
arlosi wants to merge 4 commits intorust-lang:masterfrom
arlosi:async/pkgset
Open

refactor(network): convert .crate downloads to use http_async#16902
arlosi wants to merge 4 commits intorust-lang:masterfrom
arlosi:async/pkgset

Conversation

@arlosi
Copy link
Copy Markdown
Contributor

@arlosi arlosi commented Apr 17, 2026

What does this PR try to resolve?

  • Makes the existing download and finish_download methods async. It may be possible to do a future refactor PR that combines these two methods together, since they no longer need to be separated to get parallel downloads. This could be a nice future simplification.
  • Remove the direct use of curl::Multi for downloading crates in parallel, and replace it with http_async.
  • Add .unordered() to several test file references. The async downloader is less deterministic in the order that it performs downloads, which would otherwise make the tests flaky.

cc #16845

How to test and review this PR?

Commit by commit. All commits pass tests.

r? @weihanglo

@rustbot rustbot added A-directory-source Area: directory sources (vendoring) A-git Area: anything dealing with git A-local-registry-source Area: local registry sources (vendoring) A-networking Area: networking issues, curl, etc. A-overrides Area: general issues with overriding dependencies (patch, replace, paths) A-registries Area: registries A-source-replacement Area: [source] replacement A-sparse-registry Area: http sparse registries S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Apr 17, 2026
@arlosi arlosi force-pushed the async/pkgset branch 2 times, most recently from 9f8d227 to cbbd5c4 Compare April 17, 2026 17:27
@arlosi arlosi mentioned this pull request Apr 17, 2026
5 tasks
Comment thread src/cargo/util/network/http_async.rs
Comment thread src/cargo/core/package.rs
Comment thread src/cargo/util/network/http_async.rs Outdated
Comment thread src/cargo/util/network/http_async.rs Outdated
Comment thread src/cargo/util/network/http_async.rs
@arlosi
Copy link
Copy Markdown
Contributor Author

arlosi commented Apr 17, 2026

Split off part of this change into #16903. Will wait until that one is in, then rebase this one.

@rustbot

This comment has been minimized.

arlosi added 4 commits April 17, 2026 19:22
Makes the two crate downloading methods async
Removes the existing Multi-based interface to use the http_async
abstraction for downloading crates.
@rustbot
Copy link
Copy Markdown
Collaborator

rustbot commented Apr 18, 2026

This PR was rebased onto a different master commit. Here's a range-diff highlighting what actually changed.

Rebasing is a normal part of keeping PRs up to date, so no action is needed—this note is just to help reviewers.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

A-directory-source Area: directory sources (vendoring) A-git Area: anything dealing with git A-local-registry-source Area: local registry sources (vendoring) A-networking Area: networking issues, curl, etc. A-overrides Area: general issues with overriding dependencies (patch, replace, paths) A-registries Area: registries A-source-replacement Area: [source] replacement A-sparse-registry Area: http sparse registries S-waiting-on-review Status: Awaiting review from the assignee but also interested parties.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants