Documentation for cargo vendor use cases

[Firstyear]

Problem

Cargo vendor has limited docs : https://doc.rust-lang.org/nightly/cargo/commands/cargo-vendor.html . While this describes what cargo vendor does as a tool, it doesn't encompass the larger needs of an opensource project to use vendoring for offline builds in build farms etc.

As a result, while attempting to integrate rust into a project, we have found that certain cases are non-intuitive or may not work.

I would like to ask that cargo vendor be documented with how a project would integrate it into their work flows. Important considerations are:

• We need to ship and provide .cargo/config in our source tree so that developers have consistent relative paths to vendor, and unpacked tar-balls match git trees. We also can't assume paths in offline build tools for various companies
• how to use cargo build/rustc with offline builds and vendored dependencies.
• how to use cargo test with offline builds and vendored dependencies
• how to correctly use cargo vendor to acquire dependencies
• how to perform online builds even in the presence of a source tree .cargo/config for developers who are updating dependencies, or rather, to smooth "on boarding" so we don't need to ask developers to vendor/prepare their environments to do builds.

These are all important aspects that today we have struggled with with cargo vendor, so having these generic workflows documented would really help other projects to adopt cargo with vendoring.

[alexcrichton]

Thanks for the detailed report here @Firstyear!

[rimutaka]

cargo vendor-related questions asked by the community

This may point at the common areas of confusion and misunderstanding. It may be worth mentioning reproducible builds and project archiving.

• StackOverflow search for cargo+vendor

  • https://stackoverflow.com/questions/72775250/how-to-manually-add-offline-dependency-to-rust-project-without-using-cargo-vendo
  • https://stackoverflow.com/questions/68774481/why-did-cargo-vendor-fail-to-write-the-checksum-for-a-crate-in-a-git-repository
  • https://stackoverflow.com/questions/71795175/cargo-install-cargo-generate-features-vendored-openssl-failed
  • https://stackoverflow.com/questions/60989162/how-to-list-a-projects-source-files-using-the-cargo-crate/60991227#60991227
  • https://stackoverflow.com/questions/76187534/is-the-local-cargo-registry-portable
  • https://stackoverflow.com/questions/55934605/cargo-registry-directory/56003687#56003687
  • https://stackoverflow.com/questions/72069434/vendoring-a-local-dependency-in-order-to-build-without-access-to-that-dependency
  • https://stackoverflow.com/questions/62772606/cannot-use-cargo-patch-with-a-git-dependency-in-docker
  • https://stackoverflow.com/questions/76187501/is-there-a-cargo-command-exclusively-to-download-crates-to-the-registry-director
  • https://stackoverflow.com/questions/32267233/how-to-build-a-project-using-cargo-in-an-offline-environment/65254702#65254702
  • https://stackoverflow.com/questions/69197273/cargo-build-should-i-keep-the-target-dir-after-a-compile-for-a-historical-aud/69197341#69197341
  • https://stackoverflow.com/questions/67562843/how-can-i-install-using-cargo-when-offline/67563261#67563261
  • https://stackoverflow.com/questions/74046866/how-to-add-vendored-crates-to-project-workspace
  • https://stackoverflow.com/questions/32267233/how-to-build-a-project-using-cargo-in-an-offline-environment/32267648#32267648
  • https://stackoverflow.com/questions/72774815/how-to-install-a-rust-dependency-in-an-environment-with-no-internet/72776183#72776183
  • https://stackoverflow.com/questions/76187534/is-the-local-cargo-registry-portable/76187919#76187919
  • https://stackoverflow.com/questions/61199511/how-do-i-reduce-or-speed-up-rebuilds-when-using-rustling-ontology

• Reddit: https://www.reddit.com/r/rust/search/?q=cargo+vendor&type=link&t=year

• Rust forums: https://users.rust-lang.org/search?q=vendor%20after%3A2023-01-01

• A blog post about vendoring: https://kerkour.com/rust-vendor-dependencies

• How Mozilla does it: https://mozilla.github.io/application-services/book/howtos/vendoring-into-mozilla-central.html

• A heated discussion on HN: https://news.ycombinator.com/item?id=26724833

[uncomfyhalomacro]

Hi. I know this issue is old but I wanted to know how --locked behaviour works with cargo vendor (or possibly even in general, as I might have misunderstood all of it). It seems to me that sometimes, if there is an update of a dependency with a compatible version, and a --locked flag is passed, the vendoring fails. Isn't the point of --locked to get the exact versions?

Asserts that the exact same dependencies and versions are used as when the existing Cargo.lock file was originally generated

Is this assertion about the versions mentioned in the Cargo.toml?

I have a feeling that --locked is just like an "if" condition and it will fail an operation if there is an

[...] attempt[ed] to change the lock file due to a different dependency resolution.

But that will update the lockfile right if I removed the --locked flag. Hence, it also updates the dependencies.

My thinking here is that if I don't want dependencies to be updated, should I just pass --locked or just omit it from the cargo vendor command? And if I want to update, I will just run cargo update before vendoring?

[weihanglo]

It seems to me that sometimes, if there is an update of a dependency with a compatible version, and a --locked flag is passed, the vendoring fails

If possible, would you mind sharing a reproduction and opening a new bug report? The --locked flag is to ensure lockfile, which the result of dependency resolution, stay the same. It does the same thing to cargo vendor as it does to other commands.

My thinking here is that if I don't want dependencies to be updated, should I just pass --locked or just omit it from the cargo vendor command? And if I want to update, I will just run cargo update before vendoring?

Pass --locked if you don't want to update dependencies ; yes.

[uncomfyhalomacro]

Oh! Thanks! I understood now. I think I don't need to file a bug report yet until i encounter one again. since errors usually happens if a project forgets to update their lockfile.

[jasonwilliams]

I would like to +1 this issue, I came across cargo vendor usage and wanted to read more about it so went to the docs. They were quite lackluster and just limited to the API rather than why you would use it or any more detail on use cases.

For example, if there's confusion of why vendor'd folders are big, it could be explained that all archs' and feature flags are going to be on (as vendor doesn't know ahead of time what is going to be used during build). That may have helped with this issue: nodejs/node#60897 (comment)

It would be great to see it filled out a bit