Conversation
|
|
||
| <a id="plumbing-commands"></a> | ||
|
|
||
| #### Plumbing commands |
There was a problem hiding this comment.
Plumbing commands are great, though OTOH I personally still like to see a more modularized cargo libs, especially for Cargo configurations part. They seems to be two different development directions. Not sure if they conflict.
There was a problem hiding this comment.
To clarify, breaking more crate out from the main libcargo.
There was a problem hiding this comment.
I think the refactors from plumbing commands will better enable splitting Cargo into libraries but I think the general problems remain
- complexity of abstracting away CLI assumptions
- further design constraints librification will enforce on us
- for the general case, plumbing commands should be the preferred route because you aren't locked into a specific toolchain version (e.g. if a project is using a new stable feature but their third-party tool uses cargo-the-library which thinkgs its unstable)
| If we added support for `cargo fix`, | ||
| then an agent can delegate to that, saving itself from calculating how to do the work to address lints. | ||
|
|
||
| One area I'm curious about in particular is what the role of [test coverage](#test-coverage) would have on Agents. |
There was a problem hiding this comment.
Not sure if you have thoughts on
- Formal verification or model checking.
- Fuzzing
More and more companies expand their test infra to those for AI-first workflows, though those may beyond Cargo's reach.
celinval
left a comment
There was a problem hiding this comment.
Thanks for putting this together. It's very enlightening.
| #### Adaptability | ||
|
|
||
| Cargo is [intended to be opinionated](https://doc.crates.io/contrib/design.html) | ||
| but between that and backwards compatibility, |
There was a problem hiding this comment.
How do you feel about eventual breaking changes to cargo, at least the CLI, for providing better functionalities? I think the LTS toolchain discussion fits in this a bit.
I totally understand the stability from the point of view that "Cargo strives to remain backwards compatible with projects created in previous versions.", i.e.: Cargo being able to virtually compile any crate that was ever published in crates.io.
I don't quite understand why cargo CLI cannot have breaking changes though. Does cargo need to strive to work with old CI and old scripts? Or is the concern here that build.rs scripts may invoke Cargo, hence, changing cargo would potentially break old crates?
There was a problem hiding this comment.
We need to expand some more on our compatibility surface.
I see CLI flags and json output as part of an API that is meant for integration and therefore has similar compatibility constraints as other API surfaces that ship together.
Is there an area of concern where you feel the CLI is holding things back?
There was a problem hiding this comment.
I wrote up some notes on compatibility at #t-cargo > Aggregating additional cargo design principles and decisions @ 💬
|
|
||
| In git, they divide operations into [plumbing and porcelain](https://git-scm.com/book/en/v2/Git-Internals-Plumbing-and-Porcelain). | ||
| Cargo is almost all porcelain commands with very limited programmatic use. | ||
| Adding plumbing commands and extending the programmatic output of the porcelain commands would go a long way to enabling people to adapt Cargo to meet their needs. |
There was a problem hiding this comment.
Do you think it's time for Cargo to embrace its role as a build system, or do you mostly see Cargo as an advanced package manager that has some build capabilities that needs better extension points for integration with more advanced build systems?
I.e.: Is the first line of the cargo book [1] precise?
Cargo is the Rust package manager.
There was a problem hiding this comment.
I just noticed that the Rust programming language book uses a different language[2] 😀:
Cargo is Rust’s build system and package manager.
There was a problem hiding this comment.
I think it depends on what you mean by those terms.
Currently, Cargo is an opinionated Rust build system with some escape hatches for practicality purposes (build scripts). It is not a general purpose build system. Personally, I worry the implications of Cargo becoming a general purpose build system but would like to better explore improved integration with general purpose build systems.
There was a problem hiding this comment.
That makes sense. The reason I asked the original question is that today, Cargo.toml and the build.rs contract shape what a Rust package is, and they're tightly coupled to how Cargo builds things. This means more scalable, general-purpose build systems like Nix, Bazel, and Buck2 have to maintain translation layers and patch build.rs scripts because there are assumptions that are deeply Cargo-specific, like the growing set of CARGO_* env vars, absolute paths, and cargo: directives.
This leaves me wondering: should the package manifest and build interface become more decoupled from Cargo's build semantics so other systems can consume them more naturally, or should Cargo itself scale up to offer hermetic builds, remote caching, better sandboxing, and the other things that push people toward those systems? I'd love to see the vision touch on this — or if there's a third path I'm not seeing!
There was a problem hiding this comment.
I would love to scale up Cargo with some caveats
- I'm referring to at least performance but not general purpose build system capabilities
- We likely won't get to "google scale" but there are still a lot of people in between that would benefit
- A focus on user problems and not features (e.g. "sandboxing" means a lot of things to a lot of different people, what problem are we solving?)
- We need to do so in a way that recognizes that not everyone will want or need this (e.g. linking against system libraries which is not hermetic)
That doesn't cover everything and it would be good to identify where we can improve with providing support to build systems. I have wondered about crate for parsing build script directives. Unsure what else could be worthwhile for bridging these gaps.
There was a problem hiding this comment.
Sounds good. Thanks for answering these questions. So scalable build system for Rust with better integration points for other build systems, is that right?
656a6cd to
eb40fca
Compare
| Maybe the OS can help with enforcement to create highlight these audit points. | ||
| Cargo should support a wide variety of OSs and they have different capabilities which may change with time, | ||
| likely making this be best-effort. | ||
| We will need to provide some way for build scripts and proc-macros to specify the resources they wish to access. | ||
| Maybe we could make this also work for tests and `cargo run` but it wouldn't help with deploying your production binary. | ||
|
|
||
| Maybe we go all the way to a wasm sandbox. | ||
| This is trivial for pure proc-macros though other ideas mentioned here will help. | ||
| The big challenge is supporting what is needed by impure proc-macros as well as build scripts. | ||
| Like with using OS enforcement, | ||
| we will need a way to specify what resources can be accessed. | ||
| Wasm sandboxing can likely lead to slower builds and not just because of having to run a wasm interpreter. | ||
| Cargo goes out of its way to try to allow proc-macros and build scripts to reuse intermediate build artifacts with a users application. | ||
| When we've previously benchmarked diverging these builds more for "build performance", | ||
| the results ended up being a wash because of the extra builds that happen. | ||
| This is also limited in what it helps; | ||
| it would not extend to your tests, `cargo run`, or your production binary. |
There was a problem hiding this comment.
@weihanglo do you have any feedback from your sandboxing experiments?
There was a problem hiding this comment.
When we've previously benchmarked diverging these builds more for "build performance",
I forgot. What was it? Should we link to those prior efforts?
And also we might want to look at from the angle of deterministic builds, which may help build cache reuse, though that alone could be a separate topic.
Generally looks good!
There was a problem hiding this comment.
I always have a hard time looking up those various performance changes.
The other ideas also help with deterministic builds.
0a22f9f to
0941893
Compare
| The output is much richer, allowing for better errors. | ||
|
|
||
| PubGrub has been proven out by Astral with uv, | ||
| including the ease of making changes like [changing how prerelease works](https://github.com/astral-sh/uv/releases/tag/0.12.0). |
There was a problem hiding this comment.
From my experience, pubgrub is great layer to build complex features with relative ease on top.
9fb6df4 to
6bbbe78
Compare
|
Done with the first draft |
| - Awareness of the `deprecated` feature | ||
| - Discovering of the migration guide | ||
| - The migrations are still manual | ||
| - `unstable-vX` features can have surprising effects and need more guard rails to scale up their use |
There was a problem hiding this comment.
I thought there were plans to stabilize#[unstable] attribute for regular crates [ref]. Do you know what happened to that?
There was a problem hiding this comment.
No idea but it was discussed yet again at the all hands, see https://github.com/epage/epage.github.io/blob/cargo/blog/blog/cargo-vision.md#api-lifecycle
| Maybe Cargo can use OS features to restrict build script and proc-macros access as a way to highlight these audit points. | ||
| Cargo should support a wide variety of OSs and they have different capabilities which may change with time, | ||
| likely making enforcement be best-effort. | ||
| We will need to provide some way for build scripts and proc-macros to specify the resources they wish to access. |
There was a problem hiding this comment.
Yes, please! I get that this can be painful for crate developers to maintain, but hopefully we can add tooling to help them detect things they are missing, and also provide a reasonable default for things they can access.
Sound test caching would also need something along those lines.
194860f to
83a3219
Compare
Tasks:
Rendered