When is Rust going to have a stable distro experience? [LWN.net]

When is Rust going to have a stable distro experience?

Posted Jun 26, 2025 11:42 UTC (Thu) by khim (subscriber, #9252)
In reply to: When is Rust going to have a stable distro experience? by NYKevin
Parent article: Getting extensions to work with free-threaded Python

> I think Kamilion wants to download a precompiled openssh library instead of building it from source on their workstation, and is complaining that such a package is de facto impossible to provide because the ABI is unstable.

But that's not the problem if Rust ABI: these packages don't expose anything with Rust ABI.

What you need for such an ability is an SDK: package oriented for developers that allows one to prepare binaries for your OS, preferably covering large range of OSes.

All popular OSes today do that, but in a Unix world the story was always the exact opposite: each version of OS introduced new version of packages and developers of software were spending tremendous effort to deliver anything (from object files and a linker script to autoconf and other such mess).

Rust (and most other modern languages) have rejected that madness and simply ask one to provide a new enough version of Rust: rustc 1.74.0 was released 1.5 years ago!

It's entirely not clear why developers of all programs (millions of them) have to deal with old, long-obsolete versions of tools simply because small group of OS developers (thousands, for most distros, often less) couldn't do what they needed to do ages ago and finally simply provide an SDK for their OS.

to post comments

When is Rust going to have a stable distro experience?

Posted Jun 26, 2025 13:32 UTC (Thu) by pizza (subscriber, #46) [Link] (20 responses)

> Rust (and most other modern languages) have rejected that madness and simply ask one to provide a new enough version of Rust: rustc 1.74.0 was released 1.5 years ago!

> It's entirely not clear why developers of all programs (millions of them) have to deal with old, long-obsolete versions of tools

....1.5 years old is considered "old, long-obsolete" ?

I am working in an industry where the *minimum* (As mandated by law) support lifecycle is approximately *ten years* after final sale, and it typically takes several years of R&D to get a product on the market in the first place.

When is Rust going to have a stable distro experience?

Posted Jun 26, 2025 13:56 UTC (Thu) by khim (subscriber, #9252) [Link] (2 responses)

> I am working in an industry where the *minimum* (As mandated by law) support lifecycle is approximately *ten years* after final sale, and it typically takes several years of R&D to get a product on the market in the first place.

Does the law mandate the ability to use the exact same piece of software that was released on the day device was sold to you – together with all the latest and greatest features that are supported in a new versions?

Because that is what we are talking about here. Not about support time (Rust is perfectly supported and works very well today, it's just old version of Rust that are not supported).

> ....1.5 years old is considered "old, long-obsolete" ?

It's considered “superseded”. At my $DAYJOB we also have more than 10 years of support times – and yet we don't bother to support the ability to compile our code with a compiler that's more than half-year old.

If device couldn't be used with the “top of the tree” then we have separate branch for it which includes old version of everything: old version of compiler, old versions of support libraries, etc.

And it doesn't have anything to do with a stable ABI: we also provide SDK to customers and these are supporting old versions of already released devices, too.

But we don't support superseded versions of SDK. You can use them on your own risk, if you want, but if you want to get support – you have to use the few latest versions.

P.S. I wonder where the conflation of “something is supported for X years” and “something may be used for X years without software upgrade” comes from. People often conflate these even if they are clearly very different.

When is Rust going to have a stable distro experience?

Posted Jun 26, 2025 14:22 UTC (Thu) by pizza (subscriber, #46) [Link] (1 responses)

> Does the law mandate the ability to use the exact same piece of software that was released on the day device was sold to you – together with all the latest and greatest features that are supported in a new versions?

No. It mandates that whatever has shipped has gone through extensive certification processes. Changes are permitted but each (no matter how minor) has to be separately justified, documented, and extensively tested.

...As the saying goes, safety regulations are written in blood.

When is Rust going to have a stable distro experience?

Posted Jun 26, 2025 14:34 UTC (Thu) by khim (subscriber, #9252) [Link]

> Changes are permitted but each (no matter how minor) has to be separately justified, documented, and extensively tested.

Which corresponds to the “frozen branch, no new changes allowed” model. There are no chance whatsoever to drop brand new cryptography package into that mix.

> …As the saying goes, safety regulations are written in blood.

Yes, but that doesn't mean that once written they have to stay unchanged.

But as I have said that's not really relevant to the issue: if you don't need support then you may use old versions of Rust just fine, even pre-1.0 versions.

You just have to accept the code for everything else from that time, too.

Sometimes the ability to mix and match is deemed to valuable enough to support – examples include Linux kernel (but not CADT-infested distros), Windows, Android and many others… but they, notably, also don't mandate the ability to use old tools with “latest and greatest” codebase.

That's something UNIX did, essentially “because it could” – and then GNU/Linux inherited.

No one else does that.

When is Rust going to have a stable distro experience?

Posted Jun 27, 2025 7:38 UTC (Fri) by taladar (subscriber, #68407) [Link] (16 responses)

What many people miss is that Rust is not like other compilers. The current Rust compiler is perfectly capable of compiling anything Rust 1.74.0 could compile so there is no reason to have LTS versions of old compiler branches.

Obviously there can be occasionaly bugs but the same is true for old branches with security fixes and/or backports.

When is Rust going to have a stable distro experience?

Posted Jun 27, 2025 11:48 UTC (Fri) by pizza (subscriber, #46) [Link] (10 responses)

> Obviously there can be occasionaly bugs but the same is true for old branches with security fixes and/or backports.

In other words, "reason #83 why you don't make _any_ unnecessary changes, which includes swapping out the underlying toolchain"

It's not enough to say "oh, the toolchain is perfectly capable of compiling the old code", you have to *prove* that it does so correctly without introducing any unintentional changes.

When is Rust going to have a stable distro experience?

Posted Jun 27, 2025 12:18 UTC (Fri) by khim (subscriber, #9252) [Link] (2 responses)

> In other words, "reason #83 why you don't make _any_ unnecessary changes, which includes swapping out the underlying toolchain"

In that approach you would never face compatibility issues, because the reason that stop you from upgrading toolchain would also stop you from upgrading any other code, too.

Again: we are not talking about the case of “everything is frozen, every single line of change need justification”. In that mode compatibility issues don't exist simply you don't touch or change anything.

> It's not enough to say "oh, the toolchain is perfectly capable of compiling the old code", you have to *prove* that it does so correctly without introducing any unintentional changes.

Not if you already decided “to take the plunge” and upgrade some major component of your stack, be it python version or cryptography python package version.

At that point you are playing with probabilities – and, in practice, chance of regressions from upgrade of some library supported by one person in their spare time or python version, which break compatibility regularly, are much higher than chances of regressions from rust compiler upgrade.

When is Rust going to have a stable distro experience?

Posted Jun 28, 2025 21:49 UTC (Sat) by SLi (subscriber, #53131) [Link] (1 responses)

> In that approach you would never face compatibility issues, because the reason that stop you from upgrading toolchain would also stop you from upgrading any other code, too.

> Again: we are not talking about the case of “everything is frozen, every single line of change need justification”. In that mode compatibility issues don't exist simply you don't touch or change anything.

I think that's a false dichotomy. There are changes that are allowed or mandated. They are things like "we have discovered what caused the plane to explode, and we'll fix that". Then you will fix that, and _only_ that.

Would you expect something the magnitude of a Linux kernel to stay bug compatible to an old version if you swapped the toolchain used to build it?

Upgrading the toolchain is not even on the top 1000 list of things to do. Every developer has their pet peeve, but you don't even fix a typo in an error message because how do you know it won't break something somewhere, and you don't want to do typically O(weeks) of paperwork to justify it (more so for bigger changes, like changing a compiler flag used to compile one of the functions).

It's just not at all the same world where "just update the tools" is feasible.

When is Rust going to have a stable distro experience?

Posted Jun 28, 2025 22:02 UTC (Sat) by khim (subscriber, #9252) [Link]

> Then you will fix that, and _only_ that.

Means: you don't bring new version of Python, you don't bring new version cryptography and that means you don't need new version of Rust, either… where's promised false dichotomy?

> Would you expect something the magnitude of a Linux kernel to stay bug compatible to an old version if you swapped the toolchain used to build it?

Why, yes, of course. It's in the same position as everything else: on ToT branch it's built with latest approved version of clang and on frozen “security fixed only” branch it's compiled with the same toolchain that was used when that branch was cut.

> It's just not at all the same world where "just update the tools" is feasible.

Yes, it is same world. Android AOSP build is updated every quarter and that update very much includes update to the toolchain, too. Different vendors may update their branches on a different cadence, sure, but situation where you may upgrade new version of Python or bring “latest and greatest” version cryptography package but, for some strange reason, couldn't upgrade toolchain is simply not in the cards.

These are similarly disruptive operations (I would even say that upgrade of python is much more disruptive, potentially, because rust incompatibilities tend to break compilation of your code while python upgrade often can break things that your test may miss), why should they be treated differently?

When is Rust going to have a stable distro experience?

Posted Jun 30, 2025 10:22 UTC (Mon) by taladar (subscriber, #68407) [Link] (6 responses)

Actually 90% of the bugs are bugs in the old toolchain that the new one compiles correctly but the old one did not.

When is Rust going to have a stable distro experience?

Posted Jun 30, 2025 10:53 UTC (Mon) by pizza (subscriber, #46) [Link] (5 responses)

> Actually 90% of the bugs are bugs in the old toolchain that the new one compiles correctly but the old one did not.

...And what about the other 10%?

Put that into your paperwork and see how far that gets you with $regulator.

(Remember, "known bugs" can be worked around or otherwise handled. It's those unknown bugs that might literally kill someone if encountered)

When is Rust going to have a stable distro experience?

Posted Jun 30, 2025 13:20 UTC (Mon) by farnz (subscriber, #17727) [Link] (2 responses)

In the limit, this is "don't change anything", because any change can bring in unknown bugs; in practice rather than take a new mixed feature + bugfix upgrade to (say) PyCryptography (which includes a minimum required toolchain version bump), you'd be carefully following your documented process for reimplementing fixes that you need in the older dependency's source, without bumping the toolchain.

By reimplementing each bugfix yourself, following a procedure that minimises risk, you avoid the challenges inherent in taking all the changes upstream has introduced between two versions; some of those are going to turn out to be relevant bugfixes, but others will be that nasty combination of "this change did not improve anything for us" and "this change also introduced a new, previously unknown, bug".

Ultimately, it all comes back down to "who takes responsibility for things going wrong"; it's unreasonable to expect volunteer maintainers to accept responsibility for anything involving safety or money, and thus someone has to take that responsibility on. People like Red Hat, IAR Systems, Montavista, Wind River Systems (and many more, both proprietary like IAR Systems, and open source like Red Hat), along with insurers to limit your exposure to liability, have built their business on taking some responsibility for bugs in the code - and if you're in a position where bugs can be life-and-death, you'd be well advised to at least ensure that the insurance premiums to take on your liability are affordable (even if you don't actually take out insurance).

After all, even if you do use a qualified compiler like Ferrocene, or the Arm Compiler for Embedded FuSa, you still have to ensure that you only use qualified features of the language; you can't just compile arbitrary code with a qualified compiler, and assume that it now meets standards.

When is Rust going to have a stable distro experience?

Posted Jul 1, 2025 8:51 UTC (Tue) by taladar (subscriber, #68407) [Link] (1 responses)

> By reimplementing each bugfix yourself, following a procedure that minimises risk

Who does that in practice though? Because having one developer backport fixes for dozens or hundreds of projects certainly is not "following a procedure that minimises risk". And that is what pretty much all those who advocate for minimal changes (as opposed to actually zero changes which is largely impractical in most domains) are doing.

When is Rust going to have a stable distro experience?

Posted Jul 1, 2025 10:55 UTC (Tue) by farnz (subscriber, #17727) [Link]

It's a hard requirement in some safety-critical standards; you lock dependency versions down at an early phase in development, and from then on in, you have to show that every change to a dependency is done in accordance with your policy (which has to meet the standard's requirements), or start the certification process from the beginning. Such a policy will require multiple developers involved in a backport, not just one developer.

But this is one significant way in which safety-critical software (IEC 61508 compliant, for example) differs from your run of the mill software for laptops, desktops etc. And, just as you can't switch from GCC to a qualified compiler and expect your software to magically be compliant with safety standards, you can't just take the "easy" bits of safety standards (like insisting on backported fixes, without the procedure to ensure that there's two developers communicating about the fix and one not in communication with the fixer and their reviewer, auditing the resulting code for compliance) and expect good outcomes.

When is Rust going to have a stable distro experience?

Posted Jul 1, 2025 8:48 UTC (Tue) by taladar (subscriber, #68407) [Link] (1 responses)

So how exactly do you work around known bugs that mis-compile something that then fails at runtime because some optimization went awry?

When is Rust going to have a stable distro experience?

Posted Jul 1, 2025 10:09 UTC (Tue) by farnz (subscriber, #17727) [Link]

It depends whether you're using a qualified compiler or not.

If you're using a qualified compiler, and your code meets the compiler's qualification requirements, you notify your compiler vendor, and they will, per your contract with them, supply a bug fix along with the necessary paperwork for your certification to update to this new compiler. That paperwork is rather arduous, but confirms that (a) the bug was in the compiler's handling of a qualified construct, (b) that this bug fix resolves that, and (c) that this bug fix does not break the compiler's handling of other qualified constructs.

If you're not using a qualified compiler, then you change the source code or build options, until the output is what you want, without the miscompilation.

Note that because this is the safety critical world, you must audit the system for compliance with the standards; a qualified compiler allows you to audit in the source code world, any other compiler requires you to audit the binary output.

When is Rust going to have a stable distro experience?

Posted Jun 27, 2025 12:19 UTC (Fri) by laarmen (subscriber, #63948) [Link] (4 responses)

> The current Rust compiler is perfectly capable of compiling anything Rust 1.74.0 could compile so there is no reason to have LTS versions of old compiler branches.

This statement is wrong for a couple of reasons. First, there is the odd project that uses unstable features, thus relying on semantics that are by definition subject to change. The rust-for-linux project is (was?) one of those. If you work on the kernel you definitely want to keep older versions of the toolchain around. rustc itself is also one of those, btw. Building an older version of rustc with a newer one doesn't work.

Second, it's actually pretty easy to find an example of code that was fine on 1.74 and doesn't compile today. The 1.80 release famously broke any version of the time crate that was older than 3 months at the time.

I actually think that was fine, although annoying. When you have things like type inference, autoderef, lifetime inference, etc... you're bound to break edge-cases whenever you touch the algorithms, and the resulting QoL changes were probably worth the breakage. However, I do wish people would be more nuanced when mentioning the backwards compatibility story of Rust.

When is Rust going to have a stable distro experience?

Posted Jun 27, 2025 13:12 UTC (Fri) by daroc (editor, #160859) [Link] (3 responses)

The Rust-for-Linux project does still depend on some unstable features, but reducing the number is a major priority for the Rust project, and one that they have made some decent progress on. Most of the remaining items are build flags, not language features. I would not be surprised if the kernel builds with stable Rust this time next year.

When is Rust going to have a stable distro experience?

Posted Jun 27, 2025 14:49 UTC (Fri) by ojeda (subscriber, #143370) [Link]

Yeah, we rely on essentially 2 language features at the moment, which are being worked on upstream to stabilize them.

Nevertheless, to clarify on top of what Daroc said, we already support stable Rust releases >= 1.78 (i.e. more than a year of releases at the moment).

In other words, from the kernel side we are able to support several versions of the compiler even if some details here and there may happen to change. It is essentially the same as supporting C flags or attributes conditionally, or having a workaround for a compiler bug.

When is Rust going to have a stable distro experience?

Posted Jun 27, 2025 14:57 UTC (Fri) by archaic (subscriber, #111970) [Link] (1 responses)

Can we use this as one of Jon's famous year-end predictions? :)

When is Rust going to have a stable distro experience?

Posted Jun 27, 2025 15:04 UTC (Fri) by daroc (editor, #160859) [Link]

He asks for suggestions from the rest of us when putting those articles together, so I'll certainly throw this one into the ring. Of course, by December we might have a clearer idea of exactly where things will stand.

When is Rust going to have a stable distro experience?

Posted Jun 26, 2025 18:04 UTC (Thu) by NYKevin (subscriber, #129325) [Link] (1 responses)

> Rust (and most other modern languages) have rejected that madness and simply ask one to provide a new enough version of Rust: rustc 1.74.0 was released 1.5 years ago!

This story is about Python, so I focused on the Python-specific issues, which are very different to Rust's packaging situation.

Python, unlike Rust, actually does want to distribute precompiled binaries as the "standard" form of its (non-Python, native code) libraries. But this turns out to be somewhere between difficult and impossible on Linux, unless you either reinvent Flatpak (i.e. vendor everything and stick it in a container) or reinvent Conda (i.e. extend the scope of your packaging repository until it's most of the way to being an independent Linux distro). Since the Python folks can't even agree on which tool we should invoke to manage packages, those are both complete non-starters (as standard, language-level solutions - obviously both Flatpak and Conda do exist and can be used for this purpose). The result is that source distributions are the de facto standard way to package native code Python extensions, and binary wheels mostly only exist as an optimization for people who conveniently happen to have exactly the right distro etc. on their systems already.

This is not a problem on Windows or Mac for two reasons:

1. They have richer and more elaborate OS-level APIs than just libc. Most distros also provide libraries that are not libc, but if you're packaging for "Linux," you don't know the distro in advance and can't rely on anything that is not libc (or Python.h, of course).
2. If you need something the OS doesn't provide, you can just vendor it and nobody will yell at you.

When is Rust going to have a stable distro experience?

Posted Jul 4, 2025 3:59 UTC (Fri) by zahlman (guest, #175387) [Link]

> Since the Python folks can't even agree on which tool we should invoke to manage packages

Speaking as "Python folk" (and generally outside of the world of non-Python dependencies), I *actively don't want* such agreement. I instead want the community to keep doing what it's doing with developing interoperable standards. Having a standard choice for the package manager goes against my mental model of how FOSS works. In that world, problems in that package manager can only be addressed by joining that team (and submitting to its contributor agreements etc.) or by making a fork that everyone has an explicit vested interest in ignoring. And anyway if we were going to do things that way, the effort should have started in 2011 when the Conda people first complained. And that, in turn, would likely have made it even harder to get away from all the ugly issues with `setup.py` and harder to move towards using it purely for build orchestration.

> and binary wheels mostly only exist as an optimization for people who conveniently happen to have exactly the right distro etc. on their systems already.

Manylinux (https://github.com/pypa/manylinux) compatibility is pretty broad, honestly. Especially if you build wheels for multiple manylinux tags. When I switched over to Linux it never even occurred to me to wonder whether Numpy wheels would work. They did.

> They have richer and more elaborate OS-level APIs than just libc. Most distros also provide libraries that are not libc, but if you're packaging for "Linux," you don't know the distro in advance and can't rely on anything that is not libc (or Python.h, of course).

There is musl support now, too. But from what I can see, the most popular packages don't find themselves needing to rely on things outside the standard library (well, I guess a handful of projects build *against Numpy's* ABI).