|
|
Log in / Subscribe / Register

Large language models for patch review

We're bad at marketing

We can admit it, marketing is not our strong suit. Our strength is writing the kind of articles that developers, administrators, and free-software supporters depend on to know what is going on in the Linux world. Please subscribe today to help us keep doing that, and so we don’t have to get good at marketing.

By Jonathan Corbet
October 16, 2025
There have been many discussions in the free-software community about the role of large language models (LLMs) in software development. For the most part, though, those conversations have focused on whether projects should be accepting code output by those models, and under what conditions. But there are other ways in which these systems might participate in the development process. Chris Mason recently started a discussion on the Kernel Summit discussion list about how these models can be used to review patches, rather than create them.

Mason's focus was on how LLMs might reduce the load on kernel maintainers by catching errors before they hit the mailing lists, and by helping contributors increase the quality of their submissions. To that end, he has put together a set of prompts that will produce reviews in a format that maintainers are used to: "The reviews are meant to look like emails on lkml, and even when wildly wrong they definitely succeed there". He included a long list of sample reviews, some of which hit the mark and others of which did not.

The prompts are interesting in their own right; they can be seen as constituting the sort of comprehensive patch-review documentation that nobody ever quite got around to writing for humans to use. Perhaps that reflects a higher level of confidence that the LLM will actually read all of this material. These prompts add up to thousands of lines of material, starting with core guidance like:

Struct changes → verify all users use the new struct correctly

Public API changes → verify documentation updates [...]

Tone Requirements:

  • Conversational: Target kernel experts, not beginners
  • Factual: No drama, just technical observations
  • Questions: Frame as questions about the code, not accusations

Most of the prompts consist of guidance specific to subsystems like locking ("You're not smart enough to understand smp_mb(), smp_rmb(), or smp_wmb() bugs yet") and networking ("Socket can outlive its file descriptor"). All told, it resembles the sort of rule collection one saw in the expert systems that were going to take over the world in the 1980s. As noted in the README file, "the false positive rate is pretty high right now, at ~50%", so there is still some room for improvement.

In the ensuing discussion, nobody seemed to think that using LLMs in this way was a bad idea. Sasha Levin called it "a really great subject to discuss", and said that, in the previous discussions on LLM use by kernel developers, the concerns that were raised about LLMs drowned out out any attempt to find the places where they could be useful. Paul McKenney remarked that using this technology to review code written by others "seems much safer than using it to generate actual code". Krzysztof Kozlowski noted that Qualcomm has created a similar system and made it available.

There were some concerns raised about the proprietary nature of these systems; Konstantin Ryabitsev was just one of a few who drew parallels with the BitKeeper experience that (briefly) brought kernel development to a halt just over 20 years ago. Laurent Pinchart stated clearly that there are limits to how much proprietary tools can be used or required:

Forcing contributors to pay for access to proprietary tools is not acceptable. Forcing contributors to even run proprietary tools is not acceptable.

He also expressed concerns that the companies behind LLMs would make them available to developers for free to encourage adoption — until the community is well locked in, at which point access could quickly become expensive. Mason, though, was unworried about lock-in, saying that the prompts are sufficiently generic to be adaptable to any system. James Bottomley suggested that LLMs would not be proprietary forever, but Pinchart argued against relying on proprietary software in the hope that there will eventually be free alternatives.

There was some disagreement over who an LLM-based review tool should be created for. Mason's target was maintainers, but Andrew Lunn argued that the plan should be for developers to run these tools themselves before posting code for review. That, he said, would further reduce the workload on maintainers, who would only need to run LLM review to verify the the submitter had already done so.

Pinchart, along with others, pointed out that getting developers to use the tools (such as checkpatch.pl) that exist now is difficult; he wondered how submitters could be encouraged to run any new tools. Tim Bird suggested annotating patches with a list of the tools that have been run on them so that maintainers could see that history. Bottomley, instead, said that these tools should be run automatically on patches sent to the mailing lists, much like the checks that the 0day robot runs on posted patches now. Bird, though, said that running the tools should be expected of submitters. "It then becomes a cost for the contributor instead of the upstream community, which is going to scale better."

Mason was clear in his belief that LLM-generated reviews should happen in public as part of the submission process:

I think it's also important to remember that AI is sometimes wildly wrong. Having the reviews show up on a list where more established developers can call bullshit definitely helps protect against wasting people's time.

Linus Torvalds, in his one contribution to the discussion, agreed. He was about the only one to express concerns about the technology, saying "I think we've all seen the garbage end of AI, and how it can generate more work rather than less". Mason agreed that Torvalds's concerns were relevant, based on his own experience:

My first prompts told AI to assume the patches had bugs, and it would consistently just invent bugs. That's not the end of the world, but the explanations are always convincing enough that you'd waste a bunch of time tracking it down.

Torvalds mentioned the scraper problem as well. His concerns notwithstanding, he believes that this technology will prove helpful, but he feels that its initial adoption has to be aimed at making life easier for maintainers. "So I think that only once any AI tools are actively helping maintainers in a day-to-day workflow should people even *look* at having non-maintainers use them".

The conversation wound down shortly after that. One clear conclusion, though, is that these tools seem destined to play an increasing role in the kernel-development process. At some point, we will likely start seeing machine-generated reviews showing up on the mailing lists; then, perhaps, the real value of LLM-based patch-review tools will start to become clear. It will be interesting to see how the inevitable related discussion at the 2025 Maintainer Summit in December plays out.

Index entries for this article
KernelDevelopment tools/Large language models


to post comments

What about false negatives?

Posted Oct 16, 2025 16:42 UTC (Thu) by alx.manpages (subscriber, #145117) [Link] (24 responses)

I see people talking about false positives. How about false negatives, and the false sense of confidence and safety that they can give?

FWIW, I'm working on a restrictive policy for AI that bans almost every use of AI, including uses for static analysis, etc.

<https://lore.kernel.org/linux-man/rokuuw5hmdzggovnzbrxgjv...>

What about false negatives?

Posted Oct 16, 2025 17:11 UTC (Thu) by intelfx (subscriber, #130118) [Link] (23 responses)

> This also includes AI assistive tools used in the contributing process, even if such tools do not generate the contributed code.

You are entering the SSPL territory here. No "contribution policy" can *ever* be the business of controlling what else is happening on my computer while I'm writing code.

Do you intend to somehow ban people from using predictive/AI-assisted local search on their computers (like the ones included and enabled by default in commercial OSes)? Or perhaps AI-assisted summarizations (like the ones that appear automatically when you're searching something with Google)? If I wanted to comply with your policy, would I have to close my eyes?

Good luck with that. And I'm saying this as someone who is skeptical of GenAI.

What about false negatives?

Posted Oct 16, 2025 21:29 UTC (Thu) by alx.manpages (subscriber, #145117) [Link] (22 responses)

> No "contribution policy" can *ever* be the business of controlling what else is happening on my computer while I'm writing code.

I don't mind what you do on your computer, as long as it doesn't _influence_ the quality of the contributions.

> Do you intend to somehow ban people from using predictive/AI-assisted local search on their computers (like the ones included and enabled by default in commercial OSes)? Or perhaps AI-assisted summarizations (like the ones that appear automatically when you're searching something with Google)?

I'd like people to not use any of these AI tools to contribute. If the tools spam you against your own will, but you are able to ignore the spam, then I consider that to be complying.

> If I wanted to comply with your policy, would I have to close my eyes?

If you can't avoid being spammed with the mis-information, maybe because browsers have it even if you don't want it, you should at least not use the information. I think that would comply.

What about false negatives?

Posted Oct 17, 2025 5:38 UTC (Fri) by drago01 (subscriber, #50715) [Link] (15 responses)

> I don't mind what you do on your computer, as long as it doesn't _influence_ the quality of the contributions.

Well you assume or imply that the end result will always have lower quality. In practice it depends how on which tools are used. Banning tools outright out of fear is not really a solution. There is still a human using those tools, and that person is responsible for the outcome.

What about false negatives?

Posted Oct 17, 2025 9:57 UTC (Fri) by alx.manpages (subscriber, #145117) [Link] (14 responses)

> Well you assume or imply that the end result will always have lower quality.

Yes, that's my assumption.

> In practice it depends how on which tools are used.

We need more evidence before that can be claimed solidly.

> Banning tools outright out of fear is not really a solution.

I think it is. What you call fear, I call caution.

What about false negatives?

Posted Oct 18, 2025 16:53 UTC (Sat) by ojeda (subscriber, #143370) [Link] (10 responses)

> Yes, that's my assumption.

It is a broken assumption, and a counterexample is trivial: someone runs a tool that finds an issue with a patch, and the author fixes it before submitting.

So, no, the end result will not "always have lower quality".

And note how that applies regardless of whether the tool in question uses AI or not. It could be `checkpatch.pl` I was thinking about.

In practice, applying more tools generally means better results, just like more reviewers and more review time helps. And someone's fancy AI tool can definitely find issues sometimes that other tools may not. Of course, there are costs to running many tools, diminishing returns, etc. But that is a different discussion.

What about false negatives?

Posted Oct 18, 2025 17:27 UTC (Sat) by alx.manpages (subscriber, #145117) [Link] (9 responses)

> It is a broken assumption, and a counterexample is trivial: someone runs a tool that finds an issue with a patch, and the author fixes it before submitting.

I remain unconvinced. Here are a few counter-counterexamples:

Someone runs an AI tool that finds a false positive. The AI fools the programmer to believe it is valid, and results in the introduction of a bug instead of a fix. AI tools can fool humans easier than regular tools.

> Of course, there are costs to running many tools, diminishing returns, etc. But that is a different discussion.

I think it's part of the same discussion. Let's go bad to my first post: what about false negatives? Running an AI tool might lead a programmer to be more convinced that the patch is good, and thus less prone to running other tools, or asking other humans to review a patch. Those other tools or humans would probably do a better job.

Should one run 10 regular tools vs 5 regular tools and an AI tool?

It would only make sense to run the AI tool if you've already run *all* of the existing regular tools *and* also asked all of the humans that could help. But at that point, we're deep into diminishing returns.

And there's still the possibility that the AI tool might fool you into breaking a good patch, breaking what those humans had reviewed.

What about false negatives?

Posted Oct 18, 2025 19:06 UTC (Sat) by ojeda (subscriber, #143370) [Link] (8 responses)

> Here are a few counter-counterexamples

Those are not counterexamples -- you said "always", not "sometimes".

And even if you meant "most of the time", if there is a tool so bad that it is wrong often enough and, on top of that, manages to misdirect authors often enough, then it will simply stop being used sooner or later. That applies to all tools, not just AI-based ones. We could be talking about a bad compiler warning that gets disabled, for instance.

> Running an AI tool might lead a programmer to be more convinced that the patch is good, and thus less prone to running other tools, or asking other humans to review a patch. Those other tools or humans would probably do a better job.

You could argue the same about running `checkpatch.pl`, or a static analyzer, or even warnings in your compiler, or even using a safer language...

That doesn't mean every reviewing tool should be banned.

> *and* also asked all of the humans that could help

That doesn't apply in the case I mentioned: someone reviewing their patch before submitting it.

And your policy bans even that.

> But at that point, we're deep into diminishing returns.

Not really. Tools are fairly different from one another, even in the "regular" set you mention, and LLMs are quite different from the "usual" tools. It doesn't even need to be about reviewing code.

What about false negatives?

Posted Oct 18, 2025 19:45 UTC (Sat) by alx.manpages (subscriber, #145117) [Link] (7 responses)

> Those are not counterexamples -- you said "always", not "sometimes".

A liar may sometimes tell the truth. The problem is that if it sometimes can be a lie, you need to consider it *always* as if it were a lie. The quality of a contribution doesn't exist in a vacuum, and the possibility of it being a hallucination is already lowering the quality, even if a given instance is actually good.

What about false negatives?

Posted Oct 18, 2025 20:03 UTC (Sat) by mb (subscriber, #50428) [Link] (5 responses)

>if it sometimes can be a lie, you need to consider it *always* as if it were a lie

I think that this conclusion doesn't make any sense at all.
By the same reasoning you would have to make compiler warnings being 100% correct all the time or ban the use of the compiler that gives you false warnings (every compiler).

What about false negatives?

Posted Oct 18, 2025 21:06 UTC (Sat) by alx.manpages (subscriber, #145117) [Link] (4 responses)

There's a big difference between false positives or negatives in a compiler, which are deterministic. You can reproduce them, and know when the compiler has you covered or not. Also those false Ps and Ns almost always decrease, with regressions being rare (but again, when they exist, at least you can reproduce them and file a bug).

With an AI tool, there are no rules. You may have the same exact chat with the tool twice, with different results.

I'm amazed by how few people seem to value determinism in tools.

What about false negatives?

Posted Oct 18, 2025 21:09 UTC (Sat) by mb (subscriber, #50428) [Link] (3 responses)

You are moving the goal posts.

What about false negatives?

Posted Oct 18, 2025 21:34 UTC (Sat) by alx.manpages (subscriber, #145117) [Link] (2 responses)

> You are moving the goal posts.

I don't think so. I've consistently meant that it's because you don't know when AI lies to you that you must assume it always lies to you. If I know when you're going to lie to me, I know when to ignore you, and thus you're going to have a hard work lying to me.

Maybe I wasn't explicit enough here (I tend to not repeat _everything_ every time I discuss a topic; I assume people can follow links, and also use common sense), but I've said this before in the mailing list where the policy is being worked on. I've also argued this point elsewhere when debating AI tools.

And that was the rationale for the following paragraph in the policy, in the first place:

+ AI tools should be considered adversarial, as if they
+ were a black box with Jia Tan inside them.

What about false negatives?

Posted Oct 18, 2025 21:42 UTC (Sat) by mb (subscriber, #50428) [Link] (1 responses)

>you don't know when AI lies to you that you must assume it always lies to you

Therefore, you must assume any classic tool lies to you.
A compiler gives you false positive warnings so you can't know whether a warning is real or not.

What about false negatives?

Posted Oct 20, 2025 8:45 UTC (Mon) by taladar (subscriber, #68407) [Link]

But the compiler or other classic analysis tool always gives you the false positives for the same type of edge case, the AI tool is completely random in when it hallucinates.

What about false negatives?

Posted Oct 18, 2025 21:01 UTC (Sat) by ojeda (subscriber, #143370) [Link]

A reviewing tool being wrong sometimes (AI or not) doesn't mean a patch gets always automatically modified. Even if a patch gets modified in a wrong way at some points, it doesn't follow that on average it is all a net negative. What usually happens is that tools get disabled if they are not worth it, but you haven't shown that.

And even if you are talking about second order effects like "it wasted time I could have used in a better reviewing tool", you would still need to show which ones are the better tools. And even if one tool is way better than another, it doesn't automatically follow that it isn't worth it to run both. And so on and so forth.

So, no, your logic doesn't follow, sorry.

And it is not just reviewing that your policy bans. The policy casts such a wide net (banning even tools "in the contributing process" that do not generate code) that people cannot even report bugs found with those tools when applied over the codebase. So, for instance, an external team that runs such tools for other projects will need to skip your repository, even if they manually curate the results and so on.

What about false negatives?

Posted Oct 20, 2025 4:14 UTC (Mon) by drago01 (subscriber, #50715) [Link] (2 responses)

> Yes, that's my assumption.

Others already answered that below. But that makes zero sense.

You even assume that there is no innovation in that area, tools do improve (actually quite fast due to healthy competition) - so even if the quality isn't good enough today, tomorrow's AI tools may be much better. You are trying to outright ban a technology.

What about false negatives?

Posted Oct 20, 2025 8:47 UTC (Mon) by taladar (subscriber, #68407) [Link] (1 responses)

AI tools have literally been stagnant since GPT-3 was first introduced on the issue of hallucinations.

What about false negatives?

Posted Oct 21, 2025 5:38 UTC (Tue) by comex (subscriber, #71521) [Link]

That is not true. For any given task, hallucination rates have decreased exponentially over time, as shown by benchmarks and confirmed by my own limited experience with chatbots. But the tasks we expect LLMs to complete have likewise increased in difficulty, shooting the hallucination rate back up.

To be fair, a large amount of this improvement has simply come from newer models being able to answer harder questions. LLMs have always been much more likely to hallucinate when they don’t know the answer to a question, so knowing more answers reduces hallucinations ‘for free’. What’s harder for models is admitting when they don’t know or can’t solve something. They have improved on this front too, but the improvement has been slower. And if you’re waiting for a breakthrough that brings hallucinations to human-like low rates, that has indeed yet to be found.

We will see how things develop moving forward.

What about false negatives?

Posted Oct 18, 2025 11:27 UTC (Sat) by intelfx (subscriber, #130118) [Link] (5 responses)

> I don't mind what you do on your computer, as long as it doesn't _influence_ the quality of the contributions.

All of this is none of your business.

You can judge my contribution on its merits (or not at all). That's all you can do. What I do in my life is not your concern, even if it can hypothetically impact the quality of said contribution.

What about false negatives?

Posted Oct 18, 2025 16:17 UTC (Sat) by alx.manpages (subscriber, #145117) [Link] (4 responses)

There's no legal obligation to comply. It's a contribution guideline, not a license. I'm stating that I don't want to receive contributions of low quality, and I consider submissions created with any help of AI to be of low quality.

It's the same as a guideline saying you must run X, Y, and Z tests before submitting. A project can't force you to comply, but it's not forced either to accept the submissions.

What about false negatives?

Posted Oct 18, 2025 17:18 UTC (Sat) by mb (subscriber, #50428) [Link] (3 responses)

>A project can't force you to comply, but it's not forced either to accept the submissions.

True.
And to actually reject such a submission, you need to run X, Y, and Z tests on *your* side to find out whether you have to reject it.
That would best be done in your CI.
Just saying that the developer has to run X, Y, and Z tests simply isn't enough.

What about false negatives?

Posted Oct 18, 2025 17:55 UTC (Sat) by alx.manpages (subscriber, #145117) [Link] (2 responses)

> >A project can't force you to comply, but it's not forced either to accept the submissions.

> True.
> And to actually reject such a submission, you need to run X, Y, and Z tests on *your* side to find out whether you have to reject it.

One may fool a maintainer, by saying one hasn't used any AI for contributing. Just like one can dump AI slop directly as output from a chatbot and let the maintainer figure out if it's valid code.
Depending on how plausible that output is, one might fool more or less maintainers.

See also: <https://xcancel.com/spendergrsec/status/1958264076162998771>

After all, this isn't much different from the contributor claiming to not have copied code violating a license. That's something I can't verify as a maintainer, and have to trust the contributor by its word.

But one's reputation might be busted if it is eventually found out that one lied to the maintainers of a project.

I don't mind too much if someone uses AI tools if they didn't know the guideline. That's something I'd just remind the contributor I don't want them to do. But if it knows and still does it, and I somehow find out, then it's busted.

> That would best be done in your CI.
> Just saying that the developer has to run X, Y, and Z tests simply isn't enough.

The good thing about tests is that, as you say, I can verify in CI. This means I'm not too worried if a contributor doesn't run them. I simply remind them that they can run them, but since I'm able to run them myself in my CI server, it's not enough to distrust a contributor.

What about false negatives?

Posted Oct 19, 2025 19:07 UTC (Sun) by sashal (✭ supporter ✭, #81842) [Link] (1 responses)

> See also: <https://xcancel.com/spendergrsec/status/1958264076162998771>

To clarify:

1. The code in the patch wasn't AI generated.
2. The issue described in that post is an issue as much as "root can shoot himself in the foot!".

You are parroting a toxic person's speculative and misinformed post made to support his (again, toxic) businesses practices in an attempt to convince others that AI is somehow bad.

Is AI the problem here, or is it the behavior and conduct that you are modelling?

What about false negatives?

Posted Oct 21, 2025 16:46 UTC (Tue) by alx.manpages (subscriber, #145117) [Link]

Hi Sasha!

Thanks for the clarification. I'll remove that link.

On the other hand, if this specific case wasn't such a case, I'm still worried that it will eventually happen somewhere. I still believe that reviewing output from AI is way more difficult than human output, and comparable to reviewing output from malicious humans, which is something I don't want to have to review ever. If others feel confident enough to be able to review AI output, they're welcome to accept it in their projects, but I don't feel like taking the risk.

Cheers,
Alex

Terence Tao had a similar (non-Linux specific) suggestion for LLMs recently

Posted Oct 16, 2025 17:36 UTC (Thu) by Karellen (subscriber, #67644) [Link] (1 responses)

Sorry, not read the whole mailing list thread to see if anyone else brought it up, but for anyone looking for a bit more exploration of the idea, I found Tao's mastodon thread on the subject interesting

https://mathstodon.xyz/@tao/114915604830689046

Terence Tao had a similar (non-Linux specific) suggestion for LLMs recently

Posted Oct 17, 2025 16:08 UTC (Fri) by mathstuf (subscriber, #69389) [Link]

Yes, that thread gave me the wish for an LLM-as-LSP that would stream lints and whatnot directly to my editor just like LSP tools do.

Similar opinion

Posted Oct 16, 2025 17:43 UTC (Thu) by cen (subscriber, #170575) [Link]

I also started to use AI to review my own code submissions, especially to projects I maybe know less about, think drive-by patches and similar. If AI can catch even a single bug doing the review that's a win-win for everybody.

I also find AI useful to track down weird bugs because it can scan all the possible code paths faster that I can and either find the issue directly or at least give interesting insight. But sure, sometimes it will also be completely useless and run around in circles.

Instead of doom and gloom we should use these tools to make our lives easier, code review, debugging, performance regressions and git bisects.. plenty of areas which are painful for a human to do.

Even some of the struggling open source projects who are down to 1-2 maintainers could keep on living with some additional AI help.

Git forges

Posted Oct 16, 2025 18:33 UTC (Thu) by mb (subscriber, #50428) [Link] (20 responses)

>getting developers to use the tools (such as checkpatch.pl) that exist now is difficult
>[..] running the tools should be expected of submitters

Well no, all git forges and CI solve this problem very well.

The problem is rather that for some weird reason the kernel process sticks to sending patches to overloaded mailing lists that nobody reads due to being overloaded.

Git forges

Posted Oct 16, 2025 19:11 UTC (Thu) by pizza (subscriber, #46) [Link] (4 responses)

> Well no, all git forges and CI solve this problem very well.

By "very well" you mean "completely at someone else's expense"?

It's reasonable to expect folks to perform basic sanity checks *prior* to submitting a patch. Even better if they are in the form of local pre-commit/push hooks so that the central infrastructure never even sees the patch until it's clean.

Git forges

Posted Oct 16, 2025 19:15 UTC (Thu) by mb (subscriber, #50428) [Link] (3 responses)

Sure. And cute ponies would be great.

In reality, though, it's extremely cheap to run checkpatch.pl in a pull request action and then avoid needless human work on the overloaded maintainer side.
Because that's the actually expensive thing. Not 0.2 seconds of compute time spent.

Git forges

Posted Oct 16, 2025 19:42 UTC (Thu) by pizza (subscriber, #46) [Link] (2 responses)

> Sure. And cute ponies would be great.
> avoid needless human work on the overloaded maintainer side.
> Because that's the actually expensive thing. Not 0.2 seconds of compute time spent.

How is "automatically running things locally" needless human work?

Git forges

Posted Oct 16, 2025 20:01 UTC (Thu) by mb (subscriber, #50428) [Link] (1 responses)

Because I have never said this?
That's obvious even from the part that you quoted.

And because many submitters won't run your scripts locally, just like many won't run checkpatch.pl today.
If you as a maintainer want some checkers to be run, you need to ensure that on your end or on the forge that you control.
Just establishing a rule that every submitter has to run a checker is never going to fully work.

Git forges

Posted Oct 17, 2025 1:22 UTC (Fri) by Paf (subscriber, #91811) [Link]

The project I work on encourages you to run checkpatch.pl, but also if you don't, it's run by the code management infrastructure on all submissions. It takes ~no time and the cost is negligible.

If you remember to do it locally and fix the complaints, great - but you can also screw up or forget. It can be an *and* situation, especially when something is cheap.

It also lets everyone else know whether or not you did it, which has a benefit too (I think).

Git forges

Posted Oct 17, 2025 2:17 UTC (Fri) by marcH (subscriber, #57642) [Link] (14 responses)

> Bottomley, instead, said that these tools should be run automatically on patches sent to the mailing lists, much like the checks that the 0day robot runs on posted patches now. Bird, though, said that running the tools should be expected of submitters. "It then becomes a cost for the contributor instead of the upstream community, which is going to scale better."

Not talking about AI specifically: it's BOTH. It must be run by CI _and_ be easy to run by developers. Only after getting tired of being repeatedly shamed by public test runs, do people get into the habit of running tests themselves. It's one of the most important lessons that CI has taught us.

The rate of false positive does not matter; that method works the same.

I'm sorry to write this but it's always quite depressing to see the kernel community constantly playing catch-up and rediscovering "modern" dev practices many years later. Not every developer using a web interface is ignorant. Open source: check. Open mind: not so sure.

Granted: the cost of AI makes things a bit less simple. But that should already be the rule for other, cheaper test suites - and it's not.

Git forges

Posted Oct 17, 2025 10:50 UTC (Fri) by roc (subscriber, #30627) [Link] (8 responses)

Yes, and "have every contributor put together an automated workflow to run the relevant tools on every one of their submissions" actually doesn't scale better than centralized CI. It's vastly less efficient. It's error-prone. It's a barrier for contributors. And as you say, developers aren't actually going to do it properly so you have to do it centrally as well, anyway.

Git forges

Posted Oct 17, 2025 11:50 UTC (Fri) by pizza (subscriber, #46) [Link] (7 responses)

> Yes, and "have every contributor put together an automated workflow to run the relevant tools on every one of their submissions" actually doesn't scale better than centralized CI.

You act like this is something everyone has to create from scratch. It isn't. It's literally adding one line in a file under .git/hooks

And there is no centralized CI in this context; or at least not one that is comprehensive. Each subsystem has their own testing they perform, and it's done on a periodic, not per-commit basis.

I've worked on stuff whose test suite took literally weeks to run (using hundreds of cores running 24/7) and that's just for the stuff that could be automated. The last three $dayjobs I've worked had pre-submission requirements like I've described -- one of which I was responsible for deploying. So don't tell me it "doesn't scale" or is too haaaard to set up.

Git forges

Posted Oct 17, 2025 15:18 UTC (Fri) by marcH (subscriber, #57642) [Link] (1 responses)

> You act like this is something everyone has to create from scratch. It isn't. It's literally adding one line in a file under .git/hooks

In projects with "modern" development practices, developers and CI use the same test and automation code and configurations everywhere possible. No duplication and no "CI secret". Everyone can push the same buttons. They don't always have to, but they easily can whenever they want to. Running any check is as simple as running checkpatch.pl.

In projects lagging years or even decades behind, everyone and everything has their custom, personal scripts and private configuration files. The level of duplication there is probably the simplest and best metric of project (im)maturity.

One of the interesting side effects of this duplication is: the very frequent inability to reproduce bugs - even the ones with a 100% reproduction rate.

To be fair: there is a quite wide maturity range across kernel subsystems and there are some heroic efforts like 0day to cope with the "email-based database"[*] submission process. But the average does not look great and the lowest look even worse. So, "AI code reviews" really sounds like "cart before the horse". But... new shiny?

[*] speaking of development immaturity, don't get me started on recently killing "Link:" = the only, limited alternative to a proper Change-Id:... I digress.

Git forges

Posted Oct 17, 2025 18:22 UTC (Fri) by marcH (subscriber, #57642) [Link]

> and there are some heroic efforts like 0day to cope with the "email-based database"[*] submission process.

I bet the (too) few people reading these comments all know it already, but for fairness and completeness I must mention https://lore.kernel.org , https://b4.docs.kernel.org/en/latest/ , etc.

Lore is not a CI system but it looks like a good starting point for one. Maybe it has been used as one already.

> [*] speaking of development immaturity, don't get me started on recently killing "Link:" = the only, limited alternative to a proper Change-Id:... I digress.

Breaking news: https://lore.kernel.org/all/CAHk-=wj5MATvT-FR8qNpXuuBGiJd...

Git forges

Posted Oct 17, 2025 18:51 UTC (Fri) by Cyberax (✭ supporter ✭, #52523) [Link] (2 responses)

> You act like this is something everyone has to create from scratch. It isn't. It's literally adding one line in a file under .git/hooks

So now you can't even do a local scratch commit, because some perl environment variable is not set and the hook fails.

Git forges

Posted Oct 18, 2025 11:12 UTC (Sat) by mpg (subscriber, #70797) [Link] (1 responses)

git commit --no-verify is your friend here.

Git forges

Posted Oct 19, 2025 8:14 UTC (Sun) by Cyberax (✭ supporter ✭, #52523) [Link]

Yeah. And why not set an alias for it, so you don't have to type it every time? Then just remember to use the regular "commit" when you are preparing a PR.

Oh wait....

Git forges

Posted Oct 17, 2025 20:45 UTC (Fri) by roc (subscriber, #30627) [Link] (1 responses)

> You act like this is something everyone has to create from scratch. It isn't. It's literally adding one line in a file under .git/hooks

One per tool, you mean? Or is there literally one line you can add that enables all static checks?

> And there is no centralized CI in this context; or at least not one that is comprehensive.

Sure, because the Linux community has chosen not to prioritise that.

> too haaaard to set up.

Not when it's your full-time job, maybe.

If you're a project that cares about making life easy for first-time contributors, then it's an issue. Sure, the Linux kernel is not such a project.

Git forges

Posted Oct 17, 2025 21:25 UTC (Fri) by marcH (subscriber, #57642) [Link]

> If you're a project that cares about making life easy for first-time contributors, then it's an issue. Sure, the Linux kernel is not such a project.

Right, because it's so efficient to have experts re-invent more or less the same scripts and test configurations independently. They're super smart and perfectly able to[*], so why should they not duplicate that sort of effort? With varying degrees of success - which they can use to compete against each other! As a bonus, it puts off inexperienced people: perfect hazing / rite of passage.

/s /s /s - I never know whether to laugh or cry. "Everything As Code" - except here.

> One per tool, you mean? Or is there literally one line you can add that enables all static checks?

If only most Linux subsystems were at an automation stage advanced enough for this implementation detail to be a relevant question.

[*] more serious reality check: I'm regularly amazed at how many kernel uber-experts are actually ignorant with anything related to scripting or automation. By their _own_ and honest admission; it's not me making that up.

Git forges

Posted Oct 17, 2025 17:59 UTC (Fri) by laurent.pinchart (subscriber, #71290) [Link] (4 responses)

> Only after getting tired of being repeatedly shamed by public test runs, do people get into the habit of running tests themselves. It's one of the most important lessons that CI has taught us.

That contradicts my personal experience. When some of the projects I contribute to have set up CI, I've seen developers stopping to run some of the checks and tests locally, thinking it will be caught by CI anyway.

Git forges

Posted Oct 17, 2025 18:17 UTC (Fri) by marcH (subscriber, #57642) [Link] (3 responses)

That happens too and it's fine as long as CI failures remain rare. "Repeatedly" was a very important keyword.

"Repeatedly" failing CI checks fall in two categories:

- Developers struggle to replicate the CI setup. Such divergence demonstrates the immaturity of the project as the whole, see https://lwn.net/Articles/1042487/
- Idiots who don't understand it's much faster to run locally. _Extremely_ rare.

One of the advanced engineering skills that is not going to be replaced by AI any time soon is determining which tests are relevant to the changes being made. It's OK to be too optimistic and cut one too many corners from time to time. If you never break anything, you're not trying hard enough.

Git forges

Posted Oct 17, 2025 18:53 UTC (Fri) by marcH (subscriber, #57642) [Link]

> That happens too and it's fine as long as CI failures remain rare. [...]
> One of the advanced engineering skills that is not going to be replaced by AI any time soon is determining which tests are relevant to the changes being made. It's OK to be too optimistic and cut one too many corners from time to time. If you never break anything, you're not trying hard enough.

Example: the whole test suite takes 5h to run. So before submitting, you select and run a 20 min subset only. You don't select the correct subset every time and you fail CI sometimes. But you learn and get better over time. Everything is fine, nothing's black or white.

In my experience, people who don't learn and fail CI _consistently_ are rare. If they still don't course correct after multiple reminders, then you probably want to steer them away from the project somehow.

Git forges

Posted Oct 17, 2025 20:39 UTC (Fri) by roc (subscriber, #30627) [Link] (1 responses)

> One of the advanced engineering skills that is not going to be replaced by AI any time soon is determining which tests are relevant to the changes being made.

Actually many organisations are using some kind of AI for this. Well, often really just good old-fashioned statistical analysis, but we call that AI now.

Git forges

Posted Oct 17, 2025 23:24 UTC (Fri) by marcH (subscriber, #57642) [Link]

If it's based on existing QA code and processes then I'm not too worried. Garbage in, garbage out :-D

linter != bitkeeper

Posted Oct 17, 2025 2:28 UTC (Fri) by marcH (subscriber, #57642) [Link]

> would make them available to developers for free to encourage adoption — until the community is well locked in, at which point access could quickly become expensive.

How can you get locked in some _linter_ system? A linter is not required like compilation, version control or testing is. This is very different from BitKeeper. Sure, you can become too comfortable and miss the lower productivity when it goes away. But the loss of it will never block anything. Linters are already coming and going all the time. Even better: they are not even mutually exclusive, they're only limited by hardware and human resources. The competition can be fierce. Same thing with static analyzers.

Financial AI bubble

Posted Oct 17, 2025 8:19 UTC (Fri) by taladar (subscriber, #68407) [Link] (24 responses)

I would be more worried about the fact that none of these AI companies are profitable, burn huge amounts of money and will likely crash hard within the next year or two. Or in other words, these tools will go away and not have any alternatives you can adapt your prompt to because all of them are subject to the same economics.

Financial AI bubble

Posted Oct 17, 2025 10:16 UTC (Fri) by khim (subscriber, #9252) [Link] (23 responses)

At least Google and Microsoft are still profitable, isn't it? And what they provide is very good (I wouldn't debate whether Gemini is better then ChatGPT, but they are comparable, at least), so even if all pure AI startups would crash and burn we would have something.

Financial AI bubble

Posted Oct 17, 2025 12:00 UTC (Fri) by pizza (subscriber, #46) [Link] (17 responses)

>At least Google and Microsoft are still profitable, isn't it?

You mean illegally subsidizing a line of business that's bleeding copious amounts red ink with monopoly profits from other lines of business? And are in the process of losing appeals after losing legal challenges to those practices in numerous jurisdictions?

AFAICT precisely *zero* "AI" purveyors are anywhere near breaking even, much less profitable. (A day or two ago it came out that OpenAI is effectively lighting $2 on fire for every $1 in revenue that comes in)

Financial AI bubble

Posted Oct 17, 2025 12:06 UTC (Fri) by paulj (subscriber, #341) [Link] (8 responses)

A colleague was pointing at the latest "Apple loses top AI staff" article and arguing how bad this was for Apple, that they were losing out on AI. I pointed out it might be the smart move right now, given AI is currently a massive money loser and an obvious gigantic bubble. Apple can avoid the money sink-hole of AI, let others lose their shirt, wait till the bubble pops and then pick up the pieces.

Others should take note.

Who wants to make their predictions on what will be the largest tech company to go bust from the AI bubble popping? (Crusoe? Someone larger?).

Financial AI bubble

Posted Oct 20, 2025 13:04 UTC (Mon) by egb (subscriber, #163244) [Link] (7 responses)

Long shot answer? Nvidia. I want to see how hard they'll $EXPLETIVE themselves when AI-related hardware sales plummet.

Financial AI bubble

Posted Oct 20, 2025 13:14 UTC (Mon) by paulj (subscriber, #341) [Link] (6 responses)

Not impossible. NVidia's future success is heavily invested in AI now. It's promising the equivalent of about 76% of its annual revenue for 2025 to OpenAI - a company with $12 billion in revenue. Oracle looks to be heavily exposed too to OpenAI.

OpenAI certainly could crash hard. Question is how hard that will hurt those who have invested in it.

Financial AI bubble

Posted Oct 20, 2025 13:26 UTC (Mon) by khim (subscriber, #9252) [Link] (5 responses)

> Question is how hard that will hurt those who have invested in it.

As usual: idiots who invested in it would be burnt badly, companies involved wouldn't be hurt too much. AI is a classic ponzi scheme, these days: NVIDIA invests into CoreWeave, and OpenAI, then these buy GPUs from NVIDIA. Means that they can show anything they want in books in a short term by playing paper games.

As long as idiots like Softbank would take loans to finance the whole thing exists it would move, when bubble would burst… NVIDIA still have gaming GPUs to sell.

Dot com bubble crushed Sun not because it was selling servers, but because it had nothing else to sell. The companies that could have invested into gaming GPUs and left NVIDIA in Sun position, AMD and Intel, invested in AI, instead, thus they wouldn't be able to kill NVIDIA when they NVIDIA would be vulnerable.

Financial AI bubble

Posted Oct 20, 2025 20:28 UTC (Mon) by Wol (subscriber, #4433) [Link] (4 responses)

> As long as idiots like Softbank would take loans to finance the whole thing exists it would move, when bubble would burst… NVIDIA still have gaming GPUs to sell.

That fact forgets Dicken's dictum - "Annual income £20, annual expenditure £20 0s 6d, penury; Annual income £20, annual expenditure £19 19s 6d, happiness". And as a lot of people personally are finding out when they are laid off, just because your income may be slashed, your bills aren't.

If Nvidia are making most of their money from AI GPUs, and that market evaporates overnight, can they cut their expenditure fast enough to avoid it dragging them under?

Cheers,
Wol

Financial AI bubble

Posted Oct 21, 2025 9:52 UTC (Tue) by farnz (subscriber, #17727) [Link] (3 responses)

You're also assuming that NVidia has increased expenditure to match AI incomes.

If NVidia's committed spend requires no more than 20% of their revenue, and they're getting at least 20% from things other than AI, an AI crash "merely" destroys their share price. If they've committed to long-term projects that need 30% of their revenue, and AI is 75% of their revenue, an AI crash can take the company out.

Financial AI bubble

Posted Oct 21, 2025 10:22 UTC (Tue) by paulj (subscriber, #341) [Link] (2 responses)

My assumption is the likes of NVidia will be hurt bad, but survive. The world will still need GPUs and GPGPUs after all.

Those who bet heavily on leveraged investment for building CapEx-intensive DCs will crash out. Even if you switch off the electricity to the DC you just built, you still have large loan repayments falling due regularly. There are a couple of companies in this position. Some may be large (I'm unaware of how Oracle is financing its huge DC build-out, if it's very leveraged they may end up in trouble).

Financial AI bubble

Posted Oct 21, 2025 11:00 UTC (Tue) by paulj (subscriber, #341) [Link] (1 responses)

Oracles' build-out seems to be debt leveraged. E.g., this is just 1 banking deal for 1 set of DCs: https://cryptorank.io/news/feed/17010-banks-bet-oracles-c.... And just that $38B deal represents a sizeable portion of their 2025 revenue ($57.4B). And if I read that right, there's another $23B finance deal there too for another DC build out.

Wow. All for an industry with little revenue, an industry with massive baked-in costs which are not going to reduce.

The crash will be hard.

Financial AI bubble

Posted Nov 12, 2025 9:19 UTC (Wed) by sammythesnake (guest, #17693) [Link]

If all else fails, all that AI-oriented infrastructure can still be pivoted to cryptocurrency...

Financial AI bubble

Posted Oct 17, 2025 18:21 UTC (Fri) by cpitrat (subscriber, #116459) [Link] (7 responses)

>At least Google and Microsoft are still profitable, isn't it?

I would add that Google and Microsoft are known to be trustworthy when it comes to maintaining their products and services on the long term, even if they are not profitable or don't align perfectly with their goal anymore. So no risk of any work going to waste here.

Financial AI bubble

Posted Oct 17, 2025 20:28 UTC (Fri) by laurent.pinchart (subscriber, #71290) [Link] (6 responses)

> I would add that Google and Microsoft are known to be trustworthy when it comes to maintaining their products and services on the long term

Are you maybe too young to remember Google+? :) Google is notorious for axing whole product lines out of the blue. Granted, some of them were just experiments, but not all of them. The most recent example of Chrome OS being absorbed by Android shows it also happens for products with a large user base.

I'm less familiar with the Microsoft world, I'll let someone else provide examples.

Financial AI bubble

Posted Oct 17, 2025 20:37 UTC (Fri) by cpitrat (subscriber, #116459) [Link]

I'm unfortunately not young. This was, of course, sarcasm. I do remember Google Plus. And Google Reader. And Google Code. And many others. I also remember when Microsoft threw their DRM to the bin for a new one and told people who had bought music locked by it that they could just buy it again with the new version. And more recently, Microsoft is letting Windows 10 users who can't upgrade without security updates. And that's only a fraction of the emerged part of the iceberg.

Financial AI bubble

Posted Oct 20, 2025 13:09 UTC (Mon) by khim (subscriber, #9252) [Link] (3 responses)

> I'm less familiar with the Microsoft world, I'll let someone else provide examples.

How is fate of Windows Mobile any different from ChromsOS?

But note that in all these examples products had one “fatal flaw” — they weren't even close to be market leaders in a particular segment.

That, essentially, means that we don't know which AI provider would survive, but at least one of them would.

Financial AI bubble

Posted Oct 20, 2025 20:33 UTC (Mon) by Wol (subscriber, #4433) [Link] (2 responses)

> That, essentially, means that we don't know which AI provider would survive, but at least one of them would.

You're assuming there is a viable market segment to be leader of ...

Cheers,
Wol

Financial AI bubble

Posted Oct 20, 2025 20:38 UTC (Mon) by khim (subscriber, #9252) [Link] (1 responses)

There are absolutely is a viable market segment: selling data (presumably aggregated data) that users share with AI companies to advertisers.

It's more than enough to support something like GPT-3. And GPT-3 is enough to attract a lot of people who would be willing to share that data.

Whether such AI would be sophisticated enough to review kernel patches is an interesting question, of course.

Financial AI bubble

Posted Oct 20, 2025 21:22 UTC (Mon) by pizza (subscriber, #46) [Link]

> There are absolutely is a viable market segment: selling data (presumably aggregated data) that users share with AI companies to advertisers.

"viable" meaning "can turn a profit"

Which, AFAICT, has yet to be demonstrated.

Financial AI bubble

Posted Oct 20, 2025 14:10 UTC (Mon) by geert (subscriber, #98403) [Link]

> Are you maybe too young to remember Google+? :)

You mean Orkut? Fortunately I never got a T-shirt of that ;-)

It's not just Google: the T-shirt-with-URL I once got from Intel long survived the actual website and domain...
Oh, apparently https://lesswatts.org/ has been rescued from the cybersquatters...

Financial AI bubble

Posted Oct 17, 2025 12:13 UTC (Fri) by taladar (subscriber, #68407) [Link] (4 responses)

Google and Microsoft as a whole are profitable, their AI departments are not. From what I remember Copilot operates on about 100% loss purely for inference (without counting development or anything else). Not sure Gemini has published numbers but since OpenAI and Claude have similarly bad cost to subscription price ratios I would be surprised if it was different for Google.

Financial AI bubble

Posted Oct 17, 2025 12:22 UTC (Fri) by khim (subscriber, #9252) [Link] (3 responses)

> Google and Microsoft as a whole are profitable, their AI departments are not.

Why does it matter? It would be as silly as expecting to see profits from the web search department. AI-based search is tied at the hip with their Ads business, it's more than enough for AI to work as an attractor on Bing.com and Google.com to show more ads. And it works fine for that even if doesn't work for many other things.

Financial AI bubble

Posted Oct 17, 2025 16:58 UTC (Fri) by mbunkus (subscriber, #87248) [Link] (2 responses)

Even if Microsoft as a whole is still profitable, the huge money drain that is AI has drastic consequences on other divisions and unrelated consumers.

The one I know most about is the gaming side Xbox, which is not just a console but the whole subscription business. Additionally Xbox is the owner of a lot of game studios out there — the biggest & well-known one was the ActivisionBlizzard acquisition a while ago, but there are tons of other big ones such as Bethesda Game Studios and tons of small ones. The hammer has come down on Xbox due to AI spending: both the requirement to use Microsoft's AI products in all studios, but also to radically cut cost/improve revenue. This resulted in a lot of layoffs in various studios over the last couple of months, and some studios have simply been closed. And mind you, Xbox wasn't a loss maker, the division was profitable.

As to how it effects consumers: well, Xbox just decided to increase the monthly subscription fee for their Game Pass service, the most popular & highest tier now costs a whopping 50% more than before.

So yeah, the whole AI strategy does matter, it does cause fallout. I'd bet that if AI made at least as much money as it cost the pressure on other parts of MS wouldn't be nearly as high.

Financial AI bubble

Posted Oct 17, 2025 18:46 UTC (Fri) by pizza (subscriber, #46) [Link]

> Even if Microsoft as a whole is still profitable, the huge money drain that is AI has drastic consequences on other divisions and unrelated consumers.

In 2018 my then-employer was wholly owned by Softbank, which took a huge hit from a couple of bad investments (notably including WeWork). The edict came down to drastically cut expenses everywhere, and my ~120-person division [1] working on a "next big thing" project was axed entirely because it wasn't profitable [2]. This rug-pull came while we were in the process of securing a larger office space and onboarding several new (relocated!) hires to meet Softbank's previous edict to massively scale up our goals because they "weren't ambitious enough".

[1] of a highly profitable company in Softbank's portfolio
[2] Mind you, it's hard to be profitable when you're shut down before the planned start of volume production...

Financial AI bubble

Posted Oct 18, 2025 10:26 UTC (Sat) by Wol (subscriber, #4433) [Link]

> As to how it effects consumers: well, Xbox just decided to increase the monthly subscription fee for their Game Pass service, the most popular & highest tier now costs a whopping 50% more than before.

So a division that was already profitable, has had to increase its "cost to the consumer" by 50% to subsidise losses elsewhere. Interesting ...

What's that going to do for companies like Sony, Dreamcast, Nintendo that are not - to the best of my knowledge - that into computing and AI?

And in the face of competition, doesn't increasing prices normally result in a drop in turnover? Not what MS want ... ?

Okay MS may have bought up a lot of the studios, but suddenly studios that target other consoles will be in a buyers market for talent, it sounds like ...

It wouldn't surprise me if "the largest company to go bust" could be MS or Google - call in the administrators, sell the profitable bits, and dump the wreckage ...

Cheers,
Wol

Positive experiences on other projects (curl)

Posted Oct 21, 2025 18:19 UTC (Tue) by lmb (subscriber, #39048) [Link] (5 responses)

Daniel, who is famous for complaining (rightfully!) about AI slop reports and issues raised, recently had good things to say about AI code analysis, which I think is worthwhile to review in this context: https://daniel.haxx.se/blog/2025/10/10/a-new-breed-of-ana... (referring to https://joshua.hu/llm-engineer-review-sast-security-ai-to...)

That's not exclusively LLM use, but still, clearly they were used in generating the reports. To surprising effect.

In conversation (alas not public) with Paulus Schoutsen (the Home Assistant person), he brought up an interesting point on the use of AI/LLMs for code review that they've seen in the HA world: experienced people benefit, whereas more junior people suffer.

Because the former can almost immediately dismiss the LLM going off the rails (as they, as we all know, are prone to do), and focus on what makes sense as added value, and this actually does accelerate them. A more junior person has to go through the review and understand, and now suddenly has way more feedback to deal with than a human reviewer previously would have provided.

It probably makes sense to review the patches at least quickly first before looking at the LLM review to avoid being too influenced by their report; but over all, I'd suspect this can be a valuable addition.

Positive experiences on other projects (curl)

Posted Oct 22, 2025 8:08 UTC (Wed) by taladar (subscriber, #68407) [Link] (4 responses)

On the other hand Daniel also mentions that the curl code base is several 100k lines of code and after decades of not applying a tool like that he only got a few hundred suggestions from the AI tool, most of them minor things or things they did not want to fix. Only a few were actual bugs and most of those were in implementations of protocols with decades old specs, so specs that were definitely part of the training data of the LLM.

Most code bases do not have that luxury so I would expect the useful results to be even lower in number.

Positive experiences on other projects (curl)

Posted Oct 23, 2025 21:47 UTC (Thu) by Karellen (subscriber, #67644) [Link] (3 responses)

You expect most projects would get a lower rate of errors (per-kLOC?) than curl?

I'm not sure about that. From what I've read, the curl team (and Daniel Stenberg in particular) pays a lot of attention to code quality. Notably, they try to make a habit that when one bug is found, to search the codebase for similar bugs and eliminate them at the same time. More details at

https://daniel.haxx.se/blog/2025/04/07/writing-c-for-curl/

Daniel also did a presentation on the subject at FOSDEM 2025 - Tightening every bolt: https://www.youtube.com/watch?v=Yr5fPxZvhOw (25:44)

Positive experiences on other projects (curl)

Posted Oct 24, 2025 3:53 UTC (Fri) by edgewood (subscriber, #1123) [Link] (2 responses)

I believe that the luxury the OP was referring to is "most of those (bugs) were in implementations of protocols with decades old specs, so specs that were definitely part of the training data of the LLM".

In other words, curl might be a best-case scenario for LLM bug finding, due it being code that implements well known public specs. Other codebases, which are unlikely to implement well known public specs that are part of the LLM training data, may not benefit as much.

Positive experiences on other projects (curl)

Posted Oct 24, 2025 8:28 UTC (Fri) by taladar (subscriber, #68407) [Link] (1 responses)

That is exactly what I meant.

Positive experiences on other projects (curl)

Posted Oct 24, 2025 10:57 UTC (Fri) by Karellen (subscriber, #67644) [Link]

Ah, sorry for misreading your meaning, my mistake!


Copyright © 2025, Eklektix, Inc.
This article may be redistributed under the terms of the Creative Commons CC BY-SA 4.0 license
Comments and public postings are copyrighted by their creators.
Linux is a registered trademark of Linus Torvalds