What about false negatives? [LWN.net]

What about false negatives?

Posted Oct 18, 2025 16:17 UTC (Sat) by alx.manpages (subscriber, #145117)
In reply to: What about false negatives? by intelfx
Parent article: Large language models for patch review

There's no legal obligation to comply. It's a contribution guideline, not a license. I'm stating that I don't want to receive contributions of low quality, and I consider submissions created with any help of AI to be of low quality.

It's the same as a guideline saying you must run X, Y, and Z tests before submitting. A project can't force you to comply, but it's not forced either to accept the submissions.

to post comments

What about false negatives?

Posted Oct 18, 2025 17:18 UTC (Sat) by mb (subscriber, #50428) [Link] (3 responses)

>A project can't force you to comply, but it's not forced either to accept the submissions.

True.
And to actually reject such a submission, you need to run X, Y, and Z tests on *your* side to find out whether you have to reject it.
That would best be done in your CI.
Just saying that the developer has to run X, Y, and Z tests simply isn't enough.

What about false negatives?

Posted Oct 18, 2025 17:55 UTC (Sat) by alx.manpages (subscriber, #145117) [Link] (2 responses)

> >A project can't force you to comply, but it's not forced either to accept the submissions.

> True.
> And to actually reject such a submission, you need to run X, Y, and Z tests on *your* side to find out whether you have to reject it.

One may fool a maintainer, by saying one hasn't used any AI for contributing. Just like one can dump AI slop directly as output from a chatbot and let the maintainer figure out if it's valid code.
Depending on how plausible that output is, one might fool more or less maintainers.

After all, this isn't much different from the contributor claiming to not have copied code violating a license. That's something I can't verify as a maintainer, and have to trust the contributor by its word.

But one's reputation might be busted if it is eventually found out that one lied to the maintainers of a project.

I don't mind too much if someone uses AI tools if they didn't know the guideline. That's something I'd just remind the contributor I don't want them to do. But if it knows and still does it, and I somehow find out, then it's busted.

> That would best be done in your CI.
> Just saying that the developer has to run X, Y, and Z tests simply isn't enough.

The good thing about tests is that, as you say, I can verify in CI. This means I'm not too worried if a contributor doesn't run them. I simply remind them that they can run them, but since I'm able to run them myself in my CI server, it's not enough to distrust a contributor.

What about false negatives?

Posted Oct 19, 2025 19:07 UTC (Sun) by sashal (✭ supporter ✭, #81842) [Link] (1 responses)

To clarify:

1. The code in the patch wasn't AI generated.
2. The issue described in that post is an issue as much as "root can shoot himself in the foot!".

You are parroting a toxic person's speculative and misinformed post made to support his (again, toxic) businesses practices in an attempt to convince others that AI is somehow bad.

Is AI the problem here, or is it the behavior and conduct that you are modelling?

What about false negatives?

Posted Oct 21, 2025 16:46 UTC (Tue) by alx.manpages (subscriber, #145117) [Link]

Hi Sasha!

Thanks for the clarification. I'll remove that link.

On the other hand, if this specific case wasn't such a case, I'm still worried that it will eventually happen somewhere. I still believe that reviewing output from AI is way more difficult than human output, and comparable to reviewing output from malicious humans, which is something I don't want to have to review ever. If others feel confident enough to be able to review AI output, they're welcome to accept it in their projects, but I don't feel like taking the risk.

Cheers,
Alex