Non-free drivers [LWN.net]

Non-free drivers

Posted Jan 27, 2011 8:35 UTC (Thu) by zxcvbnm (guest, #71201)
Parent article: The real BKL end game

It'll be interesting to see what kind of havoc this brings down upon the developers of closed-source/non-free drivers. Not that I care mind you; though, it should be fun to watch.

to post comments

Non-free drivers

Posted Jan 27, 2011 8:41 UTC (Thu) by bronson (subscriber, #4806) [Link] (17 responses)

Yep. And it's yet more proof that Linux's no-stable-abi policy is by far the best way to go.

Non-free drivers

Posted Jan 27, 2011 9:11 UTC (Thu) by zxcvbnm (guest, #71201) [Link] (16 responses)

I'm not sure how this proves the API policy is a bad idea. If the API was stable (read: written in stone) we'd be saddled with the BKL for years to come.

Non-free drivers

Posted Jan 27, 2011 9:18 UTC (Thu) by smitty_one_each (subscriber, #28989) [Link]

It seems like you may have inverted the logic of GP.

Non-free drivers

Posted Jan 27, 2011 9:41 UTC (Thu) by trasz (guest, #45786) [Link] (14 responses)

Actually, not at all. Systems maintaining stable ABI don't have any bigger problems with their BKL counterparts (assuming they still have something like that) than Linux has.

Non-free drivers

Posted Jan 27, 2011 10:49 UTC (Thu) by xav (guest, #18536) [Link] (13 responses)

Citation needed.

Non-free drivers

Posted Jan 27, 2011 11:05 UTC (Thu) by trasz (guest, #45786) [Link] (12 responses)

Same could be said for the opposite argument - the "stable kernel API nonsense" is just a nonsense, since all other operating systems do just fine with reasonably stable kernel API and ABI.

Non-free drivers

Posted Jan 27, 2011 16:15 UTC (Thu) by BrucePerens (guest, #2510) [Link] (11 responses)

For some arbitrary level of "fine", lower than our standard. Actually, those folks go through a tremendous lot of pain to improve their systems while preserving the old ABIs. I remember that HP had planned to put a Billion dollars into a rewrite of HP-UX around 2000.

Non-free drivers

Posted Jan 27, 2011 16:46 UTC (Thu) by trasz (guest, #45786) [Link] (10 responses)

Are you closely following the development of systems such as FreeBSD or Solaris, which keep reasonably stable API and ABI without any "tremendous lot of pain" or "Billion dollars"? Actually, the reverse is possible - if you think about interfaces before implementing them, you don't need to go through all the code base fixing it later.

Non-free drivers

Posted Jan 27, 2011 17:35 UTC (Thu) by BrucePerens (guest, #2510) [Link] (3 responses)

if you think about interfaces before implementing them, you don't need to go through all the code base fixing it later.

Can you now see the unconscious (at least I hope it was unconscious) disrespect for the kernel developers in that comment? You seem to believe that if the developers just thought about it, they'd get the ABIs right the first time and they'd never have to do it again.

Locking ABIs down means you just won't innovate that part of the system any longer, and you'll put up with what eventually turn out to be insufficient designs, as new requirements evolve, for the sake of stability.

Linux already has enough of that in the 40 year old Unix interface. One I/O context permanently glued to the process context. Asynchronous signals used where an OS event queue would make more sense. Whole-process fork. And then all of the later stuff made to work with the existing paradigm much less cleanly than would be necessary if we had the opportunity to redefine the process context.

Now, take those problems and multiply them, by spreading them across the entire kernel. That is what other OS maintainers have to cope with. It has been expensive for Sun, and it has been one of the factors that makes BSD the second fiddle.

Non-free drivers

Posted Jan 27, 2011 18:11 UTC (Thu) by trasz (guest, #45786) [Link] (2 responses)

It's not about "never", as there is no point in keeping stable kernel API and ABI forever. It's about keeping them stable for a reasonable amount of time, in a way that administrators and 3rd party developers can actually understand and adapt to. Red Hat does this, btw.

Keeping API stable does _not_ mean one can't innovate - in particular, it doesn't prevent anyone from adding new interfaces, or new functionality. In the worst case, one needs to reimplement old API as a wrapper. It sometimes happens in userland, sometimes in horrible ways (errno symbol vs. threads, for example); but for kernel interfaces is seldom needed, and does not cause any "horrible problems", contrary to what "stable API nonsense" claims.

Regarding the UNIX heritage - yes, it's there, and nothing can be done about it, unfortunately. However, the whole "stable API nonsense" thing is about interfaces for kernel code, not for userland aplications.

Non-free drivers

Posted Jan 27, 2011 18:30 UTC (Thu) by BrucePerens (guest, #2510) [Link] (1 responses)

It's about keeping them stable for a reasonable amount of time, in a way that administrators and 3rd party developers can actually understand and adapt to.

The only reason to do this is to provide an interface for outside-of-tree kernel facilities. Which are their own mistake. Even if they are only device drivers.

Semantics like those of the BKL, to the extent that they leak into the semantics of your ABI, are an excellent example of semantics that are difficult to just handwave in a wrapper. You'd have to make a real lock that at a minimum would lock out all of its callers, which would be other ABI clients. Consider the Vint Cerf piece today. They made mistakes because they simply did not consider that IP devices would be mobile. We make the same mistakes every day. It's really nice when they aren't cast in concrete. Remember the story of the syntactic white space in makefiles and why we still have it today.

I'm also thinking of the ext4 data loss argument, and the discussion of what the implied semantics of rename should be and whether programmers should have to explicitly call fsync. Some of this stuff is really hard to keep from spreading its dirty fingers far beyond your wrapper.

Non-free drivers

Posted Jan 28, 2011 22:24 UTC (Fri) by giraffedata (guest, #1954) [Link]

The only reason to do this [keep a stable interface protocol] is to provide an interface for outside-of-tree kernel facilities. Which are their own mistake.

What besides the need for the stable interface protocol makes outside-of-tree kernel facilities a mistake?

Non-free drivers

Posted Jan 27, 2011 20:38 UTC (Thu) by Cyberax (✭ supporter ✭, #52523) [Link] (3 responses)

FreeBSD still has a giant lock, which they are slowly splitting into smaller locks right now. And it has already caused several ABI breaks.

I don't know much about Solaris, however I did quite a lot of development on WinNT. It's almost possible to run WinNT4 drivers on Windows Vista there. However, it comes at a great price. For example, filesystem stack in Windows is routinely orders of magnitude slower than in Linux.

Non-free drivers

Posted Jan 28, 2011 9:48 UTC (Fri) by trasz (guest, #45786) [Link] (2 responses)

FreeBSD is roughly where Linux is in that regard - the Giant lock is gone from all but a few, not performance-critical, places. Last API or ABI-related change related to the Giant was removing support for non-MPSAFE (i.e. Giant-locked) network interfaces few years ago. Nothing that would cause problems with maintaining stable API and ABI in an expected way.

Non-free drivers

Posted Jan 28, 2011 13:01 UTC (Fri) by nix (subscriber, #2304) [Link] (1 responses)

Except that 'you cannot rely on the Giant Lock in your out-of-tree network interface driver' *is* an interface change which drivers must adapt to. I'd rather signal that by an actual API change, forcing people to update their drivers or fail to compile, than silently change it and open users up to problems due to absent locking.

Non-free drivers

Posted Jan 28, 2011 13:19 UTC (Fri) by trasz (guest, #45786) [Link]

Again - nobody is talking about not changing API and ABI _ever_. It's ok to keep it stable for a defined period of time, say, across a major version. The change you're describing happened in 2003. Since then, there were other changes - but it didn't change the fact that network drivers compiled for FreeBSD 7.0 will work fine with FreeBSD 7.4 once it's released.

Btw - of course there was a migration period, during which kernel warned about drivers that didn't declare themselves as MPSAFE ("Giant-free").

Non-free drivers

Posted Jan 28, 2011 4:14 UTC (Fri) by BenHutchings (subscriber, #37955) [Link] (1 responses)

FreeBSD doesn't even have a stable ABI between kernel and userland! Solaris, on the other hand, has obsessive API and ABI compatibility, but strangely enough it doesn't make life easy for driver developers.

Non-free drivers

Posted Jan 28, 2011 9:43 UTC (Fri) by trasz (guest, #45786) [Link]

Of course FreeBSD does have a stable syscall ABI. It's not uncommon to run FreeBSD 6 jails on FreeBSD 8 kernel, for example. The only exception here is KVM, which means several administrative tools, such as ipcs(1) or netstat(1), need to match the kernel version.

Non-free drivers

Posted Jan 27, 2011 15:28 UTC (Thu) by jengelh (subscriber, #33263) [Link]

That, together with init_MUTEX gone. Now all we need is more GPL exports..