Using the multiqueue block subsystem by default

For humans, by humans

Every article on LWN.net is written for humans, by humans. If you've enjoyed this article and want to see more like it, your subscription goes a long way to keeping the slop at bay. We are offering a free one-month trial subscription (no credit card required) to get you started.

The multiqueue block layer has been a part of the kernel for a few years now, and it works reliably, Bart Van Assche said to start a session he led at the 2016 Linux Storage, Filesystem, and Memory-Management Summit. He wondered if it was time to make multiqueue the default with an eye toward eventually removing the single-queue API.

The SCSI multiqueue (scsi-mq) and device mapper multiqueue (dm-mq) drivers have to be explicitly enabled currently, but there is a long list of block and SCSI devices that support multiqueue at this point. The challenge, Van Assche said, is that both the single and multiqueue paths have to be maintained and each needs to be tested when changes are made.

He suggested that the first step toward eliminating the single-queue code would be to enable multiqueue by default for SCSI devices, but there are some missing pieces that would need to be filled in. I/O accounting support would need to be added, he said, but Jens Axboe disagreed, saying that the accounting support was present. A bigger missing piece is I/O scheduling for block multiqueue, he said.

Evolving to a single code path will be difficult because the two paths are significantly different, Van Assche said. Hannes Reinecke said that it would require all drivers to support multiqueue, which will be hard. There is a need to document which block-layer functions do not work with multiqueue—effectively, which functions are safe to use with multiqueue and which are not. Van Assche suggested that some kind of tool might help with the conversion.

Reinecke asked if there were all that many drivers that had not been converted yet. Christoph Hellwig said that there was some "weird stuff" in the block directory that hasn't been converted. After some quick analysis of the tree, he said that there were 38 drivers that still needed conversion. It will be hard to test the conversion of some of them, since they are for hard-to-obtain hardware and the like.

In the near term, there is a question of how to handle multiqueue, Reinecke said. If there is a global switch that chooses between multiqueue and single queue, it won't be good for legacy devices. Distributions can either disable multiqueue and lose out on its performance gains, or enable it and get bad performance from legacy devices, he said.

The main problem seems to be a lack of scheduling for multiqueue, but Axboe said that it is actively being worked on. A patch that would select between single and multiqueue for different devices "on the fly" is possible, he said, and could be used as a near-term fix.

The biggest problem, Axboe said, is for systems with regular SATA drives, where the read latency goes "through the roof" with multiqueue. That is likely a problem that writeback throttling would solve. Once those patches get in, scheduling will not be as important, he said. Multiqueue is not only for fast devices, so getting I/O scheduling or throttling in will be likely to remove the barriers for multiqueue on legacy devices.