blk-mq: fix IO hang from sbitmap wakeup race
Linux
Affected versions
| Product | Affected | Fixed in |
|---|---|---|
| Linux Linux |
|
Linux
| Product | Affected | Fixed in |
|---|---|---|
| Linux Linux |
|
≥ da55f2cc78418dee88400aafbbaed19d7ac8188e · < 6d8b01624a2540336a32be91f25187a433af53a0| No fix yet |
| Linux Linux |
|
|
In the Linux kernel, the following vulnerability has been resolved:
blk-mq: fix IO hang from sbitmap wakeup race
In blk_mq_mark_tag_wait(), __add_wait_queue() may be re-ordered with the following blk_mq_get_driver_tag() in case of getting driver tag failure.
Then in __sbitmap_queue_wake_up(), waitqueue_active() may not observe the added waiter in blk_mq_mark_tag_wait() and wake up nothing, meantime blk_mq_mark_tag_wait() can't get driver tag successfully.
This issue can be reproduced by running the following test in loop, and fio hang can be observed in < 30min when running it on my test VM in laptop.
modprobe -r scsi_debug modprobe scsi_debug delay=0 dev_size_mb=4096 max_queue=1 host_max_queue=1 submit_queues=4 dev=`ls -d /sys/bus/pseudo/drivers/scsi_debug/adapter*/host*/target*/*/block/* | head -1 | xargs basename` fio --filename=/dev/"$dev" --direct=1 --rw=randrw --bs=4k --iodepth=1 \ --runtime=100 --numjobs=40 --time_based --name=test \ --ioengine=libaio
Fix the issue by adding one explicit barrier in blk_mq_mark_tag_wait(), which is just fine in case of running out of tag.
From the CVE record · Linux
A race in the Linux kernel’s blk-mq tag-wait path can prevent a waiting I/O request from being woken, causing an I/O hang. The issue was reproduced under a concurrent I/O workload; the input does not describe an attacker, access requirements, or user interaction.
Engineers running Linux kernels should check whether their kernel includes the blk-mq fix; affected versions are not specified.