mlxsw: spectrum_ptp: Fix napi_gro_receive() call from GC workqueue context
Linux
Affected versions
| Product | Affected | Fixed in |
|---|---|---|
| Linux Linux |
| No fix yet |
| Linux Linux |
|
|
Linux
| Product | Affected | Fixed in |
|---|---|---|
| Linux Linux |
| No fix yet |
| Linux Linux |
|
|
6.18.537.2.77.3In the Linux kernel, the following vulnerability has been resolved:
mlxsw: spectrum_ptp: Fix napi_gro_receive() call from GC workqueue context
Currently mlxsw_sp1_ptp_ht_gc_collect() is run from the PTP garbage-collection workqueue, rather than the NAPI poll context. For any unmatched PTP entries carrying an SKB, it calls mlxsw_sp1_ptp_unmatched_finish() -> mlxsw_sp1_ptp_packet_finish(). For ingress packets, this calls mlxsw_sp_rx_listener_no_mark_func(). The end of that function is the following:
skb->protocol = eth_type_trans(skb, skb->dev); napi_gro_receive(mlxsw_skb_cb(skb)->rx_md_info.napi, skb);
The napi pointer is one that was placed in the SKB control block when the trapped packet was received in the NAPI context. Later, when the GC reaps the unmatched entry (up to MLXSW_SP1_PTP_HT_GC_TIMEOUT later), the call to napi_gro_receive() mutates the NAPI instance's GRO list, which is unsafe if the poll is running concurrently on another CPU.
In mlxsw_sp1_ptp_ht_gc_collect(), local_bh_disable() is called to prevent softirq processing, but this only applies to the local CPU. Additionally, its comment is stale. It states that mlxsw_sp1_ptp_unmatched_finish() invokes netif_receive_skb(). This has not been accurate since the referenced commit; this patch makes that comment accurate again. mlxsw_pci_napi_devs_init() calls netif_threaded_enable() on the NAPI RX net_device without any conditions. The NAPI instance's poll, which may be running concurrent to the GC, is running as an independently-scheduled kthread which may be on a different CPU. The call to local_bh_disable() does not guard against this.
If a tx-timestamp timeout produces an unmatched entry (which can be easily reproduced by running ptp4l and waiting for a port to reach the UNCALIBRATED/SLAVE state) while the owning NAPI thread is in the middle of a poll on another CPU, both sides mutate the GRO list concurrently, as shown below:
[39.846] port 1 (swp1): MASTER to UNCALIBRATED on RS_SLAVE list_add corruption. next->prev should be prev (ffff8d620faf4138), but was ffff8d624150f700. (next=ffff8d620faf4138). kernel BUG at lib/list_debug.c:29! Oops: invalid opcode: 0000 [#1] SMP PTI CPU: 1 UID: 0 PID: 539 Comm: napi/mlxsw_rx-0 Not tainted 6.18.48 #1-NixOS PREEMPT(lazy) Hardware name: Mellanox Technologies Ltd. MSN2410/VMOD0001, BIOS 4.6.5 09/13/2018 RIP: 0010:__list_add_valid_or_report+0x79/0xb0 RSP: 0018:ffffcdf8c0f27c08 EFLAGS: 00010246 RAX: 0000000000000075 RBX: ffff8d624150fd00 RCX: 0000000000000000 RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff8d6315d1e540 RBP: ffff8d620faf4070 R08: 0000000000000000 R09: 00000000ffffdfff R10: ffffffffa5c60fe0 R11: ffffcdf8c0f27ab8 R12: 0000000000000003 R13: 000000000000003d R14: 00000000000001bc R15: 0000000000000001 FS: 0000000000000000(0000) GS:ffff8d636f63f000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000562689a60c24 CR3: 000000015f224004 CR4: 00000000001726f0 Call Trace: <TASK> gro_receive_skb+0xee/0x230 mlxsw_sp1_ptp_got_packet+0x61/0x140 [mlxsw_spectrum] mlxsw_core_skb_receive+0xdf/0x1b0 [mlxsw_core] mlxsw_pci_napi_poll_cq_rx+0x780/0x9d0 [mlxsw_pci] __napi_poll+0x31/0x1e0 napi_threaded_poll_loop+0x16b/0x1c0 napi_threaded_poll+0x71/0xa0 kthread+0xfb/0x260 ret_from_fork+0x22d/0x260 ret_from_fork_asm+0x1a/0x30 </TASK> Kernel panic - not syncing: Fatal exception in interrupt
From the CVE record · Linux
The Linux kernel's mlxsw spectrum_ptp code can modify a NAPI GRO list from a garbage-collection workqueue while a NAPI poll runs on another CPU. When an unmatched PTP entry is reaped during a concurrent poll, list corruption can cause a kernel crash. The input does not establish an attacker access path or authentication requirements.
Engineers running Linux systems that use the mlxsw spectrum_ptp code should check their kernel; affected versions are not specified.