1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
|
From: Steven Rostedt <rostedt@goodmis.org>
Date: Tue, 6 Dec 2016 17:50:30 -0500
Subject: [PATCH 204/354] net: Have __napi_schedule_irqoff() disable interrupts
on RT
Origin: https://git.kernel.org/cgit/linux/kernel/git/rt/linux-stable-rt.git/commit?id=5b01cd8898158a9ed0b54e85ca8642fe3fd17b27
A customer hit a crash where the napi sd->poll_list became corrupted.
The customer had the bnx2x driver, which does a
__napi_schedule_irqoff() in its interrupt handler. Unfortunately, when
running with CONFIG_PREEMPT_RT_FULL, this interrupt handler is run as a
thread and is preemptable. The call to ____napi_schedule() must be done
with interrupts disabled to protect the per cpu softnet_data's
"poll_list, which is protected by disabling interrupts (disabling
preemption is enough when all interrupts are threaded and
local_bh_disable() can't preempt)."
As bnx2x isn't the only driver that does this, the safest thing to do
is to make __napi_schedule_irqoff() call __napi_schedule() instead when
CONFIG_PREEMPT_RT_FULL is enabled, which will call local_irq_save()
before calling ____napi_schedule().
Cc: stable-rt@vger.kernel.org
Signed-off-by: Steven Rostedt (Red Hat) <rostedt@goodmis.org>
Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
---
include/linux/netdevice.h | 12 ++++++++++++
net/core/dev.c | 2 ++
2 files changed, 14 insertions(+)
diff --git a/include/linux/netdevice.h b/include/linux/netdevice.h
index 1644390430a3..821337d81da2 100644
--- a/include/linux/netdevice.h
+++ b/include/linux/netdevice.h
@@ -432,7 +432,19 @@ typedef enum rx_handler_result rx_handler_result_t;
typedef rx_handler_result_t rx_handler_func_t(struct sk_buff **pskb);
void __napi_schedule(struct napi_struct *n);
+
+/*
+ * When PREEMPT_RT_FULL is defined, all device interrupt handlers
+ * run as threads, and they can also be preempted (without PREEMPT_RT
+ * interrupt threads can not be preempted). Which means that calling
+ * __napi_schedule_irqoff() from an interrupt handler can be preempted
+ * and can corrupt the napi->poll_list.
+ */
+#ifdef CONFIG_PREEMPT_RT_FULL
+#define __napi_schedule_irqoff(n) __napi_schedule(n)
+#else
void __napi_schedule_irqoff(struct napi_struct *n);
+#endif
static inline bool napi_disable_pending(struct napi_struct *n)
{
diff --git a/net/core/dev.c b/net/core/dev.c
index 2c563e0f6955..cd270a6c0f46 100644
--- a/net/core/dev.c
+++ b/net/core/dev.c
@@ -5959,6 +5959,7 @@ bool napi_schedule_prep(struct napi_struct *n)
}
EXPORT_SYMBOL(napi_schedule_prep);
+#ifndef CONFIG_PREEMPT_RT_FULL
/**
* __napi_schedule_irqoff - schedule for receive
* @n: entry to schedule
@@ -5977,6 +5978,7 @@ void __napi_schedule_irqoff(struct napi_struct *n)
__napi_schedule(n);
}
EXPORT_SYMBOL(__napi_schedule_irqoff);
+#endif
bool napi_complete_done(struct napi_struct *n, int work_done)
{
|
