[LEDE-DEV] skb_warn_bad_offload + crash at high load on ipq806x

Josh Bendavid joshbendavid at gmail.com
Wed Jun 15 03:33:29 PDT 2016 

Hi,
Testing with Tp-Link Archer C2600 (ipq806x) after latest changes (move
to 4.4 kernel + restore dma ethernet corruption fix), I have random
reboots on high wired network load.

Managed to get the kernel output during the crash below*.  Note that I
also had similar crashes with 3.18 kernel, but did not get the kernel
output from those to strictly confirm they are the same.

Not clear to me whether this is a platform specific issue, a more
general upstream bug, or something related to openwrt/lede patches.

*
Wed Jun 15 04:24:09 2016 kern.warn kernel: [  693.925294]
------------[ cut here ]------------
Wed Jun 15 04:24:09 2016 kern.warn kernel: [  693.925337] WARNING:
CPU: 0 PID: 3 at net/core/dev.c:2422 skb_warn_bad_offload+0xe4/0xf8()
Wed Jun 15 04:24:09 2016 kern.warn kernel: [  693.929065] br-lan:
caps=(0x000000801fd978a9, 0x0000000000000000) len=1500 data_len=0
gso_size=3426 gso_type=35530 ip_summed=1
Wed Jun 15 04:24:09 2016 kern.warn kernel: [  693.937164] Modules
linked in: pppoe ppp_async iptable_nat pppox ppp_generic nf_nat_ipv4
nf_conntrack_ipv6 nf_conntrack_ipv4 ipt_REJECT ipt_MASQUERADE xt_time
xt_tcpudp xt_state xt_nat xt_multiport xt_mark xt_mac xt_limit xt_id
xt_conntrack xt_comment xt_TCPMSS xt_REDIRECT xt_LOG xt_CT slhc
nf_reject_ipv4 nf_nat_redirect nf_nat_masquerade_ipv4 nf_nat
nf_log_ipv4 nf_defrag_ipv6 nf_defrag_ipv4 nf_conntrack_rtcache
nf_conntrack iptable_raw iptable_mangle iptable_fWed Jun 15 04:24:09
2016 kern.warn kernel: [  694.027758] CPU: 0 PID: 3 Comm: ksoftirqd/0
Not tainted 4.4.13 #1
Wed Jun 15 04:24:09 2016 kern.warn kernel: [  694.028021] Hardware
name: Qualcomm (Flattened Device Tree)
Wed Jun 15 04:24:09 2016 kern.warn kernel: [  694.034118] [<c021fea4>]
(unwind_backtrace) from [<c021cb0c>] (show_stack+0x10/0x14)
Wed Jun 15 04:24:09 2016 kern.warn kernel: [  694.039497] [<c021cb0c>]
(show_stack) from [<c03a18e8>] (dump_stack+0x88/0x9c)
Wed Jun 15 04:24:09 2016 kern.warn kernel: [  694.047482] [<c03a18e8>]
(dump_stack) from [<c0227a4c>] (warn_slowpath_common+0x94/0xb0)
Wed Jun 15 04:24:09 2016 kern.warn kernel: [  694.054510] [<c0227a4c>]
(warn_slowpath_common) from [<c0227a98>] (warn_slowpath_fmt+0x30/0x40)
Wed Jun 15 04:24:09 2016 kern.warn kernel: [  694.062759] [<c0227a98>]
(warn_slowpath_fmt) from [<c04c50a8>] (skb_warn_bad_offload+0xe4/0xf8)
Wed Jun 15 04:24:09 2016 kern.warn kernel: [  694.071179] [<c04c50a8>]
(skb_warn_bad_offload) from [<c04c95e0>] (__skb_gso_segment+0x4c/0xd0)
Wed Jun 15 04:24:09 2016 kern.warn kernel: [  694.079858] [<c04c95e0>]
(__skb_gso_segment) from [<c04fb00c>] (ip_finish_output+0x12c/0x1fc)
Wed Jun 15 04:24:09 2016 kern.warn kernel: [  694.088536] [<c04fb00c>]
(ip_finish_output) from [<c04fb994>] (ip_output+0x4c/0xb8)
Wed Jun 15 04:24:09 2016 kern.warn kernel: [  694.097223] [<c04fb994>]
(ip_output) from [<c04f84ac>] (ip_forward+0x278/0x38c)
Wed Jun 15 04:24:09 2016 kern.warn kernel: [  694.104680] [<c04f84ac>]
(ip_forward) from [<c04f7138>] (ip_rcv+0x244/0x2cc)
Wed Jun 15 04:24:09 2016 kern.warn kernel: [  694.111971] [<c04f7138>]
(ip_rcv) from [<c04c59ec>] (__netif_receive_skb_core+0x640/0x7a4)
Wed Jun 15 04:24:09 2016 kern.warn kernel: [  694.119265] [<c04c59ec>]
(__netif_receive_skb_core) from [<c04c82f4>]
(process_backlog+0x9c/0x134)
Wed Jun 15 04:24:09 2016 kern.warn kernel: [  694.127340] [<c04c82f4>]
(process_backlog) from [<c04c7a5c>] (net_rx_action+0x120/0x2c8)
Wed Jun 15 04:24:09 2016 kern.warn kernel: [  694.136282] [<c04c7a5c>]
(net_rx_action) from [<c022b2c0>] (__do_softirq+0xe0/0x21c)
Wed Jun 15 04:24:09 2016 kern.warn kernel: [  694.144529] [<c022b2c0>]
(__do_softirq) from [<c022b420>] (run_ksoftirqd+0x24/0x44)
Wed Jun 15 04:24:09 2016 kern.warn kernel: [  694.152255] [<c022b420>]
(run_ksoftirqd) from [<c0243fcc>] (smpboot_thread_fn+0x24c/0x288)
Wed Jun 15 04:24:09 2016 kern.warn kernel: [  694.159638] [<c0243fcc>]
(smpboot_thread_fn) from [<c0241540>] (kthread+0xf0/0xf8)
Wed Jun 15 04:24:09 2016 kern.warn kernel: [  694.167966] [<c0241540>]
(kthread) from [<c0209cb8>] (ret_from_fork+0x14/0x3c)
Wed Jun 15 04:24:09 2016 kern.warn kernel: [  694.175575] ---[ end
trace 26d6bdf0ce98e93b ]---
Wed Jun 15 04:24:14 2016 kern.alert kernel: [  694.182754] Unable to
handle kernel paging request at virtual address e20fe366
Wed Jun 15 04:24:14 2016 kern.alert kernel: [  694.187486] pgd = da464000
Wed Jun 15 04:24:14 2016 kern.alert kernel: [  694.194510] [e20fe366]
*pgd=00000000
Wed Jun 15 04:24:14 2016 kern.emerg kernel: [  694.200587] Internal
error: Oops: 5 [#1] PREEMPT SMP ARM
Wed Jun 15 04:24:14 2016 kern.warn kernel: [  694.200853] Modules
linked in: pppoe ppp_async iptable_nat pppox ppp_generic nf_nat_ipv4
nf_conntrack_ipv6 nf_conntrack_ipv4 ipt_REJECT ipt_MASQUERADE xt_time
xt_tcpudp xt_state xt_nat xt_multiport xt_mark xt_mac xt_limit xt_id
xt_conntrack xt_comment xt_TCPMSS xt_REDIRECT xt_LOG xt_CT slhc
nf_reject_ipv4 nf_nat_redirect nf_nat_masquerade_ipv4 nf_nat
nf_log_ipv4 nf_defrag_ipv6 nf_defrag_ipv4 nf_conntrack_rtcache
nf_conntrack iptable_raw iptable_mangle iptable_fWed Jun 15 04:24:14
2016 kern.warn kernel: [  694.285396] CPU: 0 PID: 3 Comm: ksoftirqd/0
Tainted: G        W       4.4.13 #1
Wed Jun 15 04:24:14 2016 kern.warn kernel: [  694.285663] Hardware
name: Qualcomm (Flattened Device Tree)
Wed Jun 15 04:24:14 2016 kern.warn kernel: [  694.292779] task:
dd448880 ti: dd456000 task.ti: dd456000
Wed Jun 15 04:24:14 2016 kern.warn kernel: [  694.298340] PC is at
put_page+0x0/0x5c
Wed Jun 15 04:24:14 2016 kern.warn kernel: [  694.303894] LR is at
skb_release_data+0x6c/0xd4
Wed Jun 15 04:24:14 2016 kern.warn kernel: [  694.307536] pc :
[<c02a6898>]    lr : [<c04b9434>]    psr: 80000013
Wed Jun 15 04:24:14 2016 kern.warn kernel: [  694.307536] sp :
dd457d40  ip : 00000000  fp : c07736b0
Wed Jun 15 04:24:14 2016 kern.warn kernel: [  694.311972] r10:
da983bf0  r9 : dd41405c  r8 : ffffffea
Wed Jun 15 04:24:14 2016 kern.warn kernel: [  694.323420] r7 :
da983b40  r6 : d9d436c0  r5 : d9d436f0  r4 : 00000001
Wed Jun 15 04:24:14 2016 kern.warn kernel: [  694.328631] r3 :
000000f4  r2 : 00000000  r1 : 00000000  r0 : e20fe366
Wed Jun 15 04:24:14 2016 kern.warn kernel: [  694.335230] Flags: Nzcv
IRQs on  FIQs on  Mode SVC_32  ISA ARM  Segment none
Wed Jun 15 04:24:14 2016 kern.warn kernel: [  694.341740] Control:
10c5787d  Table: 5c46406a  DAC: 00000051
Wed Jun 15 04:24:14 2016 kern.emerg kernel: [  694.348942] Process
ksoftirqd/0 (pid: 3, stack limit = 0xdd456210)
Wed Jun 15 04:24:14 2016 kern.emerg kernel: [  694.354672] Stack:
(0xdd457d40 to 0xdd458000)
Wed Jun 15 04:24:14 2016 kern.emerg kernel: [  694.360757] 7d40:
da983b40 000005dc c07a69c0 00000000 ffffffea c04b952c da983b40
c04fb0b8
Wed Jun 15 04:24:14 2016 kern.emerg kernel: [  694.365189] 7d60:
00000001 c04f0f9c da983b40 c07a69c0 00000000 d9d43090 00000008
dd41405c
Wed Jun 15 04:24:14 2016 kern.emerg kernel: [  694.373351] 7d80:
da983bf0 c04fb994 da983b40 00000004 80000000 dd457d02 00000000
dc909000
Wed Jun 15 04:24:14 2016 kern.emerg kernel: [  694.381509] 7da0:
00000000 c07a69c0 c07a6f5c c04faee0 da983b40 dd414000 c07a69c0
c04f84ac
Wed Jun 15 04:24:14 2016 kern.emerg kernel: [  694.389669] 7dc0:
bf280284 00000002 80000000 dd457e02 dd414000 dc909000 00000000
c07a69c0
Wed Jun 15 04:24:14 2016 kern.emerg kernel: [  694.397830] 7de0:
c07a6f4c c04f81e8 da983b40 dd414000 d9d43090 dd414000 00000008
c04f7138
Wed Jun 15 04:24:14 2016 kern.emerg kernel: [  694.405989] 7e00:
05eb0404 00000000 80000000 c04c5a02 dd414000 00000000 00000000
c07a69c0
Wed Jun 15 04:24:14 2016 kern.emerg kernel: [  694.414147] 7e20:
c07a6f3c c04f6af4 da983b40 c077486c c04f6ef4 c04c59ec c0772100
00000000
Wed Jun 15 04:24:14 2016 kern.emerg kernel: [  694.422306] 7e40:
00000001 00002f54 ddc1e840 da983b40 00000001 da983b40 ddc138b8
00000002
Wed Jun 15 04:24:14 2016 kern.emerg kernel: [  694.430468] 7e60:
ddc13848 00000000 00000001 00000040 ddc138a4 c04c82f4 00000001
ddc138b0
Wed Jun 15 04:24:14 2016 kern.emerg kernel: [  694.438629] 7e80:
c0772100 ddc138b8 00000001 00000040 0000012c 000099df c076e840
dd457ec0
Wed Jun 15 04:24:14 2016 kern.emerg kernel: [  694.446787] 7ea0:
c0772100 c04c7a5c 00000003 1d4a5000 ddc13840 c077680c c07736b0
c07736b0
Wed Jun 15 04:24:14 2016 kern.emerg kernel: [  694.454947] 7ec0:
dd415600 dd414600 dd457ec8 dd457ec8 dca8cd98 00000000 00000003
c077208c
Wed Jun 15 04:24:14 2016 kern.emerg kernel: [  694.463107] 7ee0:
c0772080 40000003 dd456000 00000100 c0772080 c022b2c0 c0776d48
ddc12e40
Wed Jun 15 04:24:14 2016 kern.emerg kernel: [  694.471265] 7f00:
dd457ef8 c07ae8c0 00000004 000099de c0772100 04208040 dd404d00
dd456000
Wed Jun 15 04:24:14 2016 kern.emerg kernel: [  694.479424] 7f20:
dd404d00 00000000 00000001 c0778374 00000002 00000000 00000000
c022b420
Wed Jun 15 04:24:14 2016 kern.emerg kernel: [  694.487584] 7f40:
dd456000 c0243fcc 00000000 dd404d40 dd404d00 c0243d80 00000000
00000000
Wed Jun 15 04:24:14 2016 kern.emerg kernel: [  694.495745] 7f60:
00000000 c0241540 10106041 00000001 00000000 dd404d00 00000000
00030003
Wed Jun 15 04:24:14 2016 kern.emerg kernel: [  694.503907] 7f80:
dd457f80 dd457f80 00000000 00000000 dd457f90 dd457f90 dd457fac
dd404d40
Wed Jun 15 04:24:14 2016 kern.emerg kernel: [  694.512064] 7fa0:
c0241450 00000000 00000000 c0209cb8 00000000 00000000 00000000
00000000
Wed Jun 15 04:24:14 2016 kern.emerg kernel: [  694.520223] 7fc0:
00000000 00000000 00000000 00000000 00000000 00000000 00000000
00000000
Wed Jun 15 04:24:14 2016 kern.emerg kernel: [  694.528384] 7fe0:
00000000 00000000 00000000 00000000 00000013 00000000 90000018
41070000
Wed Jun 15 04:24:14 2016 kern.warn kernel: [  694.536545] [<c02a6898>]
(put_page) from [<c04b9434>] (skb_release_data+0x6c/0xd4)
Wed Jun 15 04:24:14 2016 kern.warn kernel: [  694.544698] [<c04b9434>]
(skb_release_data) from [<c04b952c>] (kfree_skb+0x54/0x60)
Wed Jun 15 04:24:14 2016 kern.warn kernel: [  694.552162] [<c04b952c>]
(kfree_skb) from [<c04fb0b8>] (ip_finish_output+0x1d8/0x1fc)
Wed Jun 15 04:24:14 2016 kern.warn kernel: [  694.559715] [<c04fb0b8>]
(ip_finish_output) from [<c04fb994>] (ip_output+0x4c/0xb8)
Wed Jun 15 04:24:14 2016 kern.warn kernel: [  694.567708] [<c04fb994>]
(ip_output) from [<c04f84ac>] (ip_forward+0x278/0x38c)

More information about the Lede-dev mailing list