Unhandled prefetch abort on mirabox with 3.11-rc7
Jochen De Smet
jochen.armkernel at leahnim.org
Tue Sep 3 13:39:17 EDT 2013
On 9/3/2013 12:14, Thomas Petazzoni wrote:
> Dear Jochen De Smet,
>
> On Tue, 03 Sep 2013 12:08:23 -0400, Jochen De Smet wrote:
>
>>>>> Keep on posting the oopses though, there may be a pattern to them.
>> Another clue in the heat direction might be that with the rain the last
>> few days things seem
>> at least a bit better, though it's probably too early to draw
>> conclusions. First box's been up
>> for 2 days 10 hours now, and the with this oops above 18 hours.
> Sorry if those questions have already been posted in the previous
> thread. What kind of tests / workload are you running on your Mirabox
> to trigger the crash? I might be able to get one or two Mirabox running
> here, so I could see if the problem is reproducible.
Nothing too fancy. They're running a corosync/pacemaker cluster with
apache, bind,
openvpn, dovecot, postfix and mysql, all for personal use only, so a
fairly light load.
CPU idle generally hovers just below 80%.
That said, I think at least one of the oopses happened while I was
updating the kernel;
IIRC it happened either during the git pull or make clean though, not
during the actual
make.
>
> Also, is this something you're seeing only since 3.11-rc7 ? Is the
> kernel originally provided with the Mirabox more stable ? Are earlier
> kernel versions (such as 3.10 or earlier) more stable ?
Things get a bit murky here.
I did not get any problems with the included (with the marvell patches)
2.6.35, but I didn't
run it all that long since I don't like debian and it didn't work with a
recent fedora because
of systemd. I was running that kernel + a cgroup patch for quite a
while without any issues.
A stock 3.10 kernel compiled on fedora 18 has been working without any
issues for at least
a month. That exact same kernel and exact same config compiled on
fedora 19 results in an
oops shortly after boot however. (First oops below)
3.11-rc3 compiled on FC19 results in another oops (second one below);
compiled on FC18
I initially thought it worked fine but I think it eventually oopsed as
well (didn't save it, sorry),
so I went back to my 3.10 kernel.
And my current 3.11-rc7 is the next one I tried. I don't have a FC18 box
anymore so kernel is
compiled on FC19 (directly on the mirabox).
One other thing with the stock kernels is that the network interfaces
will not work properly
unless they're activated from u-boot, i.e. if I just do an sdcard boot
the interfaces will show
up and appear ok but won't actually send/receive any data. Simply doing
a "dhcp ; setact
egiga1 ; dhcp" before continuing the boot makes them work fine. This
wasn't a problem with
the original kernel.
J.
>
> Thanks,
>
> Thomas
Unable to handle kernel NULL pointer dereference at virtual address 0000001c
pgd = ee0b8000
[0000001c] *pgd=2e2c2831, *pte=00000000, *ppte=00000000
Internal error: Oops: 17 [#1] ARM
Modules linked in: ipt_MASQUERADE iscsi_tcp libiscsi_tcp libiscsi
iptable_nat nf_nat_ipv4 nf_nat drbd lru_cache scsi_transport_iscsi
iptable_mangle ipt_REJECT xt_conntrack iptable_filter ip_tables ext3 jbd
autofs4 ext4 jbd2 mbcache sd_mod usb_storage mmc_block mvsdio mmc_core
ehci_orion
CPU: 0 PID: -1073560872 Comm: bash Not tainted 3.10.0-stock1 #23
task: ee1df440 ti: ee154000 task.ti: ee154000
PC is at __task_pid_nr_ns+0x40/0xa4
LR is at schedule_tail+0x44/0x64
pc : [<c0037c4c>] lr : [<c00439e0>] psr: 60000013
sp : ee155f88 ip : ee155f88 fp : ee155f94
r10: 00000000 r9 : 00000000 r8 : 00000000
r7 : 00000000 r6 : 00000000 r5 : bf000000 r4 : ee154000
r3 : ef181efc r2 : 00000000 r1 : 00000000 r0 : ee1df440
Flags: nZCv IRQs on FIQs on Mode SVC_32 ISA ARM Segment user
Control: 10c5387d Table: 2e0b8019 DAC: 00000015
Process bash (pid: -1073560872, stack limit = 0xee154230)
Stack: (0xee155f88 to 0xee156000)
5f80: ee155fac ee155f98 c00439e0 c0037c18 00000000
00000000
5fa0: 00000000 ee155fb0 c000df48 c00439a8 00000000 00000000 00000000
00000000
5fc0: b6fc3068 bed65f08 48a50000 00000078 000d6d64 b6fc3000 000d63cc
bed65f34
5fe0: b6fc34c0 bed65f08 00000a18 489aa0cc 60000010 01200011 00000000
00000000
Backtrace:
[<c0037c0c>] (__task_pid_nr_ns+0x0/0xa4) from [<c00439e0>]
(schedule_tail+0x44/0x64)
[<c004399c>] (schedule_tail+0x0/0x64) from [<c000df48>]
(ret_from_fork+0x4/0x3c)
r5:00000000 r4:00000000
Code: e0831101 e5913120 e3530000 0a00000c (e592101c)
---[ end trace 20369176bc42626e ]---
Unable to handle kernel paging request at virtual address 2e6f2e7a
pgd = c0004000
[2e6f2e7a] *pgd=00000000
Internal error: Oops: 15 [#2] ARM
Modules linked in: ipt_MASQUERADE iscsi_tcp libiscsi_tcp libiscsi
iptable_nat nf_nat_ipv4 nf_nat drbd lru_cache scsi_transport_iscsi
iptable_mangle ipt_REJECT xt_conntrack iptable_filter ip_tables ext3 jbd
autofs4 ext4 jbd2 mbcache sd_mod usb_storage mmc_block mvsdio mmc_core
ehci_orion
CPU: 0 PID: -1073560872 Comm: bash Tainted: G D 3.10.0-stock1 #23
task: ee1df440 ti: ee154000 task.ti: ee154000
PC is at acct_process+0x34/0x88
LR is at acct_process+0x20/0x88
pc : [<c005bb78>] lr : [<c005bb64>] psr: 20000013
sp : ee155d48 ip : ee155d48 fp : ee155d5c
r10: ef238080 r9 : ee1df440 r8 : 00000017
r7 : ee154000 r6 : c034afb0 r5 : e5911018 r4 : ee154020
r3 : 00000000 r2 : ee155d48 r1 : ef238080 r0 : 2e6f2e72
Flags: nzCv IRQs on FIQs on Mode SVC_32 ISA ARM Segment user
Control: 10c5387d Table: 2f35c019 DAC: 00000015
Process bash (pid: -1073560872, stack limit = 0xee154230)
Stack: (0xee155d48 to 0xee156000)
5d40: ee154020 00000000 ee155d94 ee155d60 c0022070
c005bb50
5d60: c03c45f0 00000001 ef2380b8 00000017 ee1df440 c03e0ac0 ee155d94
ee155d88
5d80: c001decc ee154000 ee155dd4 ee155d98 c0011afc c00219f4 ee154230
0000000b
5da0: 60000113 ee154000 c0356054 0000001c 00000017 ef238080 ee155f40
ef238080
5dc0: ee1df440 00000028 ee155dec ee155dd8 c02bd6dc c0011984 ee155f40
0000001c
5de0: ee155e8c ee155df0 c02c3794 c02bd67c ee155e14 ee155e00 c004374c
c00435a4
5e00: ee7b1b80 00000000 00010000 00000000 ef2380b8 00000000 c03c9ea0
00000001
5e20: ee155e64 ee155e30 c00465a0 c03c9ed8 ee1dfb78 00000400 ee155e5c
ee155e48
5e40: ffffffff 00000000 ee155e7c ee155e58 c02c3a88 c0009038 ffffffff
ef18001c
5e60: ee1df440 00000017 c02c35b0 c03c5064 0000001c ee155f40 00000000
00000000
5e80: ee155f3c ee155e90 c0008428 c02c35bc c02c22b4 c02c3af8 ee155f44
ee155ea8
5ea0: c002dab0 c0022f58 00000011 c02c128c c03f6ab0 c03c00d0 c03e07c6
ee154018
5ec0: 00000000 00000000 ee155ef4 ee155ed8 c0042cbc c00465c4 00000000
ee1df440
5ee0: 00000001 ee1df440 ee155f14 ee155ef8 c0044cf8 c0042c98 00000000
ee1df440
5f00: 00000004 ee0bcb80 c03cb00c ee154000 00000000 ee1df534 ee1df438
c0037c4c
5f20: 60000013 ffffffff ee155f74 00000000 ee155f94 ee155f40 c02c1f18
c00083f4
5f40: ee1df440 00000000 00000000 ef181efc ee154000 bf000000 00000000
00000000
5f60: 00000000 00000000 00000000 ee155f94 ee155f88 ee155f88 c00439e0
c0037c4c
5f80: 60000013 ffffffff ee155fac ee155f98 c00439e0 c0037c18 00000000
00000000
5fa0: 00000000 ee155fb0 c000df48 c00439a8 00000000 00000000 00000000
00000000
5fc0: b6fc3068 bed65f08 48a50000 00000078 000d6d64 b6fc3000 000d63cc
bed65f34
5fe0: b6fc34c0 bed65f08 00000a18 489aa0cc 60000010 01200011 00000000
00000000
Backtrace:
[<c005bb44>] (acct_process+0x0/0x88) from [<c0022070>] (do_exit+0x688/0x87c)
r5:00000000 r4:ee154020
[<c00219e8>] (do_exit+0x0/0x87c) from [<c0011afc>] (die+0x184/0x238)
r7:ee154000
[<c0011978>] (die+0x0/0x238) from [<c02bd6dc>]
(__do_kernel_fault.part.9+0x6c/0x7c)
[<c02bd670>] (__do_kernel_fault.part.9+0x0/0x7c) from [<c02c3794>]
(do_page_fault+0x1e4/0x3e4)
r7:0000001c r3:ee155f40
[<c02c35b0>] (do_page_fault+0x0/0x3e4) from [<c0008428>]
(do_DataAbort+0x40/0xa0)
[<c00083e8>] (do_DataAbort+0x0/0xa0) from [<c02c1f18>]
(__dabt_svc+0x38/0x60)
Exception stack(0xee155f40 to 0xee155f88)
5f40: ee1df440 00000000 00000000 ef181efc ee154000 bf000000 00000000
00000000
5f60: 00000000 00000000 00000000 ee155f94 ee155f88 ee155f88 c00439e0
c0037c4c
5f80: 60000013 ffffffff
r8:00000000 r7:ee155f74 r6:ffffffff r5:60000013 r4:c0037c4c
[<c0037c0c>] (__task_pid_nr_ns+0x0/0xa4) from [<c00439e0>]
(schedule_tail+0x44/0x64)
[<c004399c>] (schedule_tail+0x0/0x64) from [<c000df48>]
(ret_from_fork+0x4/0x3c)
r5:00000000 r4:00000000
Code: 089da830 e595002c e3500000 0a00000f (e5903008)
---[ end trace 20369176bc42626f ]---
Code: 089da830 e595002c e3500000 0a00000f (e5903008)
All code
========
0: 089da830 ldmeq sp, {r4, r5, fp, sp, pc}
4: e595002c ldr r0, [r5, #44] ; 0x2c
8: e3500000 cmp r0, #0
c: 0a00000f beq 0x50
10:* e5903008 ldr r3, [r0, #8] <-- trapping
instruction
Code starting with the faulting instruction
===========================================
0: e5903008 ldr r3, [r0, #8]
--- second oops ---
[ 330.307636] Unable to handle kernel paging request at virtual address
bf370a58
[ 330.314988] pgd = ee12c000
[ 330.316393] [bf370a58] *pgd=2dd00811, *pte=00000000, *ppte=00000000
[ 330.321402] Internal error: Oops: 7 [#1] ARM
[ 330.324371] Modules linked in: tun gfs2 sha1_generic drbd lru_cache
dlm sctp configfs ipt_MASQUERADE iptable_nat nf_nat_ipv4 nf_nat
iptable_mangle ipt_REJECT xt_conntrack ebtable_filter ebtables
iptable_filter ip_tables ext3 jbd autofs4 ext4 jbd2 mbcache sd_mod
usb_storage mmc_block xhci_hcd mvsdio mmc_core ehci_orion
[ 330.351619] CPU: 0 PID: 1774 Comm: lrmd Not tainted 3.11.0-rc3-stock1 #26
[ 330.357111] task: ee098540 ti: ee102000 task.ti: ee102000
[ 330.361220] PC is at copy_process.part.65+0x9ac/0xdd0
[ 330.364980] LR is at recalc_sigpending+0x20/0x70
[ 330.368299] pc : [<c001e1c0>] lr : [<c002cd48>] psr: 20000093
sp : ee103f00 ip : ee103efc fp : ee103f4c
[ 330.377183] r10: b6fde068 r9 : ed126b40 r8 : c03ffbf8
[ 330.381111] r7 : ed126c7c r6 : c0417f40 r5 : ee102000 r4 : 01200011
[ 330.386342] r3 : bf370a3c r2 : eea65a40 r1 : ee098540 r0 : 00000000
[ 330.391575] Flags: nzCv IRQs off FIQs on Mode SVC_32 ISA ARM
Segment user
[ 330.397503] Control: 10c5387d Table: 2e12c019 DAC: 00000015
[ 330.401951] Process lrmd (pid: 1774, stack limit = 0xee102230)
[ 330.406487] Stack: (0xee103f00 to 0xee104000)
[ 330.409549] 3f00: ed126cb4 00000000 ed126c3c 00000000 00000000
00000000 ee103f78 fffffff4
[ 330.416436] 3f20: ee102000 01200011 00020200 00000000 00000000
00000000 ee102000 00000000
[ 330.423323] 3f40: ee103f8c ee103f50 c001e734 c001d820 ee103efc
00000000 00000000 c0157bb4
[ 330.430211] 3f60: ee103f94 ee103f70 c00ba6b8 b6fde068 bee38828
48a50000 00000078 c000e6a8
[ 330.437098] 3f80: ee103fa4 ee103f90 c001ea18 c001e6a4 b6fde068
c00ba614 00000000 ee103fa8
[ 330.443986] 3fa0: c000e500 c001ea00 b6fde068 bee38828 01200011
00000000 00000000 00000000
[ 330.450874] 3fc0: b6fde068 bee38828 48a50000 00000078 46ba4000
b6fde000 0003c138 bee38864
[ 330.457761] 3fe0: b6fde4c0 bee38828 000006ee 489aa0cc 60000010
01200011 3ec52a3e 3ec53a3e
[ 330.464643] Backtrace:
[ 330.465805] [<c001d814>] (copy_process.part.65+0x0/0xdd0) from
[<c001e734>] (do_fork+0x9c/0x2c4)
[ 330.473303] [<c001e698>] (do_fork+0x0/0x2c4) from [<c001ea18>]
(SyS_clone+0x24/0x2c)
[ 330.479750] r8:c000e6a8 r7:00000078 r6:48a50000 r5:bee38828 r4:b6fde068
[ 330.485218] [<c001e9f4>] (SyS_clone+0x0/0x2c) from [<c000e500>]
(ret_fast_syscall+0x0/0x30)
[ 330.492279] Code: e5933138 e5893138 e59c3004 e08c3203 (e593201c)
[ 330.497077] ---[ end trace 836c3039ee5ba43a ]---
[ 333.601499] ------------[ cut here ]------------
[ 333.605836] Kernel BUG at c00c4f18 [verbose debug info unavailable]
[ 333.610809] Internal error: Oops - BUG: 0 [#2] ARM
[ 333.614300] Modules linked in: tun gfs2 sha1_generic drbd lru_cache
dlm sctp configfs ipt_MASQUERADE iptable_nat nf_nat_ipv4 nf_nat
iptable_mangle ipt_REJECT xt_conntrack ebtable_filter ebtables
iptable_filter ip_tables ext3 jbd autofs4 ext4 jbd2 mbcache sd_mod
usb_storage mmc_block xhci_hcd mvsdio mmc_core ehci_orion
[ 333.641540] CPU: 0 PID: 3498 Comm: httpd Tainted: G D
3.11.0-rc3-stock1 #26
[ 333.648077] task: ed17f480 ti: ed228000 task.ti: ed228000
[ 333.652188] PC is at dput+0x150/0x154
[ 333.654552] LR is at __fput+0x108/0x1f4
[ 333.657090] pc : [<c00c4f18>] lr : [<c00b3a78>] psr: 60000013
sp : ed229f10 ip : ed229f28 fp : ed229f24
[ 333.665974] r10: ee359b08 r9 : 00000000 r8 : 40000010
[ 333.669901] r7 : ed8fe080 r6 : ee7fb810 r5 : ed903778 r4 : ed903778
[ 333.675132] r3 : 00000000 r2 : 20000013 r1 : c04020c8 r0 : ed903778
[ 333.680364] Flags: nZCv IRQs on FIQs on Mode SVC_32 ISA ARM
Segment user
[ 333.686204] Control: 10c5387d Table: 2d1c4019 DAC: 00000015
[ 333.690653] Process httpd (pid: 3498, stack limit = 0xed228230)
[ 333.695276] Stack: (0xed229f10 to 0xed22a000)
[ 333.698336] 9f00: ee359b00
ed903778 ed229f5c ed229f28
[ 333.705224] 9f20: c00b3a78 c00c4dd4 00000000 00000000 c00b3efc
ed17f738 ed17f480 c041e694
[ 333.712111] 9f40: 00000000 c000e6a8 ed228000 00000000 ed229f6c
ed229f60 c00b3bbc c00b397c
[ 333.718999] 9f60: ed229f8c ed229f70 c00398a4 c00b3bb8 ed228010
ed228000 c000e6a8 ed229fb0
[ 333.725887] 9f80: ed229fac ed229f90 c00118b0 c0039810 00000004
b705d790 b7023a68 00000006
[ 333.732774] 9fa0: 00000000 ed229fb0 c000e540 c0011830 00000000
00000000 00000001 00000001
[ 333.739662] 9fc0: 00000004 b705d790 b7023a68 00000006 b705d8d8
b705d770 b6fcb0a8 bedac5e8
[ 333.746549] 9fe0: 00000000 bedabf00 b6bad2b8 b6b76294 60000010
00000004 00000000 00000000
[ 333.753431] Backtrace:
[ 333.754592] [<c00c4dc8>] (dput+0x0/0x154) from [<c00b3a78>]
(__fput+0x108/0x1f4)
[ 333.760692] r5:ed903778 r4:ee359b00
[ 333.762991] [<c00b3970>] (__fput+0x0/0x1f4) from [<c00b3bbc>]
(____fput+0x10/0x14)
[ 333.769277] [<c00b3bac>] (____fput+0x0/0x14) from [<c00398a4>]
(task_work_run+0xa0/0xb4)
[ 333.776086] [<c0039804>] (task_work_run+0x0/0xb4) from [<c00118b0>]
(do_work_pending+0x8c/0xac)
[ 333.783490] r7:ed229fb0 r6:c000e6a8 r5:ed228000 r4:ed228010
[ 333.787902] [<c0011824>] (do_work_pending+0x0/0xac) from [<c000e540>]
(work_pending+0xc/0x20)
[ 333.795131] r7:00000006 r6:b7023a68 r5:b705d790 r4:00000004
[ 333.799540] Code: e5820004 e2812001 e583207c eaffffe4 (e7f001f2)
[ 333.804339] ---[ end trace 836c3039ee5ba43b ]---
[ 333.799540] Code: e5820004 e2812001 e583207c eaffffe4 (e7f001f2)
All code
========
0: e5820004 str r0, [r2, #4]
4: e2812001 add r2, r1, #1
8: e583207c str r2, [r3, #124] ; 0x7c
c: eaffffe4 b 0xffffffa4
10: e7f001f2 ; <UNDEFINED> instruction:*
0xe7f001f2 <-- trapping instruction
Code starting with the faulting instruction
===========================================
0: e7f001f2 ; <UNDEFINED> instruction:
0xe7f001f2
More information about the linux-arm-kernel
mailing list