kernel panic on or after bootup

[David Lord-3]

I have four pcs running NetBSD 3.0. On reboot I'm regularly seeing a
kernel panic from three of them, either during bootup of within a few
minutes later. The pc that doesn't seem to show the problem is a p4-
2000, whilst slowest an amd586-133 is most likely to give the panic,
maybe 1 in 3 reboots. The k6-400 has managed uptime of 41 days and
then had to be rebooted a couple of times to use a new kernel. I've
seen the problem with GENERIC kernel but needed some extra options
and took the oportunity to remove others that weren't relevant.

Error is same:
kernel: page fault trap, code=0
Stopped at: netbsd:fr_derefrule+0x1c7: cmpl $0x2,0x88(%edx)

Any ideas as to how to track this down?


David

[Christos Zoulas-2]

In article <44AC3080.15957.784ADE@localhost>,
David Lord <david@...> wrote:

>
>I have four pcs running NetBSD 3.0. On reboot I'm regularly seeing a
>kernel panic from three of them, either during bootup of within a few
>minutes later. The pc that doesn't seem to show the problem is a p4-
>2000, whilst slowest an amd586-133 is most likely to give the panic,
>maybe 1 in 3 reboots. The k6-400 has managed uptime of 41 days and
>then had to be rebooted a couple of times to use a new kernel. I've
>seen the problem with GENERIC kernel but needed some extra options
>and took the oportunity to remove others that weren't relevant.
>
>Error is same:
>kernel: page fault trap, code=0
>Stopped at: netbsd:fr_derefrule+0x1c7: cmpl $0x2,0x88(%edx)
>
>Any ideas as to how to track this down?

You can compile the offending file adding -S -gstabs to the compile
line to produce an assembly file. Then you can objdump --disassemble
the kernel and locate the offending code. Finally locating the line
by matching the assembly instructions in the .s file should give
you the c code line number from the stabs. From there, you can look
at the c code and possibly add some debugging to help you track
the problem down.

christos

[David Lord-3]

On 5 Jul 2006, at 23:07, Christos Zoulas wrote:

> In article <44AC3080.15957.784ADE@localhost>,
> David Lord <david@...> wrote:
> >
> >I have four pcs running NetBSD 3.0. On reboot I'm regularly seeing a
> >kernel panic from three of them, either during bootup of within a few
> > minutes later. The pc that doesn't seem to show the problem is a p4-
> >2000, whilst slowest an amd586-133 is most likely to give the panic,
> >maybe 1 in 3 reboots. The k6-400 has managed uptime of 41 days and
> >then had to be rebooted a couple of times to use a new kernel. I've
> >seen the problem with GENERIC kernel but needed some extra options
> >and took the oportunity to remove others that weren't relevant.
> >
> >Error is same:
> >kernel: page fault trap, code=0
> >Stopped at: netbsd:fr_derefrule+0x1c7: cmpl $0x2,0x88(%edx)
> >
> >Any ideas as to how to track this down?
>
> You can compile the offending file adding -S -gstabs to the compile
> line to produce an assembly file. Then you can objdump --disassemble
> the kernel and locate the offending code. Finally locating the line by
> matching the assembly instructions in the .s file should give you the
> c code line number from the stabs. From there, you can look at the c
> code and possibly add some debugging to help you track the problem
> down.

OK, thanks for that. First I didn't even know what the format of the
error message was other than to me it appeared to refer to a routine
and possibly give the instruction that caused the problem.

I've just run grep on the kernel source and only found fr_derefrule
to be present in several files in /usr/src/sys/dist/ipf.

It will be next week before I get time to compile to get the assembly
code. I've also downloaded more recent ipf and will compare with
that.

Thanks

David

[Pavel Cahyna-2]

On Thu, Jul 06, 2006 at 10:56:14AM -0000, David Lord wrote:

> On 5 Jul 2006, at 23:07, Christos Zoulas wrote:
>
> > In article <44AC3080.15957.784ADE@localhost>,
> > David Lord <david@...> wrote:
> > >
> > >I have four pcs running NetBSD 3.0. On reboot I'm regularly seeing a
> > >kernel panic from three of them, either during bootup of within a few
> > > minutes later. The pc that doesn't seem to show the problem is a p4-
> > >2000, whilst slowest an amd586-133 is most likely to give the panic,
> > >maybe 1 in 3 reboots. The k6-400 has managed uptime of 41 days and
> > >then had to be rebooted a couple of times to use a new kernel. I've
> > >seen the problem with GENERIC kernel but needed some extra options
> > >and took the oportunity to remove others that weren't relevant.
> > >
> > >Error is same:
> > >kernel: page fault trap, code=0
> > >Stopped at: netbsd:fr_derefrule+0x1c7: cmpl $0x2,0x88(%edx)
> > >
> > >Any ideas as to how to track this down?
> >
> > You can compile the offending file adding -S -gstabs to the compile
> > line to produce an assembly file. Then you can objdump --disassemble
> > the kernel and locate the offending code. Finally locating the line by
> > matching the assembly instructions in the .s file should give you the
> > c code line number from the stabs. From there, you can look at the c
> > code and possibly add some debugging to help you track the problem
> > down.
>
> OK, thanks for that. First I didn't even know what the format of the
> error message was other than to me it appeared to refer to a routine
> and possibly give the instruction that caused the problem.
>
> I've just run grep on the kernel source and only found fr_derefrule
> to be present in several files in /usr/src/sys/dist/ipf.
>
> It will be next week before I get time to compile to get the assembly
> code. I've also downloaded more recent ipf and will compare with
> that.

It should be enough to compile the kernel with makeoptions DEBUG="-g".
You'll get a netbsd.gdb file which can be used to extract line number
information, like

gdb netbsd.gdb
(gdb) list *(fr_derefrule+0x1c7)

(or whathever address is shown in the panic message)

Should be simpler than disassembling..

Oh, and when this happen please also do bt/l in ddb and note the output.
You can also try to save a kernel crash dump with a
reboot 0x104
command (in ddb).

Pavel

[Das]

For me when I see that someone is having a kernel panic, two things typically, bad hardware, so be sure you really check it out, and if the hardware all checks out, then most likely, hardware compatibility issues, meaning your hardware might be listed as compatible, but it's just not that great is all.

GL  :)

[David Lord-3]

On 6 Jul 2006, at 18:10, Das (sent by Nabble.com) wrote:

>
> For me when I see that someone is having a kernel panic, two things
> typically, bad hardware, so be sure you really check it out, and if
> the hardware all checks out, then most likely, hardware compatibility
> issues, meaning your hardware might be listed as compatible, but it's
> just not that great is all.

It's just possible the hardware is faulty, all three pcs, pd6000
mini-itx, k6-400 and a 586-133, with exactly same panic?

I've also upgraded the mini-itx from 3.0 to 3.0.1, from source, which
went through more smoothly than expected until first boot of new
system when I had same panic but since it's mostly booted ok. Fourth
pc, a p4-2000, does have some hardware problem causing hang at agp
detection on some bootups, but hasn't shown the same panic, and
ipfilter isn't enabled.

David

[Das]

I would install FreeBSD, and see what happens as a test if you don't suspect hardware problems, and if FreeBSD gives you problems, then it's just hardware compatibility is all.

Then you might need to disable, or enable different hardware features, in the BIOS, like APIC, ACPI, dma, hard drive controllers, PNP Bios, etc.. From what I have seen as BSD steps more into the world of APIC, things get a bit more touchy, I'm not sure, but I think the GENERIC kernel might have some APIC support compiled in it.

Either way if it's in there, or not, it's good to find out what your board wants, or can get away with.
You might just need to recompile a kernel to get some better support by turning off alot of unnecessary things that might just be fouling it up.

BSD hardware support can be problematic at times.

[Pavel Cahyna-2]

On Fri, Jul 07, 2006 at 10:42:46AM -0000, David Lord wrote:

> On 6 Jul 2006, at 18:10, Das (sent by Nabble.com) wrote:
>
> >
> > For me when I see that someone is having a kernel panic, two things
> > typically, bad hardware, so be sure you really check it out, and if
> > the hardware all checks out, then most likely, hardware compatibility
> > issues, meaning your hardware might be listed as compatible, but it's
> > just not that great is all.
>
> It's just possible the hardware is faulty, all three pcs, pd6000
> mini-itx, k6-400 and a 586-133, with exactly same panic?

I think that in this case it's quite clear that it is some bug in IPFilter.

Pavel