Is it posible to run FreeBSD/sparc64 on SunFire V215?

[KOT MATPOCKuH]

Hello!

Is it possible to run FreeBSD/sparc64 on SunFire V215?
This system is similar to SunFire V210/V240/V125, but have PCI-X and SAS disks.

I'm tried to boot 8.0-BETA4, but it hangs after "jumping to kernel entry":

{1} ok boot net
Boot device: /pci@1e,600000/pci@0/pci@9/pci@0/network@4  File and args:
100 Mbps FDX Link up
Requesting Internet Address for 0:14:4f:a2:ef:2c
Requesting Internet Address for 0:14:4f:a2:ef:2c
Consoles: Open Firmware console

Booting with sun4u support.

FreeBSD/sparc64 bootstrap loader, Revision 1.0
(root@..., Sun Sep  6 10:07:26 UTC 2009)
bootpath="/pci@1e,600000/pci@0/pci@9/pci@0/network@4"
boot: ethernet address: 00:14:4f:a2:ef:2c
100 Mbps FDX Link up
Loading /boot/defaults/loader.conf
/boot/kernel/kernel data=0x9eb048+0x215da8 syms=[0x8+0xaca58+0x8+0x9e05e]
/
Hit [Enter] to boot immediately, or any other key for command prompt.
Booting [/boot/kernel/kernel]...
jumping to kernel entry at 0xc0088000.

As I know, OpenBSD 4.5 works on this platform, and I successfuly booted kernel.
Why FreeBSD isn't ported to v215?

--
MATPOCKuH
_______________________________________________
freebsd-sparc64@... mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-sparc64
To unsubscribe, send any mail to "freebsd-sparc64-unsubscribe@..."

[Mark Linimon-2]

On Thu, Sep 17, 2009 at 01:00:32PM +0400, KOT MATPOCKuH wrote:
> Is it possible to run FreeBSD/sparc64 on SunFire V215?

Looking at the list of dmesgs, I don't see anything for V215.  You may
be the first person to try it :-)

http://wiki.freebsd.org/FreeBSD/sparc64/dmesgs

mcl
_______________________________________________
freebsd-sparc64@... mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-sparc64
To unsubscribe, send any mail to "freebsd-sparc64-unsubscribe@..."

[Marius Strobl]

On Thu, Sep 17, 2009 at 01:00:32PM +0400, KOT MATPOCKuH wrote:
> Hello!
>
> Is it possible to run FreeBSD/sparc64 on SunFire V215?
> This system is similar to SunFire V210/V240/V125, but have PCI-X and SAS disks.

The main difference is that V215 are based on the 'Fire'
JBus to PCI-Express bridges which the mainline FreeBSD
doesn't have a driver for so far.

>
> I'm tried to boot 8.0-BETA4, but it hangs after "jumping to kernel entry":
>

<...>

>
> As I know, OpenBSD 4.5 works on this platform, and I successfuly booted kernel.
> Why FreeBSD isn't ported to v215?
>

It's not like V215 smell bad or something but adding support
for additional sparc64 machines typically always requires a
developer with the necessary knowledge, time and interest to
have access to that hardware. I recently started working on
support for V215 and I at least can already netboot the
machine (which actually is UP):
http://people.freebsd.org/~marius/v215.txt
Things still missing are at least ata(4) (which as-is just
causes a panic when compiled in) and MSI support.

Marius

_______________________________________________
freebsd-sparc64@... mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-sparc64
To unsubscribe, send any mail to "freebsd-sparc64-unsubscribe@..."

[KOT MATPOCKuH]

2009/9/18 Marius Strobl <marius@...>:

> The main difference is that V215 are based on the 'Fire'
> JBus to PCI-Express bridges which the mainline FreeBSD
> doesn't have a driver for so far.
Why is not possible to port this driver from openbsd?

> http://people.freebsd.org/~marius/v215.txt
Omg. I will try to repeat this :)

> Things still missing are at least ata(4) (which as-is just
> causes a panic when compiled in)
It's not a problem to me, I have no ata(pi) devices on this system.

> and MSI support.
What from this follows?

--
MATPOCKuH
_______________________________________________
freebsd-sparc64@... mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-sparc64
To unsubscribe, send any mail to "freebsd-sparc64-unsubscribe@..."

[KOT MATPOCKuH]

Hello!

>> http://people.freebsd.org/~marius/v215.txt
> Omg. I will try to repeat this :)
I sup'ed to today CURRENT and compiled GENERIC kernel without ata,
firewire and ipv6 support, but system again hangs after:
OK boot -sh
jumping to kernel entry at 0xc0080000.

And... Why on my system I see "jumping...", but your system says
"Booting [/boot/kernel/kernel]..." ?

What I should do for success kernel start? :)

PS. Why net boot is too slow? Loading process occupies ~15 minutes
from start to last message.

--
MATPOCKuH
_______________________________________________
freebsd-sparc64@... mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-sparc64
To unsubscribe, send any mail to "freebsd-sparc64-unsubscribe@..."

[Marius Strobl]

On Fri, Sep 18, 2009 at 10:29:43AM +0400, KOT MATPOCKuH wrote:
> 2009/9/18 Marius Strobl <marius@...>:
>
> > The main difference is that V215 are based on the 'Fire'
> > JBus to PCI-Express bridges which the mainline FreeBSD
> > doesn't have a driver for so far.
> Why is not possible to port this driver from openbsd?

Nobody said it would be impossible to port it, actually
that wouldn't make much sense though as their driver
lacks stuff and has bugs in the Fire-specific parts and
the MD and MI frameworks are incomparible anyway.

>
> > and MSI support.
> What from this follows?

Suboptimal performance as the on-board PCI devices share
INTx interrupt lines (which up to V215 actually was
rather uncommon for sun4u machines).

On Fri, Sep 18, 2009 at 02:23:49PM +0400, KOT MATPOCKuH wrote:

> Hello!
>
> >> http://people.freebsd.org/~marius/v215.txt
> > Omg. I will try to repeat this :)
> I sup'ed to today CURRENT and compiled GENERIC kernel without ata,
> firewire and ipv6 support, but system again hangs after:
> OK boot -sh
> jumping to kernel entry at 0xc0080000.
>
> And... Why on my system I see "jumping...", but your system says
> "Booting [/boot/kernel/kernel]..." ?
>
> What I should do for success kernel start? :)

Like I said, mainline FreeBSD doesn't support V215 so
far. If you like you can give the following patch a try:
http://people.freebsd.org/~marius/fire.diff
It requires head r197164 or later.

>
> PS. Why net boot is too slow? Loading process occupies ~15 minutes
> from start to last message.

I don't know and it works fine here, you'd need to
analyze the network traffic in order to get an idea
which side is the culprit. IIRC I've seen similar
effects when not using the full speed of the network
interface of a sun4u machine, i.e. for example a
Gigabit Ethernet interface with a Fast Ethernet
switch, given that the loader relies on the firmware
to provide support for the NIC there isn't much that
can be done about that.

Marius

_______________________________________________
freebsd-sparc64@... mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-sparc64
To unsubscribe, send any mail to "freebsd-sparc64-unsubscribe@..."

[KOT MATPOCKuH]

2009/9/18 Marius Strobl <marius@...>:

> On Fri, Sep 18, 2009 at 02:23:49PM +0400, KOT MATPOCKuH wrote:

> Like I said, mainline FreeBSD doesn't support V215 so
> far. If you like you can give the following patch a try:
> http://people.freebsd.org/~marius/fire.diff
> It requires head r197164 or later.
I'm tried to apply this patch, also add "device ofw_console" to kernel
configuration, but system panic after:
pcib1: <OFW PCI-PCI bridge> mem 0x100pcib0: DMC error: MMU error 0x4000000040
panic: pcib0: fatal DMC/PEC error

Full "boot -sv" log here: http://kot.spb.ru/tmp/fbsd/boot.log
Kernel configuration here: http://kot.spb.ru/tmp/fbsd/sunrise

--
MATPOCKuH
_______________________________________________
freebsd-sparc64@... mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-sparc64
To unsubscribe, send any mail to "freebsd-sparc64-unsubscribe@..."

[Gavin Atkinson]

On Fri, 18 Sep 2009, Marius Strobl wrote:

> On Fri, Sep 18, 2009 at 02:23:49PM +0400, KOT MATPOCKuH wrote:
>> PS. Why net boot is too slow? Loading process occupies ~15 minutes
>> from start to last message.
>
> I don't know and it works fine here, you'd need to
> analyze the network traffic in order to get an idea
> which side is the culprit. IIRC I've seen similar
> effects when not using the full speed of the network
> interface of a sun4u machine, i.e. for example a
> Gigabit Ethernet interface with a Fast Ethernet
> switch, given that the loader relies on the firmware
> to provide support for the NIC there isn't much that
> can be done about that.

Last time I looked at this, it looked like the problem was that for every
block transferred over the wire (which I think are 4k in size, but I may
be misremembering) the loader was calling into OpenFirmware to open the
network card, read the block, and close the network card.  Each open/close
pair took enough time to be noticeable.  I never had a chance to sit down
and fix this.

Gavin
_______________________________________________
freebsd-sparc64@... mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-sparc64
To unsubscribe, send any mail to "freebsd-sparc64-unsubscribe@..."

[Marius Strobl]

On Mon, Sep 21, 2009 at 03:09:44PM +0100, Gavin Atkinson wrote:

> On Fri, 18 Sep 2009, Marius Strobl wrote:
> >On Fri, Sep 18, 2009 at 02:23:49PM +0400, KOT MATPOCKuH wrote:
> >>PS. Why net boot is too slow? Loading process occupies ~15 minutes
> >>from start to last message.
> >
> >I don't know and it works fine here, you'd need to
> >analyze the network traffic in order to get an idea
> >which side is the culprit. IIRC I've seen similar
> >effects when not using the full speed of the network
> >interface of a sun4u machine, i.e. for example a
> >Gigabit Ethernet interface with a Fast Ethernet
> >switch, given that the loader relies on the firmware
> >to provide support for the NIC there isn't much that
> >can be done about that.
>
> Last time I looked at this, it looked like the problem was that for every
> block transferred over the wire (which I think are 4k in size, but I may
> be misremembering) the loader was calling into OpenFirmware to open the
> network card, read the block, and close the network card.  Each open/close
> pair took enough time to be noticeable.  I never had a chance to sit down
> and fix this.
>

The behaviour you describe was present between r177108
and r182919 (and before r60506), since r182919 the
loader just leaves the network interface instance
opened forever again for sparc64 so this shouldn't be
the reason for what KOT MATPOCKuH is seeing. Of course
leaving the NIC instance opened isn't the right to do
either and in fact likely is part of the problem
which triggeres the panic he is seeing, I don't see
an easy way to properly fix the loader/libstand to
not do the open-close-dance for every access though.

Marius

_______________________________________________
freebsd-sparc64@... mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-sparc64
To unsubscribe, send any mail to "freebsd-sparc64-unsubscribe@..."

[Gavin Atkinson]

On Mon, 21 Sep 2009, Marius Strobl wrote:

> The behaviour you describe was present between r177108
> and r182919 (and before r60506), since r182919 the
> loader just leaves the network interface instance
> opened forever again for sparc64 so this shouldn't be
> the reason for what KOT MATPOCKuH is seeing. Of course
> leaving the NIC instance opened isn't the right to do
> either and in fact likely is part of the problem
> which triggeres the panic he is seeing, I don't see
> an easy way to properly fix the loader/libstand to
> not do the open-close-dance for every access though.

Ah, thanks for that update - I hadn't realised one cause of this had been
fixed.  In that case, when I get a chance, I'll try to analyse why
netbooting is so slow on my machines too.

Gavin
_______________________________________________
freebsd-sparc64@... mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-sparc64
To unsubscribe, send any mail to "freebsd-sparc64-unsubscribe@..."

[Marius Strobl]

On Mon, Sep 21, 2009 at 10:46:53AM +0400, KOT MATPOCKuH wrote:

> 2009/9/18 Marius Strobl <marius@...>:
>
> > On Fri, Sep 18, 2009 at 02:23:49PM +0400, KOT MATPOCKuH wrote:
>
> > Like I said, mainline FreeBSD doesn't support V215 so
> > far. If you like you can give the following patch a try:
> > http://people.freebsd.org/~marius/fire.diff
> > It requires head r197164 or later.
> I'm tried to apply this patch, also add "device ofw_console" to kernel
> configuration, but system panic after:

You should not add ofw_console(4).

> pcib1: <OFW PCI-PCI bridge> mem 0x100pcib0: DMC error: MMU error 0x4000000040
> panic: pcib0: fatal DMC/PEC error
>

Please re-fetch fire.diff and try again.

Marius

_______________________________________________
freebsd-sparc64@... mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-sparc64
To unsubscribe, send any mail to "freebsd-sparc64-unsubscribe@..."

[KOT MATPOCKuH]

Hello!

2009/9/21 Marius Strobl <marius@...>:
>> pcib1: <OFW PCI-PCI bridge> mem 0x100pcib0: DMC error: MMU error 0x4000000040
>> panic: pcib0: fatal DMC/PEC error
> Please re-fetch fire.diff and try again.
I have a new panic logs :)
http://kot.spb.ru/tmp/fbsd/boot.log.0 (with -sv)
http://kot.spb.ru/tmp/fbsd/boot.log.1 (without)

--
MATPOCKuH
_______________________________________________
freebsd-sparc64@... mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-sparc64
To unsubscribe, send any mail to "freebsd-sparc64-unsubscribe@..."

[Marius Strobl]

On Tue, Sep 22, 2009 at 06:22:22PM +0400, KOT MATPOCKuH wrote:

> Hello!
>
> 2009/9/21 Marius Strobl <marius@...>:
> >> pcib1: <OFW PCI-PCI bridge> mem 0x100pcib0: DMC error: MMU error 0x4000000040
> >> panic: pcib0: fatal DMC/PEC error
> > Please re-fetch fire.diff and try again.
> I have a new panic logs :)
> http://kot.spb.ru/tmp/fbsd/boot.log.0 (with -sv)
> http://kot.spb.ru/tmp/fbsd/boot.log.1 (without)
>

Please try booting the same kernel using the following loader:
http://people.freebsd.org/~marius/loader

If that still results in the following panic:
pcib0: DMC error: MMU error 0x4000000040
panic: pcib0: fatal DMC/PEC error
locate the following lines in fire.c:
        /* not enabled by OpenSolaris *
        FIRE_PCI_SET(sc, FO_PCI_MMU_INT_EN,
            FO_PCI_MMU_ERR_INT_S_MASK | FO_PCI_MMU_ERR_INT_P_MASK);
and replace them with:
        FIRE_PCI_SET(sc, FO_PCI_MMU_INT_EN,
            (FO_PCI_MMU_ERR_INT_S_MASK | FO_PCI_MMU_ERR_INT_P_MASK) &
            ~(FO_PCI_MMU_ERR_INT_TTE_INV_S | FO_PCI_MMU_ERR_INT_TTE_INV_P));
Then try again with a kernel built with that change in place
(the loader shouldn't matter in this case).

Marius

_______________________________________________
freebsd-sparc64@... mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-sparc64
To unsubscribe, send any mail to "freebsd-sparc64-unsubscribe@..."

[KOT MATPOCKuH]

2009/9/23 Marius Strobl <marius@...>:

> Please try booting the same kernel using the following loader:
> http://people.freebsd.org/~marius/loader
Wow!
I'm booted the kernel! :)
boot -sv log here: http://kot.spb.ru/tmp/fbsd/boot-ok.log
Is your loader specially patched or build with some options?

--
MATPOCKuH
_______________________________________________
freebsd-sparc64@... mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-sparc64
To unsubscribe, send any mail to "freebsd-sparc64-unsubscribe@..."

[Marius Strobl]

On Wed, Sep 23, 2009 at 06:53:18PM +0400, KOT MATPOCKuH wrote:
> 2009/9/23 Marius Strobl <marius@...>:
>
> > Please try booting the same kernel using the following loader:
> > http://people.freebsd.org/~marius/loader
> Wow!
> I'm booted the kernel! :)
> boot -sv log here: http://kot.spb.ru/tmp/fbsd/boot-ok.log
> Is your loader specially patched or build with some options?
>

It's built with the attached quick hack. I think I've
actually found a solution to solve the underlying problem
properly in a MI way but haven't had time to implement
and test that, yet.

Marius

---

Index: boot/sparc64/loader/main.c
===================================================================
--- boot/sparc64/loader/main.c (revision 197069)
+++ boot/sparc64/loader/main.c (working copy)
@@ -64,6 +64,10 @@
 #include <machine/ver.h>
 #include <machine/vmparam.h>
 
+#include <netinet/in.h>
+
+#include <netif.h>
+
 #include "bootstrap.h"
 #include "libofw.h"
 #include "dev_net.h"
@@ -383,10 +387,16 @@
  return (error);
 
  printf("jumping to kernel entry at %#lx.\n", e->e_entry);
+
 #ifdef LOADER_DEBUG
  pmap_print_tlb_sun4u();
 #endif
 
+#ifdef LOADER_NET_SUPPORT
+ /* Close the network instance (see net_open()). */
+ ofwnet.netif_end(NULL);
+#endif
+
  entry = e->e_entry;
 
  OF_release((void *)heapva, HEAPSZ);

---

_______________________________________________
freebsd-sparc64@... mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-sparc64
To unsubscribe, send any mail to "freebsd-sparc64-unsubscribe@..."

[KOT MATPOCKuH]

Hello!

I'm got three host panic three times with this message:
panic: pcib12: XCB error status 0x1 interrupt status 0x8
cpuid = 0
Uptime: 1d14h51m50s
Cannot dump. Device not defined or unavailable.
Automatic reboot in 15 seconds - press a key on the console to abort

Is it information enough or I need to collect a crash dump from a system?

--
MATPOCKuH
_______________________________________________
freebsd-sparc64@... mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-sparc64
To unsubscribe, send any mail to "freebsd-sparc64-unsubscribe@..."

[Marius Strobl]

On Thu, Nov 12, 2009 at 03:50:58PM +0300, KOT MATPOCKuH wrote:

> Hello!
>
> I'm got three host panic three times with this message:
> panic: pcib12: XCB error status 0x1 interrupt status 0x8
> cpuid = 0
> Uptime: 1d14h51m50s
> Cannot dump. Device not defined or unavailable.
> Automatic reboot in 15 seconds - press a key on the console to abort
>
> Is it information enough or I need to collect a crash dump from a system?
>

A crash dump wouldn't provide information relevant for this problem.
The panic message indicates an unsolicited interrupt ACK/NACK has
occured, which actually are not fatal. Please use the current patchset
from:
http://people.freebsd.org/~marius/fire.diff
It fixes a possible cause for this problem (inappropriate interrupt
group controller assignment) as well as hopefully handles such
interrupts gracefully should they still occur. You'll need to either
watch the dev.pcib.X.jbc_unsol_int (probably 0 and 12) SYSCTLs or
boot verbosely to still get ahold of them though.

Marius

_______________________________________________
freebsd-sparc64@... mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-sparc64
To unsubscribe, send any mail to "freebsd-sparc64-unsubscribe@..."

[KOT MATPOCKuH]

2009/11/17 Marius Strobl <marius@...>:

>> I'm got three host panic three times with this message:
>> panic: pcib12: XCB error status 0x1 interrupt status 0x8
>> cpuid = 0
>> Uptime: 1d14h51m50s
>> Cannot dump. Device not defined or unavailable.
>> Automatic reboot in 15 seconds - press a key on the console to abort

> It fixes a possible cause for this problem (inappropriate interrupt
> group controller assignment) as well as hopefully handles such
> interrupts gracefully should they still occur. You'll need to either
> watch the dev.pcib.X.jbc_unsol_int (probably 0 and 12) SYSCTLs or
> boot verbosely to still get ahold of them though.
After 8 days of uptime:
10:22AM  up 8 days, 16:42, 11 users, load averages: 0.01, 0.00, 0.00
I have:
# sysctl -a | grep jbc_unsol_int
dev.pcib.0.jbc_unsol_int: 0
dev.pcib.12.jbc_unsol_int: 0

Is it indicates that the problem is solved?

--
MATPOCKuH
_______________________________________________
freebsd-sparc64@... mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-sparc64
To unsubscribe, send any mail to "freebsd-sparc64-unsubscribe@..."

[Marius Strobl]

On Thu, Nov 26, 2009 at 10:27:58AM +0300, KOT MATPOCKuH wrote:

> 2009/11/17 Marius Strobl <marius@...>:
>
> >> I'm got three host panic three times with this message:
> >> panic: pcib12: XCB error status 0x1 interrupt status 0x8
> >> cpuid = 0
> >> Uptime: 1d14h51m50s
> >> Cannot dump. Device not defined or unavailable.
> >> Automatic reboot in 15 seconds - press a key on the console to abort
>
> > It fixes a possible cause for this problem (inappropriate interrupt
> > group controller assignment) as well as hopefully handles such
> > interrupts gracefully should they still occur. You'll need to either
> > watch the dev.pcib.X.jbc_unsol_int (probably 0 and 12) SYSCTLs or
> > boot verbosely to still get ahold of them though.
> After 8 days of uptime:
> 10:22AM  up 8 days, 16:42, 11 users, load averages: 0.01, 0.00, 0.00
> I have:
> # sysctl -a | grep jbc_unsol_int
> dev.pcib.0.jbc_unsol_int: 0
> dev.pcib.12.jbc_unsol_int: 0
>
> Is it indicates that the problem is solved?

Well, I don't know how frequently you've hit this problem
before (was it always in the 1-2 days range?) but an uptime
of 8 days without seeing an unsolicited interrupt ACK/NACK
indeed suggest that the bug fixed was the culprit in this
case. I haven't seen this issue myself as I had fixed the
bug here before upgrading my v215 to SMP. FYI, pending
some final tests I think that the code for supporting
PCIe-based sun4u machines is now ready for hitting HEAD
(but given that it touches several pieces, commit it while
take a bit).

Marius

_______________________________________________
freebsd-sparc64@... mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-sparc64
To unsubscribe, send any mail to "freebsd-sparc64-unsubscribe@..."

[KOT MATPOCKuH]

2009/11/30 Marius Strobl <marius@...>:

> Well, I don't know how frequently you've hit this problem
> before (was it always in the 1-2 days range?)
Before applying the patch, reboot happened every night and sometimes
during the day.

> FYI, pending
> some final tests I think that the code for supporting
> PCIe-based sun4u machines is now ready for hitting HEAD
> (but given that it touches several pieces, commit it while
> take a bit).
I'm looking forward to this event :)
Thanks you for support!

--
MATPOCKuH
_______________________________________________
freebsd-sparc64@... mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-sparc64
To unsubscribe, send any mail to "freebsd-sparc64-unsubscribe@..."