Future of the s390 port

[Bastian Blank]

Hi folks

The s390 port was released with Lenny. However it is not in the best
condition.  There are mainly two problems which needs attention, lack of
manpower and a 64 bit userland.

The first problem is the worst.  Currently only Frans Pop and I do work
on it.  Frans only does the Debian-Installer part and I simply have not
enough time to do the rest.  The s390 architecture is quite different to
anything else, so it needs several specialized packages to work[1] and
they need lasting attention.  So if anyone wants to help (especially
Debian developers) for the continuity of this port please speak up.

The second problem is not that critical. No other distribution still
supports a complete 31 bit s390 userland and even Debian dropped the 31
bit kernel support in the meantime[2]. Strategies for an upgrade to a 64
bit userland was discussed lately[3].

I doubt that I would be able to push this port through another release
in the current state. The consequence would by that the port dies
completely and with it the only free and released distribution for this
machines.

Bastian

[1]: Mainly the kernel, generic tools (s390-tools), hardware support
(sysconfig-hardware) and a whole bunch of debian-installer packages.
[2]: <20090524185816.GA21482@...>
[3]: <20090818204335.GA6874@...>
--
It would be illogical to assume that all conditions remain stable.
                -- Spock, "The Enterprise Incident", stardate 5027.3

[Marco d'Itri]

On Aug 31, Bastian Blank <waldi@...> wrote:

> I doubt that I would be able to push this port through another release
> in the current state. The consequence would by that the port dies
> completely and with it the only free and released distribution for this
> machines.
Is this really an important problem?
Does a significant number of people actually use Debian/s390 on
production servers? And if they exist, why they are not helping?

--
ciao,
Marco

[Adam Thornton-2]

On Aug 31, 2009, at 11:28 AM, Bastian Blank wrote:

>
> The first problem is the worst.  Currently only Frans Pop and I do  
> work
> on it.  Frans only does the Debian-Installer part and I simply have  
> not
> enough time to do the rest.  The s390 architecture is quite  
> different to
> anything else, so it needs several specialized packages to work[1] and
> they need lasting attention.  So if anyone wants to help (especially
> Debian developers) for the continuity of this port please speak up.

I'd like to help; my time has become much more limited than during  
previous release cycles, though, and my access to modern zSeries  
hardware has also become more limited.

I can test installation, but on nothing anywhere near a full  
complement of device types, and I can make recommendations and  
amendments to a fair number of the sysconfig shell scripts and the  
shell scripts in the d-i packages.  And given the pressures of my day  
job I can't really guarantee that I will be able to respond to any  
specific item in a timely fashion.

I wish I could do more.

Adam


--
To UNSUBSCRIBE, email to debian-s390-REQUEST@...
with a subject of "unsubscribe". Trouble? Contact listmaster@...

[Michael Casadevall-7]

On Mon, Aug 31, 2009 at 12:46 PM, Marco d'Itri<md@...> wrote:

> On Aug 31, Bastian Blank <waldi@...> wrote:
>
>> I doubt that I would be able to push this port through another release
>> in the current state. The consequence would by that the port dies
>> completely and with it the only free and released distribution for this
>> machines.
> Is this really an important problem?
> Does a significant number of people actually use Debian/s390 on
> production servers? And if they exist, why they are not helping?
>

I think a bigger question is where do you find hardware where you can
get remote root on; I'm not very familiar with s390 or mainframes in
general, but its not a piece of hardware one individual person would
own. I'm aware of the Hercules emulator, but that doesn't seem like it
would be useful for general development of a port.
Michael

> --
> ciao,
> Marco
>
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.9 (GNU/Linux)
>
> iEYEARECAAYFAkqb/koACgkQFGfw2OHuP7G4NACfSPn2hWvMsEU1DhfSCk0M9boh
> vi0AnR2IzJJyChe76F4vORV79qdP/9hi
> =QbZi
> -----END PGP SIGNATURE-----
>
>

--
To UNSUBSCRIBE, email to debian-s390-REQUEST@...
with a subject of "unsubscribe". Trouble? Contact listmaster@...

[Adam Thornton]

On Aug 31, 2009, at 12:01 PM, Michael Casadevall wrote:

> I think a bigger question is where do you find hardware where you can
> get remote root on; I'm not very familiar with s390 or mainframes in
> general, but its not a piece of hardware one individual person would
> own. I'm aware of the Hercules emulator, but that doesn't seem like it
> would be useful for general development of a port.

The Debian project has access to a couple of machines hosted by OSDL,  
or at least it used to (I haven't actually checked the status  
recently).  These are older machines (z800?) but still 64-bit.

The death of Flex-ES and the lack of a P/390, Integrated Server, or  
even H50/H70 equivalent for zSeries has left a hole in the market for  
lower-end developers (not just in the Debian or even Linux space).  
Hercules is actually pretty useful except in that it doesn't emulate a  
lot of modern peripheral hardware, particularly the QDIO OSA  
interfaces and the Fibre Channel interfaces which are a fact of life  
on modern z boxes.

Adam


--
To UNSUBSCRIBE, email to debian-s390-REQUEST@...
with a subject of "unsubscribe". Trouble? Contact listmaster@...

[Frans Pop-3]

Marco d'Itri wrote:
> On Aug 31, Bastian Blank <waldi@...> wrote:
>> I doubt that I would be able to push this port through another release
>> in the current state. The consequence would by that the port dies
>> completely and with it the only free and released distribution for this
>> machines.
>
> Is this really an important problem?
> Does a significant number of people actually use Debian/s390 on
> production servers?

That's hard to say, but I've seen 4 or 5 separate reports from people
who've wanted to install stable on s390 systems in the past 2 months [1].
That was more than I'd expected for s390.

Cheers,
FJP

[1] We got the reports because the stable kernel for s390 was broken.


--
To UNSUBSCRIBE, email to debian-s390-REQUEST@...
with a subject of "unsubscribe". Trouble? Contact listmaster@...

[The Fungi]

I've been meaning to set up a used representative for testing in my
lab at home, and would appreciate suggestions on some of the
"smaller" options (from a space and power/thermal perspective). I've
worked with a larger S/390 sysplex in the past (unfortunately not
running Linux natively), but am curious if there's anything spported
in the ballpark of an AS/400, size-wise. Preferably something which
will run on 110V/15A power...
--
{ IRL(Jeremy_Stanley); PGP(9E8DFF2E4F5995F8FEADDC5829ABF7441FB84657);
SMTP(fungi@...); IRC(fungi@...#ccl); ICQ(114362511);
AIM(dreadazathoth); YAHOO(crawlingchaoslabs); FINGER(fungi@...);
MUD(fungi@...:6669); WWW(http://fungi.yuggoth.org/); }


--
To UNSUBSCRIBE, email to debian-s390-REQUEST@...
with a subject of "unsubscribe". Trouble? Contact listmaster@...

[Adam Thornton]

On Aug 31, 2009, at 1:47 PM, The Fungi wrote:

> I've been meaning to set up a used representative for testing in my
> lab at home, and would appreciate suggestions on some of the
> "smaller" options (from a space and power/thermal perspective). I've
> worked with a larger S/390 sysplex in the past (unfortunately not
> running Linux natively), but am curious if there's anything spported
> in the ballpark of an AS/400, size-wise. Preferably something which
> will run on 110V/15A power...

A Multiprise H50 or H70 would fit your needs, maybe....it's about that  
size, probably pretty cheap on the used market, could run z/VM 4.4.  
It is, however, 31-bit only.  There is nothing at all 64-bit yet that  
fits the bill.  All real z boxes are basically 2 rack cabinets' worth  
of stuff and don't have internal disk.

For a 64-bit box currently your only viable option in that footprint  
is Hercules.

Adam


--
To UNSUBSCRIBE, email to debian-s390-REQUEST@...
with a subject of "unsubscribe". Trouble? Contact listmaster@...

[The Fungi]

On Mon, Aug 31, 2009 at 03:39:10PM -0500, Adam Thornton wrote:
> A Multiprise H50 or H70 would fit your needs, maybe....it's about
> that size, probably pretty cheap on the used market, could run
> z/VM 4.4.  It is, however, 31-bit only.  There is nothing at all
> 64-bit yet that fits the bill.

How thorough is the 31-bit Linux support, and does it take the same
userspace binaries as 64-bit? (Feel free to RTFM me--didn't see what
I was looking for online, but then again I'm not entirely certain I
grok some of the platform specifics I was reading about either.)

> All real z boxes are basically 2 rack cabinets' worth of stuff and
> don't have internal disk.
[...]

Right--those are the same size as the ones I helped manage MVS and
USS on some years back. Definitely not something I have room or
electrical/cooling for, not to mention budget (think we paid around
US$5M for that sysplex at the time).
--
{ IRL(Jeremy_Stanley); PGP(9E8DFF2E4F5995F8FEADDC5829ABF7441FB84657);
SMTP(fungi@...); IRC(fungi@...#ccl); ICQ(114362511);
AIM(dreadazathoth); YAHOO(crawlingchaoslabs); FINGER(fungi@...);
MUD(fungi@...:6669); WWW(http://fungi.yuggoth.org/); }


--
To UNSUBSCRIBE, email to debian-s390-REQUEST@...
with a subject of "unsubscribe". Trouble? Contact listmaster@...

[Martin Grimm-3]

Am 31.08.2009 18:28 schrieb Bastian Blank:

> Hi folks
>
> The s390 port was released with Lenny. However it is not in the best
> condition.  There are mainly two problems which needs attention, lack of
> manpower and a 64 bit userland.
>
> The first problem is the worst.  Currently only Frans Pop and I do work
> on it.  Frans only does the Debian-Installer part and I simply have not
> enough time to do the rest.  The s390 architecture is quite different to
> anything else, so it needs several specialized packages to work[1] and
> they need lasting attention.  So if anyone wants to help (especially
> Debian developers) for the continuity of this port please speak up.
>
> The second problem is not that critical. No other distribution still
> supports a complete 31 bit s390 userland and even Debian dropped the 31
> bit kernel support in the meantime[2]. Strategies for an upgrade to a 64
> bit userland was discussed lately[3].
>
> I doubt that I would be able to push this port through another release
> in the current state. The consequence would by that the port dies
> completely and with it the only free and released distribution for this
> machines.
>
> Bastian
>
> [1]: Mainly the kernel, generic tools (s390-tools), hardware support
> (sysconfig-hardware) and a whole bunch of debian-installer packages.
> [2]: <20090524185816.GA21482@...>
> [3]: <20090818204335.GA6874@...>

Hi

we'd like to help.

We've currently running 240+ Linux guests on 2 IBM System z10 EC, that's
the newest hardware of this kind for those not so familiar with this
architecture. 236 of these are running Debian, mostly etch, some still
sarge and some lenny. Amongst these are zelenka.debian.org and a build
system for security updates.

We've no debian developers here and our staff is rather small but we are
willing to test new packages and contribute to existing tools in when
some debian developer can tell us where to start and what's necessary.

I've also clearance to provide more systems like zelenka to developers,
so access to current hardware should be possible.

Greetings
Martin

--
Martin Grimm
Zentrum für Informationsverarbeitung und Informationstechnik
Dienstsitz Bonn
An der Küppe 2
53225 Bonn
Tel.: [phone]
e-mail: extern.martin.grimm@...


--
To UNSUBSCRIBE, email to debian-s390-REQUEST@...
with a subject of "unsubscribe". Trouble? Contact listmaster@...

[Petter Reinholdtsen]

[Martin Grimm]
> We've currently running 240+ Linux guests on 2 IBM System z10 EC,
> that's the newest hardware of this kind for those not so familiar
> with this architecture. 236 of these are running Debian, mostly
> etch, some still sarge and some lenny. Amongst these are
> zelenka.debian.org and a build system for security updates.

One simple way to help a bit which will make it more visible to the
Debian community at large that the s390 port is used, is to install
and activate popularity-contest on these machines and thus make sure
236 machines show up on <URL: http://popcon.debian.org/ > as running
the s390 port. :)

At the moment, only 8 machines are reporting to popularity-contest,
which put s390 in line with hurd-i386, kfreebsd-amd64 and m68k as
ports with very small user groups.  That number made me and probably
others believe that very few are using the s390 port - at least until
your email came along. :)

Happy hacking,
--
Petter Reinholdtsen


--
To UNSUBSCRIBE, email to debian-s390-REQUEST@...
with a subject of "unsubscribe". Trouble? Contact listmaster@...

[Martin Grimm-3]

Am 02.09.2009 20:03 schrieb Petter Reinholdtsen:

> [Martin Grimm]
>> We've currently running 240+ Linux guests on 2 IBM System z10 EC,
>> that's the newest hardware of this kind for those not so familiar
>> with this architecture. 236 of these are running Debian, mostly
>> etch, some still sarge and some lenny. Amongst these are
>> zelenka.debian.org and a build system for security updates.
>
> One simple way to help a bit which will make it more visible to the
> Debian community at large that the s390 port is used, is to install
> and activate popularity-contest on these machines and thus make sure
> 236 machines show up on <URL: http://popcon.debian.org/ > as running
> the s390 port. :)
>
> At the moment, only 8 machines are reporting to popularity-contest,
> which put s390 in line with hurd-i386, kfreebsd-amd64 and m68k as
> ports with very small user groups.  That number made me and probably
> others believe that very few are using the s390 port - at least until
> your email came along. :)
>
> Happy hacking,

I'm aware of popcon and as much as I'd appreciate it to see our systems
counted there this will not happen because these are mainly production
systems behind firewalls or in internal networks with no internet access
and I've generally a bad feeling when thinking of software that's
talking to outside systems when there is sensitive data on my server ;-)

So as long as there is no easy manual way to provide anonymized figures
without installing software on our production servers we can't deliver
such data :-( and yes, I know there are many reasons why a manual upload
form would be a bad idea regarding accuracy and actuality.

Greetings,
Martin

--
Martin Grimm
Zentrum für Informationsverarbeitung und Informationstechnik
Dienstsitz Bonn
An der Küppe 2
53225 Bonn
Tel.: [phone]
e-mail: extern.martin.grimm@...


--
To UNSUBSCRIBE, email to debian-s390-REQUEST@...
with a subject of "unsubscribe". Trouble? Contact listmaster@...

[Bastian Blank]

On Thu, Sep 03, 2009 at 11:32:14AM +0200, Martin Grimm wrote:
> So as long as there is no easy manual way to provide anonymized figures
> without installing software on our production servers we can't deliver
> such data :-(

Hmm. You could collect the /var/lib/dpkg/status files and do a
mass submit with the data out of this files. It would lack the usage
data, but at least shows something.

Okay, this also depends on the condition that you don't consider the
package names themself sensitive.

Bastian

--
We have phasers, I vote we blast 'em!
                -- Bailey, "The Corbomite Maneuver", stardate 1514.2


--
To UNSUBSCRIBE, email to debian-s390-REQUEST@...
with a subject of "unsubscribe". Trouble? Contact listmaster@...

[Mike Hommey]

On Thu, Sep 03, 2009 at 12:16:53PM +0200, Bastian Blank <waldi@...> wrote:

> On Thu, Sep 03, 2009 at 11:32:14AM +0200, Martin Grimm wrote:
> > So as long as there is no easy manual way to provide anonymized figures
> > without installing software on our production servers we can't deliver
> > such data :-(
>
> Hmm. You could collect the /var/lib/dpkg/status files and do a
> mass submit with the data out of this files. It would lack the usage
> data, but at least shows something.
>
> Okay, this also depends on the condition that you don't consider the
> package names themself sensitive.

That could surely grant a wishlist bug for popcon, to allow to, say, only
report package names that are found in the debian archive.

Mike


--
To UNSUBSCRIBE, email to debian-s390-REQUEST@...
with a subject of "unsubscribe". Trouble? Contact listmaster@...

[Bastian Blank]

On Mon, Aug 31, 2009 at 01:01:24PM -0400, Michael Casadevall wrote:
> I think a bigger question is where do you find hardware where you can
> get remote root on;

I know at least two posibilites
- IBM provides access for evaluation purposes and
- OSDL provides access for project work.

Bastian

--
A princess should not be afraid -- not with a brave knight to protect her.
                -- McCoy, "Shore Leave", stardate 3025.3


--
To UNSUBSCRIBE, email to debian-s390-REQUEST@...
with a subject of "unsubscribe". Trouble? Contact listmaster@...

[Russ Allbery-2]

Martin Grimm <extern.martin.grimm@...> writes:

> I'm aware of popcon and as much as I'd appreciate it to see our systems
> counted there this will not happen because these are mainly production
> systems behind firewalls or in internal networks with no internet access
> and I've generally a bad feeling when thinking of software that's
> talking to outside systems when there is sensitive data on my server ;-)

We're in a similar situation, although not with s390.  In specific, our
information security office (rightfully) considers the relationship
between system and list of installed packages to be confidential data
because of the potential use of such data in determining which systems to
attack following a publicly announced vulnerability.  I can therefore only
report popcon results for a handful of personal and test systems, rather
than ~300 production servers.

Is there an easy way (read: the software already exists and I can just
install it) for all of the systems to report to an internal proxy that
then resubmits the data so that no one else can know where it's coming
from exactly other than from our servers somewhere?

--
Russ Allbery (rra@...)               <http://www.eyrie.org/~eagle/>


--
To UNSUBSCRIBE, email to debian-s390-REQUEST@...
with a subject of "unsubscribe". Trouble? Contact listmaster@...

[Adam Thornton]

On Sep 3, 2009, at 4:32 AM, Martin Grimm wrote:
>
> I'm aware of popcon and as much as I'd appreciate it to see our  
> systems
> counted there this will not happen because these are mainly production
> systems behind firewalls or in internal networks with no internet  
> access
> and I've generally a bad feeling when thinking of software that's
> talking to outside systems when there is sensitive data on my  
> server ;-)

I'm running about 20 Debian guests on z, but like Martin's, mine  
cannot actually get to the outside world directly, and that is not  
going to change.

Apt-proxy is a wonderful thing, though.

Adam


--
To UNSUBSCRIBE, email to debian-s390-REQUEST@...
with a subject of "unsubscribe". Trouble? Contact listmaster@...

[Petter Reinholdtsen]

[Russ Allbery]
> Is there an easy way (read: the software already exists and I can
> just install it) for all of the systems to report to an internal
> proxy that then resubmits the data so that no one else can know
> where it's coming from exactly other than from our servers
> somewhere?

I expect you will get this if you use HTTP to submit and any HTTP
proxy specified using the HTTP_PROXY variable in
/etc/popularity-contest.conf.  The only identity submitted would be
the random ID generated by popcon to make sure the weekly resubmission
of information replaces the last one.  Or one could use a SMTP
remailer stripping headers, I guess.  But that seem to be more work
than just using a HTTP proxy.

The information submitted is generated and available in
/var/log/popularity-contest for those that want to have a look.  There
is no information recorded about the source, except for the unique ID
of the host which is generated using

  dd if=/dev/urandom bs=1k count=1 2>/dev/null | md5sum

when the package is installed.

Happy hacking,
--
Petter Reinholdtsen


--
To UNSUBSCRIBE, email to debian-s390-REQUEST@...
with a subject of "unsubscribe". Trouble? Contact listmaster@...

[Samuel Thibault-8]

Russ Allbery, le Thu 03 Sep 2009 13:32:46 -0700, a écrit :
> In specific, our information security office (rightfully) considers
> the relationship between system and list of installed packages to
> be confidential data because of the potential use of such data in
> determining which systems to attack following a publicly announced
> vulnerability.

Could perhaps an almost empty popcon report be considered as valid at
least just for the number of installed systems?

Samuel


--
To UNSUBSCRIBE, email to debian-s390-REQUEST@...
with a subject of "unsubscribe". Trouble? Contact listmaster@...

[Christian Perrier]

Quoting Petter Reinholdtsen (pere@...):

> I expect you will get this if you use HTTP to submit and any HTTP
> proxy specified using the HTTP_PROXY variable in
> /etc/popularity-contest.conf.  The only identity submitted would be
> the random ID generated by popcon to make sure the weekly resubmission
> of information replaces the last one.  Or one could use a SMTP
> remailer stripping headers, I guess.  But that seem to be more work
> than just using a HTTP proxy.


FWIW, I have personnally always considered this as way enough to
guarantee the level of "anonymization" mentioned by Russ.

Enough to have all "my" production Debian servers at ONERA
(www.onera.fr) to report by popcon through our HTTP proxy. ONERA is a
governmental research agency which activity is focused on both
Defense-oriented research and industry-related research in aeronautics
and space. You probably get the drill: we *are* concerned about
security and confidentiality and our servers do report to popcon.

I would of course be much more reluctant to do this for servers
located directly on the Internet but we have (sigh) no Debian servers
there. We create our own security holes by using Solaris
everywhere..:-)