Piping sed stalls on OpenSolaris 2010.03

[Mats Erik Andersson-8]

Dear all,

at some stage the generated make file "lib/Makefile" began using
a three-member pipe for sed in order to generate "lib/unistd.h" and
"lib/wchar.h". This change is not portable to OpenSolaris 2010.03
using snv_134, since "make" as well as "gmake" stalls at

  GEN unistd.h

  GEN wchar.h

and only at these. The change took place somewhere between the release
of inetutils-1.8 and inetutils-1.9.1! However, neither NexentaCore with
snv_134f, nor OpenIndiana are affected by the present issue.

The problem is that "modules/unistd" and "modules/wchar" are, as sole
modules, implementing a rewrite action using the template model

    { echo "A comment";
      sed -e 's,re,txt,g' < file |
      sed -e 's,re2,txt2,g' |
      sed -e 's,re3,txt3,g'; } > outfile

All other modules implement pipes using one or two levels, not three.
This three level construct stalls on OpenSolaris 2010.03, presumably
waiting for input, since the expected output file contains only the
comment at inspection time after abortion. Rewriting the make file
to implement the shorter pipe

    { echo "A comment";
      sed -e 's,re,txt,g' < file > tmpfile ;
      sed -e 's,re2,txt2,g' < tmpfile |
      sed -e 's,re3,txt3,g'; } > outfile

fully resolves the issue with said OpenSolaris variant.

It is conceivable that the issue arises since the Korn shell
does not execute the final pipe member in a subshell, but since
neither OpenBSD, nor some other OpenIndiana/OpenSolaris systems
display the same behaviour, the matter is convoluted! Still,
I recommend a reformulation of the sed-scripting as found in
"modules/unistd" and "modules/wchar".

Best regards,
  Mats Erik Andersson, ex officio GNU Inetutils.

[Paul Eggert]

On 04/13/2012 03:32 AM, Mats Erik Andersson wrote:
>     { echo "A comment";
>       sed -e 's,re,txt,g' < file |
>       sed -e 's,re2,txt2,g' |
>       sed -e 's,re3,txt3,g'; } > outfile
>
> All other modules implement pipes using one or two levels, not three.
> This three level construct stalls on OpenSolaris 2010.03, presumably
> waiting for input

This sounds like a serious bug with OpenSolaris.
But perhaps it is a bug in the script.  Can you
reproduce the problem by invoking

  make
  [^C]
  truss -f -o truss.log make
  [^C]

and then look at truss.log and see which system calls are
going wrong and why?  If it's puzzling please feel free
to compress truss.log and send it along.

[Mats Erik Andersson-5]

fredag den 13 april 2012 klockan 07:59 skrev Paul Eggert detta:

> On 04/13/2012 03:32 AM, Mats Erik Andersson wrote:
> >     { echo "A comment";
> >       sed -e 's,re,txt,g' < file |
> >       sed -e 's,re2,txt2,g' |
> >       sed -e 's,re3,txt3,g'; } > outfile
> >
> > All other modules implement pipes using one or two levels, not three.
> > This three level construct stalls on OpenSolaris 2010.03, presumably
> > waiting for input
>
> This sounds like a serious bug with OpenSolaris.
> But perhaps it is a bug in the script.

The relevant trace part is the one given below.
Prior to the this "/usr/gnu/bin/sed", i.e., "/usr/bin/gsed"
has been put to work, using ksh93 and gmake. Same "brk" lines
were removed, and my identification of I/O has been inserted.
I find this inconclusive, except that the expected write(2)
never appears.

Best regards,
  Mats E A


   MEA: Comments as made by Mats Erik Andersson

14115: write(1, " o s e ,   " c l o s e  ".., 5120) = 5120
   MEA: Writing for line 299, lib/unistd.in.h
14115: read(0, " r   f o r   p o r t a b".., 4096) = 4096
   MEA: Reading from line 243 or 481, lib/unistd.in.h
14115: read(0, " E T C W D\n _ G L _ W A".., 4096) = 4096
   MEA: Reading from line 598, lib/unistd.in.h
14115: write(1, " o i d   _ g l _ u n r e".., 5120) = 5120
   MEA: Writing for line 466, lib/unistd.in.h
14115: read(0, " c a s t ,   b e c a u s".., 4096) = 4096
   MEA: Reading from line 708 or later, lib/unistd.in.h
14054: waitid(P_ALL, 0, 0x08046B70, WEXITED|WTRAPPED) (sleeping...)
14077: waitid(P_ALL, 0, 0x08046AE0, WEXITED|WTRAPPED) (sleeping...)
14099: waitid(P_ALL, 0, 0x08045070, WEXITED|WTRAPPED|WSTOPPED|WCONTINUED) (sleeping...)
14107: waitid(P_ALL, 0, 0x08044310, WEXITED|WTRAPPED|WSTOPPED|WCONTINUED) (sleeping...)
14109: waitid(P_ALL, 0, 0x080469C0, WEXITED|WTRAPPED) (sleeping...)
14117: read(0, 0xFEF6AFD8, 1024) (sleeping...)
14119: read(0, 0x0809991C, 5120) (sleeping...)
14111: waitid(P_ALL, 0, 0x08042B50, WEXITED|WTRAPPED|WSTOPPED|WCONTINUED) (sleeping...)
14115: write(1, 0x080A462C, 5120) (sleeping...)
14119:    Received signal #2, SIGINT, in read() [default]
14119: read(0, 0x0809991C, 5120) Err#4 EINTR
14109:    Received signal #2, SIGINT, in waitid() [caught]
14109: waitid(P_ALL, 0, 0x080469C0, WEXITED|WTRAPPED) Err#91 ERESTART
14109: lwp_sigmask(SIG_SETMASK, 0x00000002, 0x00000000) = 0xFFBFFEFF [0x0000FFFF]

[Paul Eggert]

Sorry, from that snippet I don't know which process
is which.  Can you compress it and attach the result?

Is it the case that process 14115 is writing to the
same pipe that process 14119 is reading from, and
that both processes are waiting (one on a write,
the other on a read)?  That shouldn't happen; pipes
are not supposed to do that.  Either I don't understand
the trace, or OpenSolaris 2010.03 has a serious bug in
its pipe handling.

[Bruno Haible]

Hi Mats,

> ... OpenSolaris 2010.03
> using snv_134, since "make" as well as "gmake" stalls at
>
>   GEN unistd.h
>
>   GEN wchar.h
> ...
> The problem is that "modules/unistd" and "modules/wchar" are, as sole
> modules, implementing a rewrite action using the template model
>
>     { echo "A comment";
>       sed -e 's,re,txt,g' < file |
>       sed -e 's,re2,txt2,g' |
>       sed -e 's,re3,txt3,g'; } > outfile

This is most likely the same bug that Paul has already analyzed [1]
and/or the bug that Jim has analyzed [2]. We cannot support such a buggy
system [3][4]. I would suggest you upgrade to a newer version [5].

Bruno

[1] http://lists.gnu.org/archive/html/bug-gnulib/2011-05/msg00086.html
[2] http://lists.gnu.org/archive/html/bug-gnulib/2011-04/msg00156.html
[3] http://lists.gnu.org/archive/html/bug-gnulib/2011-05/msg00091.html
[4] http://lists.gnu.org/archive/html/bug-gnulib/2011-08/msg00090.html
[5] http://wiki.openindiana.org/oi/Installing+or+Upgrading

[Vladimir 'φ-coder/phcoder' Serbinenko]

On 13.04.2012 22:06, Bruno Haible wrote:
> Hi Mats,
>
> This is most likely the same bug that Paul has already analyzed [1]
> and/or the bug that Jim has analyzed [2]. We cannot support such a buggy
> system [3][4]. I would suggest you upgrade to a newer version [5].
Would it be possible to detect it on configure time and issue
appropriate error?


--
Regards
Vladimir 'φ-coder/phcoder' Serbinenko

[Bruno Haible]

Hi Vladimir,

> > This is most likely the same bug that Paul has already analyzed [1]
> > and/or the bug that Jim has analyzed [2]. We cannot support such a buggy
> > system [3][4]. I would suggest you upgrade to a newer version [5].

> Would it be possible to detect it on configure time and issue
> appropriate error?

I wouldn't mind. Can you propose a patch? Maybe testing whether `uname -sv`
is "SunOS snv_134" is enough?

Bruno