About Section 7.2.2 "Utilities in Makefiles" of the GCS

[Vincent Lefevre]

Hi,

I have a few comments about Section 7.2.2 "Utilities in Makefiles" of
the GNU Coding Standards.


* It is written: "Write the Makefile commands (and any shell scripts,
such as configure) to run in sh, not in csh. Don't use any special
features of ksh or bash."

But there are two incompatible sh (the traditional Bourne shell, as
used by default under Solaris, and POSIX sh), and there are some
significant differences in practice. This paragraph is not clear if
the traditional Bourne shell needs to be supported. If both need to
be taken into account, the following should be added, IMHO: "[to run
in sh] (both traditional Bourne shell and POSIX shell)".


* The list of utilities includes "diff". But unfortunately, "diff" is
not present on the Maemo/OS2008 GNU/Linux distribution (e.g. on the
Nokia N810), probably to save some space. Very often, one just needs
to know whether two files are identical or different, in which case
"cmp" could be used. But it has been reported that "cmp" has problems
on some platforms, so that "cmp" is no longer used by autoconf. Could
something be said about that? Is one of them preferred, to detect
whether two files are identical or different? Could it be advised
to test the presence of "diff", and use "cmp" if "diff" couldn't be
found?


* "Stick to the generally supported options for these programs."
is vague and misleading. People might think that POSIX options are
OK despite the given example ("mkdir -p", where -p is specified by
POSIX). I even saw developers using non-POSIX options! Could this
paragraph be more specific?

Also, concerning the "test" utility, the following should probably
be added:

  With the "test" utility, avoid ambiguity. Even though the behavior
  is specified by POSIX, many implementations behave differently.
  For instance, to test whether the length of a string is zero or
  not, use an explicit "-z" or "-n".


* "It is a good idea to avoid creating symbolic links in makefiles,
since a few systems don't support them."

It should be better to mention that some *file systems* don't support
them (now that vfat-formatted Flash memory is more and more used, e.g.
on USB keys and memory cards).

--
Vincent Lefèvre <vincent@...> - Web: <http://www.vinc17.org/>
100% accessible validated (X)HTML - Blog: <http://www.vinc17.org/blog/>
Work: CR INRIA - computer arithmetic / Arenaire project (LIP, ENS-Lyon)

[Ralf Wildenhues]

Hello,

* Vincent Lefevre wrote on Wed, Mar 19, 2008 at 04:19:21PM CET:
> I have a few comments about Section 7.2.2 "Utilities in Makefiles" of
> the GNU Coding Standards.

FWIW, this was spawned by
<http://thread.gmane.org/gmane.comp.sysutils.autoconf.bugs/5961>.

> * "Stick to the generally supported options for these programs."
> is vague and misleading. People might think that POSIX options are
> OK despite the given example ("mkdir -p", where -p is specified by
> POSIX). I even saw developers using non-POSIX options! Could this
> paragraph be more specific?

Maybe it should link to the Autoconf manual section about portable shell
scripts.  But mentioning to generally not assume more than POSIX except
in a few well-known-portable cases is probably a good idea.

FWIW, mkdir -p is listed because it is not parallel build-safe on some
non-free systems.

> Also, concerning the "test" utility, the following should probably
> be added:
>
>   With the "test" utility, avoid ambiguity. Even though the behavior
>   is specified by POSIX, many implementations behave differently.
>   For instance, to test whether the length of a string is zero or
>   not, use an explicit "-z" or "-n".

This could be elegantly solved with above: a link to
<http://www.gnu.org/software/autoconf/manual/html_node/Portable-Shell.html>

which covers this and many more issues.

Cheers,
Ralf

[Karl Berry]

Hi Vincent, Ralf, all,

(Combining replies ...)

    But there are two incompatible sh (the traditional Bourne shell, as
    used by default under Solaris, and POSIX sh), and there are some

POSIX did not exist at the time that text in standards.texi was written.
You're right that it should be addressed now.

    This paragraph is not clear if the traditional Bourne shell needs to
    be supported.

Traditional sh should be supported, unless there is some overwhelming
practical reason why it cannot be.  (I don't think that is the case.)

    be taken into account, the following should be added, IMHO: "[to run
    in sh] (both traditional Bourne shell and POSIX shell)".

Agreed.

    * "It is a good idea to avoid creating symbolic links in makefiles,
    since a few systems don't support them."

    It should be better to mention that some *file systems* don't support

Agreed.

    > * "Stick to the generally supported options for these programs."
    > is vague and misleading.

Well, we can't possibly document every sh portability issue in
standards.texi.  That's not its purpose, and anyway as you all know the
Autoconf manual is already making that attempt ...

    Maybe it should link to the Autoconf manual section about portable shell
    scripts.  

Sounds good.

    But mentioning to generally not assume more than POSIX except
    in a few well-known-portable cases is probably a good idea.

Sounds good.

Can one of you propose a patch?


I'll write about the diff/cmp issue separately, since it seems more
contentious and complicated.

Thanks,
k

[Karl Berry]

    * The list of utilities includes "diff". But unfortunately, "diff"
    is not present on the Maemo/OS2008 GNU/Linux distribution (e.g. on
    the Nokia N810), probably to save some space.

How unfortunate.  Is there any chance of having that decision changed?

    Very often, one just needs to know whether two files are identical
    or different, in which case "cmp" could be used. But it has been
    reported that "cmp" has problems on some platforms,

Ralf (or anyone), do you know what these cmp "problems" are on "some
platforms"?  Oh well, I suppose there is no hope of fixing them, anyway.

    so that "cmp" is no longer used by autoconf. Could something be said
    about that? Is one of them preferred, to detect whether two files
    are identical or different?  

I don't know what could be said.  The Autoconf folks are in the best
position to know what works most widely.
   
    Could it be advised to test the presence of "diff", and use "cmp" if
    "diff" couldn't be found?

Well, that sounds like a sensible approach on the face of it.  I don't
think that standards.texi should make it mandatory, but it could be a
suggestion/explanation.  Ralf?

In general, all bets are off when the "required" utilities aren't
present anyway.  I don't know why both diff and cmp got in the list
originally.

thanks,
k

[Ralf Wildenhues]

* Karl Berry wrote on Fri, Mar 21, 2008 at 12:36:50AM CET:
>
>     Very often, one just needs to know whether two files are identical
>     or different, in which case "cmp" could be used. But it has been
>     reported that "cmp" has problems on some platforms,
>
> Ralf (or anyone), do you know what these cmp "problems" are on "some
> platforms"?  Oh well, I suppose there is no hope of fixing them, anyway.

\n vs \r\n issue, I think.  It's sometimes convenient to ignore those
differences.

>     Could it be advised to test the presence of "diff", and use "cmp" if
>     "diff" couldn't be found?
>
> Well, that sounds like a sensible approach on the face of it.  I don't
> think that standards.texi should make it mandatory, but it could be a
> suggestion/explanation.  Ralf?

It's a possibility.  Just needing to take care of diff is certainly less
of a problem than trying out all GCS-listed tools in configure.  I'll
try to write an Autoconf patch (will be a couple of days).

Cheers,
Ralf

[Ralf Wildenhues]

* Karl Berry wrote on Fri, Mar 21, 2008 at 12:29:02AM CET:
>
> Can one of you propose a patch?

Here's a quick writeup in a hurry of what was listed so far.

I should note that the Bourne sh vs. Posix sh is not an easy question.
Autoconf would like to move to shell functions in the not too distant
future, and practical experience shows that outside of museums, that
is pretty portable nowadays.  Also, virtually all systems have a posixy
shell somewhere (the problem sometimes being to find it).

Cheers,
Ralf

Index: make-stds.texi
===================================================================
RCS file: /sources/gnustandards/gnustandards/make-stds.texi,v
retrieving revision 1.51
diff -u -r1.51 make-stds.texi
--- make-stds.texi 14 Feb 2008 17:03:48 -0000 1.51
+++ make-stds.texi 21 Mar 2008 00:05:18 -0000
@@ -33,7 +33,10 @@
 @end iftex
 describes conventions for writing the Makefiles for GNU programs.
 Using Automake will help you write a Makefile that follows these
-conventions.
+conventions.  For more information on portable Makefiles, see
+@sc{posix} and @ref{Portable Make, Portable Make Programming,, autoconf,
+Autoconf}.
+
 
 @menu
 * Makefile Basics::             General conventions for Makefiles.
@@ -144,7 +147,8 @@
 @section Utilities in Makefiles
 
 Write the Makefile commands (and any shell scripts, such as
-@code{configure}) to run in @code{sh}, not in @code{csh}.  Don't use any
+@code{configure}) to run in @code{sh} (both traditional Bourne shell
+and @sc{posix} shell), not in @code{csh}.  Don't use any
 special features of @code{ksh} or @code{bash}.
 
 The @code{configure} script and the Makefile rules for building and
@@ -159,15 +163,20 @@
 ln ls mkdir mv pwd rm rmdir sed sleep sort tar test touch true
 @end example
 
+Generally, try to stick to @sc{posix} features of these tools.
+For a list of known incompatibilities, see @ref{Portable Shell,
+Portable Shell Programming,, autoconf, Autoconf}.
+
 Compression programs such as @code{gzip} can be used in the
 @code{dist} rule.
 
 Stick to the generally supported options for these programs.  For
 example, don't use @samp{mkdir -p}, convenient as it may be, because
-most systems don't support it.
+some systems don't support it at all and with others, it is not safe
+for parallel execution.
 
 It is a good idea to avoid creating symbolic links in makefiles, since a
-few systems don't support them.
+few file systems don't support them.
 
 The Makefile rules for building and installation can also use compilers
 and related programs, but should do so via @code{make} variables so that the
cvs diff: Diffing work.m
cvs diff: Diffing work.s

[Vincent Lefevre]

On 2008-03-20 18:36:50 -0500, Karl Berry wrote:
>     * The list of utilities includes "diff". But unfortunately, "diff"
>     is not present on the Maemo/OS2008 GNU/Linux distribution (e.g. on
>     the Nokia N810), probably to save some space.
>
> How unfortunate.  Is there any chance of having that decision changed?

I don't know. I reported a bug:

  https://bugs.maemo.org/show_bug.cgi?id=3032

--
Vincent Lefèvre <vincent@...> - Web: <http://www.vinc17.org/>
100% accessible validated (X)HTML - Blog: <http://www.vinc17.org/blog/>
Work: CR INRIA - computer arithmetic / Arenaire project (LIP, ENS-Lyon)

[Karl Berry]

    Here's a quick writeup in a hurry of what was listed so far.

Thanks.  I proposed it to rms, with minor changes.

    Autoconf would like to move to shell functions in the not too distant

I agree that shell functions are ok.  I don't think the wording
prohibits that.

    virtually all systems have a posixy shell

Yes, and that's fine for scripts which can search for it and exec
themselves to use it, but it doesn't help with Makefile rules -- or am I
missing something?  Those should still be least-common-denominator
vanilla sh, I think.

Thanks,
k

[Ralf Wildenhues]

Hi Kerl,

On Sat, Mar 22, 2008 at 01:11:27PM -0500, Karl Berry wrote:
>
>     virtually all systems have a posixy shell
>
> Yes, and that's fine for scripts which can search for it and exec
> themselves to use it, but it doesn't help with Makefile rules -- or am I
> missing something?  Those should still be least-common-denominator
> vanilla sh, I think.

Well, typically
  SHELL = @SHELL@

appears in Makefile.in's (in automake-generated files anyway).  Not all
make implementations obey this setting (AFAIR they are allowed by Posix
to ignore it) but most do.

Cheers,
Ralf