debian/rules "make -f" restriction

[Tobias Grimm]

Hello!

Debian Policy 4.9 says about debian/rules:

"It must start with the line #!/usr/bin/make -f, so that it can be
invoked by saying its name rather than invoking make explicitly."

In the VDR and VDR plugin packages, we use something like this:

/bin/sh debian/make-special-vdr.sh

make-special-vdr.sh [1] normally simply does a '/usr/bin/make -f "$@"'.
But when a special environment variable is set, it modifies the source
package to build binaries with a different name.

The reason for this is, that this allows us to build VDR and VDR plugin
packages for the current development version of VDR, which can be
installed side-by-side to the normal VDR packages, allowing the
development version to be easily tested without needing to replace the
stable VDR packages.

This makes it easier for users to take a peek at newer VDR versions,
before they become stable and will be released in Debian, which we
believe is good, because it helps to improve the code quality.

Until now, we silently ignored the Lintian warning about this possible
policy violation, because the condition "so that it can be invoked by
saying its name rather than invoking make explicitly." still is true.
And it still can be built by invoking make explicitly.

Manoj Srivastava now filed some (automatic) bug reports about this,
so I'm seeking advice on what to do.

Should the policy be changed to something like:

"It must be able to be invoked by saying its name or by invoking make
explicitly. This can usually be done by using #!/usr/bin/make -f" as
the shebang line."

Or should we just add a Linitan override? Or do we really need to use
"#!/usr/bin/make -f" as the shebang line in debian/rules?

Personally I would vote for dropping the make requirement from the
policy all together. I might be mistaken, but I think none of the
build tools calls make explicitly with debian/rules. A debian/rules
might even be a Python or Rake script.

Tobias

[1]:
http://svn.opensourcefactory.com/svn/vdr/trunk/debian/make-special-vdr.sh


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@...
with a subject of "unsubscribe". Trouble? Contact listmaster@...

[Julien Cristau-6]

On Wed, 2009-10-28 at 16:02 +0100, Tobi wrote:
> [1]:
> http://svn.opensourcefactory.com/svn/vdr/trunk/debian/make-special-vdr.sh
>
>
asks for a password.  also nothing in what you said explains why you
can't do what you want using a makefile.

Cheers,
Julien


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@...
with a subject of "unsubscribe". Trouble? Contact listmaster@...

[Tobias Grimm]

Julien Cristau schrieb:

> asks for a password.

Sorry, wrong link:

http://svn.debian.org/wsvn/pkg-vdr-dvb/vdr/vdr/trunk/debian/make-special-vdr.sh

 >  also nothing in what you said explains why you
> can't do what you want using a makefile.

Because make-special-vdr.sh needs to modify debian/rules itself. This
way debian/rules doesn't get "contaminated" with stuff that goes beyond
the scope of building the regular Debian package -e except for the
shebang line.

Tobias




--
To UNSUBSCRIBE, email to debian-devel-REQUEST@...
with a subject of "unsubscribe". Trouble? Contact listmaster@...

[Manoj Srivastava]

On Wed, Oct 28 2009, Tobi wrote:

> Hello!
>
> Debian Policy 4.9 says about debian/rules:
>
> "It must start with the line #!/usr/bin/make -f, so that it can be
> invoked by saying its name rather than invoking make explicitly."
>
> In the VDR and VDR plugin packages, we use something like this:
>
> /bin/sh debian/make-special-vdr.sh
>
> make-special-vdr.sh [1] normally simply does a '/usr/bin/make -f "$@"'.
> But when a special environment variable is set, it modifies the source
> package to build binaries with a different name.

        I am pretty sure this can easily be done in a make file, and I
 suspect it is fairly trivial to do so.

> The reason for this is, that this allows us to build VDR and VDR
> plugin packages for the current development version of VDR, which can
> be installed side-by-side to the normal VDR packages, allowing the
> development version to be easily tested without needing to replace the
> stable VDR packages.

        I do not see why this needs a indirection using a shell script.

> This makes it easier for users to take a peek at newer VDR versions,
> before they become stable and will be released in Debian, which we
> believe is good, because it helps to improve the code quality.

        Can you show us a technical reason why this can't be done using
 a makefile?  This smacks much more of a personal preference than a
 technical need.

> Until now, we silently ignored the Lintian warning about this possible
> policy violation, because the condition "so that it can be invoked by
> saying its name rather than invoking make explicitly." still is true.
> And it still can be built by invoking make explicitly.
>
> Manoj Srivastava now filed some (automatic) bug reports about this,
> so I'm seeking advice on what to do.

        They were all manually filed, with checks.

> Should the policy be changed to something like:
>
> "It must be able to be invoked by saying its name or by invoking make
> explicitly. This can usually be done by using #!/usr/bin/make -f" as
> the shebang line."
>
> Or should we just add a Linitan override? Or do we really need to use
> "#!/usr/bin/make -f" as the shebang line in debian/rules?
>
> Personally I would vote for dropping the make requirement from the
> policy all together. I might be mistaken, but I think none of the
> build tools calls make explicitly with debian/rules. A debian/rules
> might even be a Python or Rake script.

        There are a number of features that one may rely on if
 ./debian/rules is a Makefile. It has a uniform calling convention, you
 can specify -n, -d, -p, and -t and get consistent results, you may pass
 options using MAKEFLAGS, other Maekfiles may include debian/rules for
 debugging and statistical  purposes (Yes, I have done that).

        There is also a clear way of passing in variable values, and
 overriding internal variables on the command line.

        Finally, most of the source packages already conform to this
 policy rule, with the vdr packages being the major standouts. To change
 policy, especially one which the project mostly complies with, you must
 come up  with a more compelling reason than "We like doing things our
 way".


> [1]:
> http://svn.opensourcefactory.com/svn/vdr/trunk/debian/make-special-vdr.sh

        This seems to want a user name and password.

        manoj
--
Every journalist has a novel in him, which is an excellent place for it.
Manoj Srivastava <srivasta@...> <http://www.debian.org/~srivasta/>  
1024D/BF24424C print 4966 F272 D093 B493 410B  924B 21BA DABB BF24 424C


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@...
with a subject of "unsubscribe". Trouble? Contact listmaster@...

[Fabian Greffrath-6]

> Because make-special-vdr.sh needs to modify debian/rules itself.
> This way debian/rules doesn't get "contaminated" with stuff that
> goes beyond the scope of building the regular Debian package -e
> except for the shebang line.

Why not so it the other way round, i.e. start two different scripts
(or the same script with different parameters) from a debian/rules
Makefile depending on the environment variable?

--
Dipl.-Phys. Fabian Greffrath

Ruhr-Universität Bochum
Lehrstuhl für Energieanlagen und Energieprozesstechnik (LEAT)
Universitätsstr. 150, IB 3/134
D-44780 Bochum

Telefon: +49 (0)234 / 32-26334
Fax:     +49 (0)234 / 32-14227
E-Mail:  greffrath@...


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@...
with a subject of "unsubscribe". Trouble? Contact listmaster@...

[Manoj Srivastava]

On Wed, Oct 28 2009, Tobi wrote:

> Julien Cristau schrieb:
>
>> asks for a password.
>
> Sorry, wrong link:
>
> http://svn.debian.org/wsvn/pkg-vdr-dvb/vdr/vdr/trunk/debian/make-special-vdr.sh
>
>>  also nothing in what you said explains why you
>> can't do what you want using a makefile.
>
> Because make-special-vdr.sh needs to modify debian/rules itself. This
> way debian/rules doesn't get "contaminated" with stuff that goes
> beyond the scope of building the regular Debian package -e except for
> the shebang line.

        This is what the make directive 'include' is all
 about. Conditionally, include fileA or fileB. Each file is all
 uncontaminated now.

        This is not a technical  shortcoming of using Makefiles.

        manoj
--
The reason why worry kills more people than work is that more people
worry than work.
Manoj Srivastava <srivasta@...> <http://www.debian.org/~srivasta/>  
1024D/BF24424C print 4966 F272 D093 B493 410B  924B 21BA DABB BF24 424C


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@...
with a subject of "unsubscribe". Trouble? Contact listmaster@...

[Bernd Zeimetz]

Tobi wrote:
> Or should we just add a Linitan override? Or do we really need to use
> "#!/usr/bin/make -f" as the shebang line in debian/rules?

Use make. it is able to do all the things you're doing right now, including to
do different stuff based on an environment setting.


> Personally I would vote for dropping the make requirement from the
> policy all together. I might be mistaken, but I think none of the
> build tools calls make explicitly with debian/rules. A debian/rules
> might even be a Python or Rake script.

Oh god, no. And I'm not even going to explain why.Think about it.

>
> Tobias
>
> [1]:
> http://svn.opensourcefactory.com/svn/vdr/trunk/debian/make-special-vdr.sh
>
>


--
 Bernd Zeimetz                            Debian GNU/Linux Developer
 http://bzed.de                                http://www.debian.org
 GPG Fingerprints: 06C8 C9A2 EAAD E37E 5B2C BE93 067A AD04 C93B FF79
                   ECA1 E3F2 8E11 2432 D485 DD95 EB36 171A 6FF9 435F


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@...
with a subject of "unsubscribe". Trouble? Contact listmaster@...

[Peter Samuelson]

> > Personally I would vote for dropping the make requirement from the
> > policy all together. I might be mistaken, but I think none of the
> > build tools calls make explicitly with debian/rules. A debian/rules
> > might even be a Python or Rake script.

[Bernd Zeimetz]
> Oh god, no. And I'm not even going to explain why.Think about it.

That's a very unhelpful response.  If that's all you had to say, why
even bother to reply?  Was it just so you could sound smart, or make
Tobias sound dumb?  At least he bothered to explain his reasoning.

See Manoj's mail for a perfect example of explaining (a) why they don't
need the hack, and (b) why it is useful to require 'make'.
--
Peter Samuelson | org-tld!p12n!peter | http://p12n.org/


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@...
with a subject of "unsubscribe". Trouble? Contact listmaster@...

[Tobias Grimm]

Fabian Greffrath wrote:

> Why not so it the other way round, i.e. start two different scripts (or
> the same script with different parameters) from a debian/rules Makefile
> depending on the environment variable?

Might be possible, but it would require major changes to debian/rules, but
our goal is to keep debian/rules as simple as possible without any stuff
you wouldn't expect in a Debian package.

Before adding any ugly stuff to debian/rules I would rather let an
external build script to the dirty work. That's just harder to integrate
into our pbuilder build process.

Tobias




--
To UNSUBSCRIBE, email to debian-devel-REQUEST@...
with a subject of "unsubscribe". Trouble? Contact listmaster@...

[Tobias Grimm]

Manoj Srivastava wrote:

>         This is what the make directive 'include' is all
>  about. Conditionally, include fileA or fileB. Each file is all
>  uncontaminated now.
>
>         This is not a technical  shortcoming of using Makefiles.

You're right. What we do might be possible from "within" the Makefile
itself. Maybe even a custom cdbs rule might be possible. But it's not that
easy and it would make the debian/rules less readable.

The solution we have right now is in some way "elegant", because you have
only to deal with a standard debian/rules and besides the different
shebang line there's nothing else to care about.

But putting the technical aspect completeley aside - with our "hack", the
debian/rules still bahaves as it should be. You can run "debian/rules" and
you can run "make -f debian/rules". It's still a "self executing" Makefile.

IMHO the policy is a little bit over-specific, when stating "It must start
with the line #!/usr/bin/make -f".

It seems nobody else has ever thought about changing the shebang line of
debian/rules, so most likely the policy will not get changed just because
I stumbled upon this issue.

So what about just adding a Linitan override and leave everything else as
it is? Our debian/rules still follows the "spirit" of the Debian policy,
even if it does not start with "#!/usr/bin/make -f".

Tobias


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@...
with a subject of "unsubscribe". Trouble? Contact listmaster@...

[Manoj Srivastava]

On Wed, Oct 28 2009, Tobi wrote:

> Manoj Srivastava wrote:
>
>>         This is what the make directive 'include' is all
>>  about. Conditionally, include fileA or fileB. Each file is all
>>  uncontaminated now.
>>
>>         This is not a technical  shortcoming of using Makefiles.
>
> You're right. What we do might be possible from "within" the Makefile
> itself. Maybe even a custom cdbs rule might be possible. But it's not that
> easy and it would make the debian/rules less readable.

        I beg to differ.  It is really trivial, and it does not make the
        rules file less readable

#!/usr/bin/make -f
ifeq (,$(srip $(ENV_VAR_WE_LOOK_FOR)))
include regular.mk
else
include special.mk
endif

        Done.

> The solution we have right now is in some way "elegant", because you have
> only to deal with a standard debian/rules and besides the different
> shebang line there's nothing else to care about.

        Actually, there is. My states makefile can no longer
include debian/rules, because, you see, it is not a makefile. There are
other ways that it does not look like a makefile, walk like a makefile,
or quack like one.


> But putting the technical aspect completeley aside - with our "hack",
> the debian/rules still bahaves as it should be. You can run
> "debian/rules" and you can run "make -f debian/rules". It's still a
> "self executing" Makefile.

        That is just one aspect.

> IMHO the policy is a little bit over-specific, when stating "It must start
> with the line #!/usr/bin/make -f".

        It is also a policy that everyone else seems to follow.

> It seems nobody else has ever thought about changing the shebang line
> of debian/rules, so most likely the policy will not get changed just
> because I stumbled upon this issue.

        We have thought of it, and rejected it as needless inconsistency.


> So what about just adding a Linitan override and leave everything else
> as it is? Our debian/rules still follows the "spirit" of the Debian
> policy, even if it does not start with "#!/usr/bin/make -f".

        I do not think it follows the policy, no.

        manoj
--
Karl's version of Parkinson's Law: Work expands to exceed the time
allotted it.
Manoj Srivastava <srivasta@...> <http://www.debian.org/~srivasta/>  
1024D/BF24424C print 4966 F272 D093 B493 410B  924B 21BA DABB BF24 424C


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@...
with a subject of "unsubscribe". Trouble? Contact listmaster@...

[Manoj Srivastava]

On Wed, Oct 28 2009, Tobi wrote:

> Fabian Greffrath wrote:
>
>> Why not so it the other way round, i.e. start two different scripts (or
>> the same script with different parameters) from a debian/rules Makefile
>> depending on the environment variable?
>
> Might be possible, but it would require major changes to debian/rules,
> but our goal is to keep debian/rules as simple as possible without any
> stuff you wouldn't expect in a Debian package.

        It requires almost no changes to the rules file.

> Before adding any ugly stuff to debian/rules I would rather let an
> external build script to the dirty work. That's just harder to
> integrate into our pbuilder build process.

        It is not at all harder to include it into pbuilder.

        manoj
--
Houdini escaping from New Jersey! Film at eleven.
Manoj Srivastava <srivasta@...> <http://www.debian.org/~srivasta/>  
1024D/BF24424C print 4966 F272 D093 B493 410B  924B 21BA DABB BF24 424C


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@...
with a subject of "unsubscribe". Trouble? Contact listmaster@...

[Charles Plessy-12]

Le Wed, Oct 28, 2009 at 04:02:32PM +0100, Tobi a écrit :
>
> Debian Policy 4.9 says about debian/rules:
>
> "It must start with the line #!/usr/bin/make -f, so that it can be  
> invoked by saying its name rather than invoking make explicitly."

Dear all,

I also do not understand that rule. There are a larger number of packages that
are simply removing all the content from the make file, for instance like:

  #!/usr/bin/make -f
  %:
  dh $@

or

  #!/usr/bin/make -f
  include /usr/share/R/debian/r-cran.mk

or any other variation on the theme.

I think that the key part of the Policy is the interface: debian/rules can be
called with arguments such as ‘build’, ‘clean’, etc. When unique features of
GNU Make are not needed, I do not see much advantage in requiring that the
parts that actually do the work are wrapped into a makefile. Can’t we just
trust the maintainers instead of putting restrictions that in the end are only
increasing complexity for no benefit?

You know, this ‘Do-O-cracy’ stuff that is supposed to make Debian different and
is progrssively becoming a ‘Do-what-I-say’ with increasing archive restrictions
and penury of DDs caused by a too long recruitment process.

Have a nice day,

--
Charles Plessy
Tsurumi, Kanagawa, Japan


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@...
with a subject of "unsubscribe". Trouble? Contact listmaster@...

[Ryan Niebur-3]

On Wed, Oct 28, 2009 at 07:05:30PM -0500, Manoj Srivastava wrote:

> On Wed, Oct 28 2009, Tobi wrote:
>
> > Manoj Srivastava wrote:
> >
> >>         This is what the make directive 'include' is all
> >>  about. Conditionally, include fileA or fileB. Each file is all
> >>  uncontaminated now.
> >>
> >>         This is not a technical  shortcoming of using Makefiles.
> >
> > You're right. What we do might be possible from "within" the Makefile
> > itself. Maybe even a custom cdbs rule might be possible. But it's not that
> > easy and it would make the debian/rules less readable.
>
>         I beg to differ.  It is really trivial, and it does not make the
>         rules file less readable
>
> #!/usr/bin/make -f
> ifeq (,$(srip $(ENV_VAR_WE_LOOK_FOR)))
> include regular.mk
> else
> include special.mk
> endif
>

ummm, I don't think so. Have you actually looked at the packages? I'd
suggest trying to provide a patch for one of them, because looking at
this now it seems very non-trivial to "fix"...and this is from an
outsider just like you, I've never even heard of these packages (just
did a quick apt-get source and looked around).

I don't see what the problem is...this really feels like bikeshedding
to me. why not just let them do it their way? it's still a Makefile
from every point that matters (just not "head -1").

Honestly this doesn't affect me so I don't care whether or not you let
them do this, but I just think you shouldn't call it "trivial", it (at
least to me) clearly isn't, and your proposed solution doesn't provide
the ellegance that their current one does.

--
_________________________
Ryan Niebur
ryanryan52@...

[Michael Tautschnig-4]

> Le Wed, Oct 28, 2009 at 04:02:32PM +0100, Tobi a écrit :

> >
> > Debian Policy 4.9 says about debian/rules:
> >
> > "It must start with the line #!/usr/bin/make -f, so that it can be  
> > invoked by saying its name rather than invoking make explicitly."
>
> Dear all,
>
> I also do not understand that rule. There are a larger number of packages that
> are simply removing all the content from the make file, for instance like:
>

[...]

I don't think there's a point in disussing whether or not the shebang-line must
be there (and that it must read as stated in policy). The fundamental point is
that debian/rules must be a Makefile (and not just looks like a Makefile).
Debian Policy 4.9 guarantees that the behavior of debian/rules will be the same
if called as either make -f debian/rules or simply debian/rules.

>
> I think that the key part of the Policy is the interface: debian/rules can be
> called with arguments such as ‘build’, ‘clean’, etc. When unique features of
> GNU Make are not needed, I do not see much advantage in requiring that the
> parts that actually do the work are wrapped into a makefile. Can’t we just
> trust the maintainers instead of putting restrictions that in the end are only
> increasing complexity for no benefit?
>

[...]

Yes, it's the interface. And the first sentence of 4.9 reads as:

"This file must be an executable makefile, and contains the package-specific
recipes for compiling the package and building binary package(s) from the
source."

This implies that the interface is to a larger extent documented in the make
documentation. Policy also specifies that a set of targets must be supported by
this makefile, but it doesn't say that these are the (only) supported arguments.
What about -j, for example? Not part of 4.9, but supported and documented by
make.

Adhering to a standard actually decreases complexity. What may seem "elegant" at
first makes it a lot harder for other people to step in. For example, the
VDR-solution IMHO doesn't decrease complexity, it merely hides it.

Best,
Michael

[Thomas Schmidt-2]

Am Mittwoch, den 28.10.2009, 19:05 -0500 schrieb Manoj Srivastava:
> > The solution we have right now is in some way "elegant", because you have
> > only to deal with a standard debian/rules and besides the different
> > shebang line there's nothing else to care about.
>
>         Actually, there is. My states makefile can no longer
> include debian/rules, because, you see, it is not a makefile. There are
> other ways that it does not look like a makefile, walk like a makefile,
> or quack like one.

I do not see how the shebang-line of debian/rules affects the ability to
include debian/rules from other Makefiles, as far as i can see, the the
shebang-line would be completely ignored in this case. (Which would
result in behaving like it does in the normal Debian build process.)


Regards,
Thomas

--
Thomas Schmidt, Debian VDR Team
http://pkg-vdr-dvb.alioth.debian.org/

[Lucas Nussbaum]

On 28/10/09 at 19:05 -0500, Manoj Srivastava wrote:
>         Actually, there is. My states makefile can no longer
> include debian/rules, because, you see, it is not a makefile. There are
> other ways that it does not look like a makefile, walk like a makefile,
> or quack like one.

The debian/rules file in the vdr* packages _is_ a makefile. It just
doesn't have the shebang line required by policy.
I don't think there's anything that prevents you from including it.

Also, if SPECIAL_VDR_SUFFIX is not set,
/usr/share/vdr-dev/make-special-vdr.sh just calls make -f "$@", so I
don't see why calling, for example, "debian/rules -d clean" would not
work as expected.

It sounds like several people in this discussion are shooting at an easy
target without looking at the specific details.
--
| Lucas Nussbaum
| lucas@...   http://www.lucas-nussbaum.net/ |
| jabber: lucas@...             GPG: 1024D/023B3F4F |


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@...
with a subject of "unsubscribe". Trouble? Contact listmaster@...

[Tobias Grimm]

Michael Tautschnig wrote:

> Adhering to a standard actually decreases complexity. What may seem "elegant" at
> first makes it a lot harder for other people to step in. For example, the
> VDR-solution IMHO doesn't decrease complexity, it merely hides it.

Yes, it indeed hides some complexity. But it only hides the stuff that is
beyond the scope of the actual standard build process. It has no impact on
the maintainability of the packages.

Tobias


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@...
with a subject of "unsubscribe". Trouble? Contact listmaster@...

[Michael Tautschnig-4]

> Michael Tautschnig wrote:
>
> > Adhering to a standard actually decreases complexity. What may seem "elegant" at
> > first makes it a lot harder for other people to step in. For example, the
> > VDR-solution IMHO doesn't decrease complexity, it merely hides it.
>
> Yes, it indeed hides some complexity. But it only hides the stuff that is
> beyond the scope of the actual standard build process. It has no impact on
> the maintainability of the packages.
>

In an earlier post you mentioned a pbuilder build process: If that is what you
are using, why not go for pbuilder hooks? A hook named A<something> should be
fine, as far as I understood. It should get executed after unpacking the source,
but before starting the build. You could fiddle around with debian/rules in any
way you like and you would (a) not need the special shebang line, (b) not
contaminate the package _at all_ and (c) be happy :-)

HTH,
Michael

[Goswin von Brederlow-2]

Tobi <listaccount@...> writes:

> Fabian Greffrath wrote:
>
>> Why not so it the other way round, i.e. start two different scripts (or
>> the same script with different parameters) from a debian/rules Makefile
>> depending on the environment variable?
>
> Might be possible, but it would require major changes to debian/rules, but
> our goal is to keep debian/rules as simple as possible without any stuff
> you wouldn't expect in a Debian package.
>
> Before adding any ugly stuff to debian/rules I would rather let an
> external build script to the dirty work. That's just harder to integrate
> into our pbuilder build process.
>
> Tobias

Or verry little (although I probably get crucified for this):

mv debian/rules debian/rules-no-make
cat >debian/rules << EOF
#! /usr/bin/make -f

%:
        $(MAKE) debian/rules-no-make $@
EOF

But I would really go with the idea to include different makefile
sniplets depending on your variable directly in the Makefile. You can
even be conform and use DEB_BUILD_OPTIONS like many other packages do
to allow altered builds.

MfG
        Goswin


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@...
with a subject of "unsubscribe". Trouble? Contact listmaster@...