pkgsrc RC scripts

[Joerg Sonnenberger]

Hi all,
I am consider to make the RC script support in pkgsrc a bit more useful
out of the box with the following changes:

(1) Install pkgsrc RC scripts by default into /etc/rc.pkg.d, not
/etc/rc.d and adjust the default value for rc_directories accordingly.
This keeps the logical separation between base and pkgsrc RC scripts
and still allows /usr (or /usr/pkg) to be on a separate filesystem.

(2) Provide a default value of NO for the control variables. This
ensures that no stupid "Missing variable $foo" clutters up the boot
messages.

Comments?

Joerg

[Joerg Sonnenberger]

On Sat, Oct 03, 2009 at 11:48:11PM +0200, Joerg Sonnenberger wrote:
> I am consider to make the RC script support in pkgsrc a bit more useful
> out of the box with the following changes:

To avoid confusing, this is about NetBSD and PREFIX=/usr/pkg, other
cases for left for the reader :)

Joerg

[Adam Hoka]

On Sat, 3 Oct 2009 23:48:11 +0200
Joerg Sonnenberger <joerg@...> wrote:

> Hi all,
> I am consider to make the RC script support in pkgsrc a bit more useful
> out of the box with the following changes:
>
> (1) Install pkgsrc RC scripts by default into /etc/rc.pkg.d, not
> /etc/rc.d and adjust the default value for rc_directories accordingly.
> This keeps the logical separation between base and pkgsrc RC scripts
> and still allows /usr (or /usr/pkg) to be on a separate filesystem.

Please do, it's a very much wanted feature amongst NetBSD users.
It will also greatly simplify my desktop installer work.

> (2) Provide a default value of NO for the control variables. This
> ensures that no stupid "Missing variable $foo" clutters up the boot
> messages.

I'm wondering if that warning makes sense at all, shouldn't it just silently
assume NO?

[Steven Bellovin]

On Oct 3, 2009, at 5:48 PM, Joerg Sonnenberger wrote:

> Hi all,
> I am consider to make the RC script support in pkgsrc a bit more  
> useful
> out of the box with the following changes:
>
> (1) Install pkgsrc RC scripts by default into /etc/rc.pkg.d, not
> /etc/rc.d and adjust the default value for rc_directories accordingly.
> This keeps the logical separation between base and pkgsrc RC scripts
> and still allows /usr (or /usr/pkg) to be on a separate filesystem.
>
> (2) Provide a default value of NO for the control variables. This
> ensures that no stupid "Missing variable $foo" clutters up the boot
> messages.
>

Both parts strike me as excellent ideas.  However -- where will you  
put the default "NO" information?  What about /etc/defaults/rc.pkg.conf?

Thinking further -- perhaps we should have /etc/pkg, and under it  
rc.d, rc.conf, mk.conf, defaults/rc.conf, etc.


                --Steve Bellovin, http://www.cs.columbia.edu/~smb

[Joerg Sonnenberger]

On Sat, Oct 03, 2009 at 06:13:03PM -0400, Steven Bellovin wrote:
> Both parts strike me as excellent ideas.  However -- where will you
> put the default "NO" information?  What about
> /etc/defaults/rc.pkg.conf?

Just inside the script, e.g.
gdm=${gdm:=NO}
after sourcing /etc/rc.subr.

That part is useful on all platforms and independent of the prefix.

Joerg

[Hubert Feyrer-4]

On Sat, 3 Oct 2009, Joerg Sonnenberger wrote:
>> I am consider to make the RC script support in pkgsrc a bit more useful
>> out of the box with the following changes:
>
> To avoid confusing, this is about NetBSD and PREFIX=/usr/pkg, other
> cases for left for the reader :)

Sounds good. Please make sure that also works for binary packages.


  - Hubert

[Jens Rehsack-2]

Joerg Sonnenberger wrote:
> Hi all,
> I am consider to make the RC script support in pkgsrc a bit more useful
> out of the box with the following changes:
>
> (1) Install pkgsrc RC scripts by default into /etc/rc.pkg.d, not
> /etc/rc.d and adjust the default value for rc_directories accordingly.
> This keeps the logical separation between base and pkgsrc RC scripts
> and still allows /usr (or /usr/pkg) to be on a separate filesystem.

How about /usr/pkg/etc/rc.d (read: ${PREFIX}/etc/rc.d)?
So it's possible to logical assign the rc.d script to the installed pkg's
(if someone wants to use more than one PREFIX - e.g. for testing).

> (2) Provide a default value of NO for the control variables. This
> ensures that no stupid "Missing variable $foo" clutters up the boot
> messages.
>
> Comments?

An rc.subr package would be great and the scripts should look for it in
/etc/rc.subr and ${PREFIX}/etc/rc.subr - this increases the portability.

Jens

[Joerg Sonnenberger]

On Sun, Oct 04, 2009 at 10:02:06AM +0000, Jens Rehsack wrote:

> Joerg Sonnenberger wrote:
> > Hi all,
> > I am consider to make the RC script support in pkgsrc a bit more useful
> > out of the box with the following changes:
> >
> > (1) Install pkgsrc RC scripts by default into /etc/rc.pkg.d, not
> > /etc/rc.d and adjust the default value for rc_directories accordingly.
> > This keeps the logical separation between base and pkgsrc RC scripts
> > and still allows /usr (or /usr/pkg) to be on a separate filesystem.
>
> How about /usr/pkg/etc/rc.d (read: ${PREFIX}/etc/rc.d)?
> So it's possible to logical assign the rc.d script to the installed pkg's
> (if someone wants to use more than one PREFIX - e.g. for testing).

Read the end of the second sentence. It is not possible to have the
scripts on a separate filesystem from /etc/rc.d without having to
magically reprocess the rcorder output. That is somewhat nasty as the
order constraints of rcorder are intentionally weak.  The same restraint
would apply for other prefixes, but I don't see a sane default for that,
so they would keep either /etc/rc.d as current default or
${PKG_SYSCONFDIR}/rc.d, but I think the latter would make more sense.

>
> > (2) Provide a default value of NO for the control variables. This
> > ensures that no stupid "Missing variable $foo" clutters up the boot
> > messages.
> >
> > Comments?
>
> An rc.subr package would be great and the scripts should look for it in
> /etc/rc.subr and ${PREFIX}/etc/rc.subr - this increases the portability.

There is pkgtools/rc.subr. If there is interest, it is easy to auto-fix
the rc scripts to use PREFIX/etc/rc.subr and therefore do the right
thing.

Joerg

[Robert Elz]

    Date:        Sat, 3 Oct 2009 23:48:11 +0200
    From:        Joerg Sonnenberger <joerg@...>
    Message-ID:  <20091003214811.GA20410@...>

  | (1) Install pkgsrc RC scripts by default into /etc/rc.pkg.d, not
  | /etc/rc.d and adjust the default value for rc_directories accordingly.
  | This keeps the logical separation between base and pkgsrc RC scripts

That's not a desirable goal.  One directory of rc scripts makes sense,
multiple directories of rc scripts just leads to even more directories of
rc scripts, so as to "keep logical separation" - if pkgsrc has its own,
perhaps X needs one for its scripts, and certainly my local (non pkgsrc)
additions need yet another, and perhaps we should have the networking
startup kept separate from the rest of the system, and put its scripts
in a different directory - no, make that two different directories, so we
can separate out ipv4 and ipv6 startup ... and then we'd better have more
for appletalk, and netiso, and ...

Absurd?  Yes, of course it is, but it all starts from the decision to
add a second script directory.   One is enough, there's no rational
reason to want to "keep a logical separation" - we don't do that in other
places, if pkgsrc wants to add a new user, the user name goes in /etc/passwd
(and /etc/master.passwd of course), not in /etc/pkgsrc-passwd or something
(with corresponding additions to the rest of the system so it can deal with
two password files).   We could do that to "keep a logical separation"
between users that are part of the base system, and others added by pkgsrc
(and then create yet another password for file locally added users, like me)
but I don't see anyone suggesting that as the path to adopt (and a good thing
that they're not).  "Logical separation" for rc scripts is just as
meaningless.

If it looks like it simplifies things for users, that's mistaken, it just
means that (unless we proceed down the above absurd path) I now have a
decision to make when I install something outside pkgsrc (which many of us
do, occasionally), should I put my application's startup script in with the
base system scripts, or in with the pkgsrc scripts?  It is neither, but
it has to go somewhere.   As things are now, things are simple, there's
just a directory of startup scripts, it belongs to no-one, and everything
goes there.  Simple, and clean.

  | (2) Provide a default value of NO for the control variables. This
  | ensures that no stupid "Missing variable $foo" clutters up the boot
  | messages.

This half is entirely reasonable, but the implementation method you
suggested in a later message is the wrong way to go about it.   I know
you work (mostly) on pkgsrc, and so the natural thing to do is to fix
everything in pkgsrc, and only there, and if that was a constraint, then
your suggested fix method would work - but there's no reason to confine
the solution there.

Just fix the rc system to stop bitching about undefined control variables
and assume "NO" instead.   A trivial change to rc.subr and no need to
clutter all the scripts with meaningless extra noise just to make sure
the variable is always set.

After all, all the base system scripts have (or should have) their
control variables in /etc/defaults/rc.conf so the test can really only be for
add on scripts, which is mostly pkgsrc added scripts, if pkgsrc then
defeats that by always defining the variables, the test has no purpose
left, and may as well be rmeoved.

Further, as a solution this prevents pkgsrc from subverting the base
system guidelines for how rc scripts should be installed - if the base system
(be it NetBSD or something else) really wants people to have to explicitly
set the control variable (one way or the other) when a script is installed,
perhaps in order to show people how to enable the thing, pkgsrc shouldn't
really be subverting that.

kre

[Joerg Sonnenberger]

On Sun, Oct 04, 2009 at 07:56:52PM +0700, Robert Elz wrote:

>     Date:        Sat, 3 Oct 2009 23:48:11 +0200
>     From:        Joerg Sonnenberger <joerg@...>
>     Message-ID:  <20091003214811.GA20410@...>
>
>   | (1) Install pkgsrc RC scripts by default into /etc/rc.pkg.d, not
>   | /etc/rc.d and adjust the default value for rc_directories accordingly.
>   | This keeps the logical separation between base and pkgsrc RC scripts
>
> That's not a desirable goal.  One directory of rc scripts makes sense,
> multiple directories of rc scripts just leads to even more directories of
> rc scripts, so as to "keep logical separation" - if pkgsrc has its own,
> perhaps X needs one for its scripts, and certainly my local (non pkgsrc)
> additions need yet another, and perhaps we should have the networking
> startup kept separate from the rest of the system, and put its scripts
> in a different directory - no, make that two different directories, so we
> can separate out ipv4 and ipv6 startup ... and then we'd better have more
> for appletalk, and netiso, and ...

Let's ignore the use of absurdity as rhetoric device for a moment.
One of the often voiced complains about installing the scripts
automatically is that it is hard to distinguish between components from
base and from pkgsrc. In fact, using a separate directory for local
additions makes a lot of sense. The main point you seemed to have missed
is that mixing different management domains is not necessarily a good
idea. /etc/rc.d is normally managed by postinstall. The pkgsrc RC
scripts by pkgsrc. The local additions by yourself. A clean separation
makes the process easier. It also helps in the not too unlikely case
that things moves in and out of base. It doesn't answer the problem of
having multiple RC scripts with the same base functionality, but it is a
step towards being able to resolve that.

>   | (2) Provide a default value of NO for the control variables. This
>   | ensures that no stupid "Missing variable $foo" clutters up the boot
>   | messages.
>
> This half is entirely reasonable, but the implementation method you
> suggested in a later message is the wrong way to go about it.   I know
> you work (mostly) on pkgsrc, and so the natural thing to do is to fix
> everything in pkgsrc, and only there, and if that was a constraint, then
> your suggested fix method would work - but there's no reason to confine
> the solution there.
>
> Just fix the rc system to stop bitching about undefined control variables
> and assume "NO" instead.   A trivial change to rc.subr and no need to
> clutter all the scripts with meaningless extra noise just to make sure
> the variable is always set.

Changing rc.subr won't help on any system already released. The trivial
one line change to the RC scripts is self-contained.

Joerg

[Geert Hendrickx]

On Sat, Oct 03, 2009 at 11:48:11PM +0200, Joerg Sonnenberger wrote:
> Hi all,
> I am consider to make the RC script support in pkgsrc a bit more useful
> out of the box with the following changes:
>
> (1) Install pkgsrc RC scripts by default into /etc/rc.pkg.d, not
> /etc/rc.d and adjust the default value for rc_directories accordingly.
> This keeps the logical separation between base and pkgsrc RC scripts
> and still allows /usr (or /usr/pkg) to be on a separate filesystem.


Sounds like a good plan.  How will you handle duplicated rc.d scripts,
eg. /etc/rc.d/sshd vs. /etc/rc.pkg.d/sshd if sshd=yes in /etc/rc.conf?


        Geert


--
Geert Hendrickx  -=-  ghen@...  -=-  PGP: 0xC4BB9E9F
This e-mail was composed using 100% recycled spam messages!

[Paul Goyette]

On Sun, 4 Oct 2009, Geert Hendrickx wrote:

>> (1) Install pkgsrc RC scripts by default into /etc/rc.pkg.d, not
>> /etc/rc.d and adjust the default value for rc_directories accordingly.
>> This keeps the logical separation between base and pkgsrc RC scripts
>> and still allows /usr (or /usr/pkg) to be on a separate filesystem.
>
>
> Sounds like a good plan.  How will you handle duplicated rc.d scripts,
> eg. /etc/rc.d/sshd vs. /etc/rc.pkg.d/sshd if sshd=yes in /etc/rc.conf?

And how do you handle rc.d scripts that are only present after some
other script mounts the containing filesystem?  Do you keep track of all
the scripts that have already been executed, and re-run rcorder?


-------------------------------------------------------------------------
|   Paul Goyette   | PGP DSS Key fingerprint: |  E-mail addresses:      |
| Customer Service | FA29 0E3B 35AF E8AE 6651 |  paul at whooppee.com   |
| Network Engineer | 0786 F758 55DE 53BA 7731 | pgoyette at juniper.net |
| Kernel Developer |                          | pgoyette at netbsd.org  |
-------------------------------------------------------------------------

[Joerg Sonnenberger]

On Sun, Oct 04, 2009 at 03:43:24PM +0200, Geert Hendrickx wrote:

> On Sat, Oct 03, 2009 at 11:48:11PM +0200, Joerg Sonnenberger wrote:
> > Hi all,
> > I am consider to make the RC script support in pkgsrc a bit more useful
> > out of the box with the following changes:
> >
> > (1) Install pkgsrc RC scripts by default into /etc/rc.pkg.d, not
> > /etc/rc.d and adjust the default value for rc_directories accordingly.
> > This keeps the logical separation between base and pkgsrc RC scripts
> > and still allows /usr (or /usr/pkg) to be on a separate filesystem.
>
>
> Sounds like a good plan.  How will you handle duplicated rc.d scripts,
> eg. /etc/rc.d/sshd vs. /etc/rc.pkg.d/sshd if sshd=yes in /etc/rc.conf?

ATM, it is not handled. I think the best approach for this is to add a
simple configuration file for rcorder to select one provider of a given
script. E.g. something like

/etc/rc.pkg.d/dhcpcd
/etc/rc.d/*

with first match wins. So the example would take the dhcpcd script from
pkgsrc and otherwise prefer the base scripts.

Joerg

[Joerg Sonnenberger]

On Sun, Oct 04, 2009 at 06:56:10AM -0700, Paul Goyette wrote:

> On Sun, 4 Oct 2009, Geert Hendrickx wrote:
>
> >>(1) Install pkgsrc RC scripts by default into /etc/rc.pkg.d, not
> >>/etc/rc.d and adjust the default value for rc_directories accordingly.
> >>This keeps the logical separation between base and pkgsrc RC scripts
> >>and still allows /usr (or /usr/pkg) to be on a separate filesystem.
> >
> >
> >Sounds like a good plan.  How will you handle duplicated rc.d scripts,
> >eg. /etc/rc.d/sshd vs. /etc/rc.pkg.d/sshd if sshd=yes in /etc/rc.conf?
>
> And how do you handle rc.d scripts that are only present after some
> other script mounts the containing filesystem?  Do you keep track of
> all the scripts that have already been executed, and re-run rcorder?

That's why you put all the script in /etc.

Joerg

[David Brownlee]

2009/10/3 Joerg Sonnenberger <joerg@...>

Hi all,
I am consider to make the RC script support in pkgsrc a bit more useful
out of the box with the following changes:

(1) Install pkgsrc RC scripts by default into /etc/rc.pkg.d, not
/etc/rc.d and adjust the default value for rc_directories accordingly.
This keeps the logical separation between base and pkgsrc RC scripts
and still allows /usr (or /usr/pkg) to be on a separate filesystem.

As long as there is an easy way to override this seems sane.
 

(2) Provide a default value of NO for the control variables. This
ensures that no stupid "Missing variable $foo" clutters up the boot
messages.

I'd prefer it if all of the pkgsrc rc.d scripts set some variable which rc could test to determine whether to warn about a missing variable (I'm happy for the warnings to default off, but it would allow toggling them back on for those who like that by setting a single additional var)

[Lloyd Parkes]

Robert Elz wrote:
>   | (1) Install pkgsrc RC scripts by default into /etc/rc.pkg.d, not
>   | /etc/rc.d and adjust the default value for rc_directories accordingly.
>   | This keeps the logical separation between base and pkgsrc RC scripts
>
> That's not a desirable goal.  One directory of rc scripts makes sense,
> multiple directories of rc scripts just leads to even more directories of
> rc scripts, so as to "keep logical separation" -

I agree completely. If I tried telling some of my colleagues that there were now
two directories that rc scripts might be in, and there is no way to tell which
scripts belong in which directory, they would tell me exactly where to go. Not
only do you need a complete knowledge of either pkgsrc packages or the base rc
scripts to know where the quagga startup script is installed, but for some
software such as web servers you have to know whether it's the NetBSD httpd, or
one of the many pkgsrc httpds. Many people on these mailing lists won't have a
problem with this, but you shouldn't have to be a genius to administer NetBSD.

You also run the risk of people thinking that the directories of rc scripts are
run at different points in time just like SVR4 rc directories.

The first thing I thought when the /etc/rc.d directory was created was "Good.
Now we have the rc order calculated at boot time and everything is in one place.
None of this crazy mess like SVR4 and Linux." Nice and simple.

The pkgsrc system manages adding and removing shells, users and groups well. It
can handle rc scripts in /etc/rc.d just as well.

>   | (2) Provide a default value of NO for the control variables. This
>   | ensures that no stupid "Missing variable $foo" clutters up the boot
>   | messages.
>
> This half is entirely reasonable,

I agree. If rc.subr is simply updated to default to NO, then all those annoying
messages go away and everyone is happy.

Cheers,
Lloyd

[Steven Drake-3]

On Sat, 3 Oct 2009, Joerg Sonnenberger wrote:

> (1) Install pkgsrc RC scripts by default into /etc/rc.pkg.d, not
> /etc/rc.d and adjust the default value for rc_directories accordingly.
> This keeps the logical separation between base and pkgsrc RC scripts
> and still allows /usr (or /usr/pkg) to be on a separate filesystem.

Sounds like your talking about changeing the default value
of RCD_SCRIPTS_DIR.

Personaly I would like config variables that are used by the
+INSTALL +DEINSTALL script set in pkg_install.conf and passed to the scripts
when thay get run. I know this would mean writing extra code in pkg_install
but it would mean that the +INSTALL +DEINSTALL behaviour is defined when
a binary package is installed not built.  

--
Steven

[Steven Drake-3]

On Sun, 4 Oct 2009, Joerg Sonnenberger wrote:

> There is pkgtools/rc.subr. If there is interest, it is easy to auto-fix
> the rc scripts to use PREFIX/etc/rc.subr and therefore do the right
> thing.
>

How about a RC_SUBR_DIR variable!  Also does changing the loction of rc.subr
mean that the location of rc.conf & rc.conf.d is changed as well.

--
Steven

[Alan Barrett]

On Sun, 04 Oct 2009, Robert Elz wrote:
>   | (2) Provide a default value of NO for the control variables. This
>   | ensures that no stupid "Missing variable $foo" clutters up the
>   | boot messages.
  [...]
> After all, all the base system scripts have (or should have) their
> control variables in /etc/defaults/rc.conf so the test can really only
> be for add on scripts, which is mostly pkgsrc added scripts, if pkgsrc
> then defeats that by always defining the variables, the test has no
> purpose left, and may as well be rmeoved.

That test appeared in rc.subr revision 1.1 dated 1997-08-29.
/etc/default/rc.conf apepared on 2000-08-22, and moved to
/etc/defaults/rc.conf on 2000-10-01.

The test might have made sense for the base system back in the time
before /etc/defaults/rc.conf existed, when variables had to be set
excplicitly in /etc/rc.conf, but I don't think it makes sense for
the base system today, and the warnings for pkgsrc scripts annoyed
me enough that I removed the warning from my own rc.subr a long
time ago.  I suggest that the warning

        warn "\$${1} is not set properly - see ${rcvar_manpage}."

should be removed from the checkyesno function in the base system's
/etc.rc.subr.

Whether or not pkgsrc-supplied scripts leave their variables
undefined by default is a separate issue from whether or not rc.subr
warns about undefined variables.  I don't care either way about
this separate issue.

--apb (Alan Barrett)

[Alan Barrett]

On Sun, 04 Oct 2009, Joerg Sonnenberger wrote:
> One of the often voiced complains about installing the scripts
> automatically is that it is hard to distinguish between components
> from base and from pkgsrc.

A comment at the top of the pkgsrc-supplied script could help.

> In fact, using a separate directory for local additions makes a
> lot of sense. The main point you seemed to have missed is that
> mixing different management domains is not necessarily a good
> idea. /etc/rc.d is normally managed by postinstall. The pkgsrc RC
> scripts by pkgsrc. The local additions by yourself. A clean separation
> makes the process easier.

A clean separation makes the process easier in some ways, and more
difficult in other ways; it also raises questions about how many
separate directories to have.  My preference is to leave them all
in one directory.

> >   | ensures that no stupid "Missing variable $foo" clutters up the boot
> >   | messages.
>
> Changing rc.subr won't help on any system already released. The trivial
> one line change to the RC scripts is self-contained.

That argument works both ways:  Changing pkgsrc RC scripts won't
help systems that have already installed old packages.  The trivial
change to rc.subr is self-contained and benefits both old and new
packages.

--apb (Alan Barrett)