common, FHS-compliant, default document root for the various web servers

[Stefano Zacchiroli]

[ adding -policy to Cc: ]

On Wed, Nov 04, 2009 at 04:08:02PM +0100, Holger Levsen wrote:
> Uhm, why postpone this so long? I'd hope we could find a consensus quite soon.
> Then, we might not be able to fix _all_ web apps until squeeze, but at least
> tthose few with dir-or-file-in-var-www :-)

I see it a tad more complicate than you, let's hope its me
overestimating the task :-)

- the agreement actually should not come among web app maintainers, but
  rather among web *server* maintainers: they should agree over a
  specific dir and change the default configuration of the web server so
  that that dir is the document root (for the default vhost, for web
  servers supporting vhosts)

  * possibly, migrating to that would require offering migration paths
    to package users

- then you might start migrating web apps packages so that they install
  (static) stuff under that dir, preserving the per-package path as
  detailed in the webapps-common policy

- then, the rule should go into policy (possibly under §9.1.1, has an
  exception to FHS, not sure about the section though) and that can't
  happen before due to the usual practice-should-predate-policy

If it were me to try to achieve this, I would go for a DEP to keep track
of consensus, ... but no, I'm not willing to drive this, at least not
now :-)

Cheers.

--
Stefano Zacchiroli -o- PhD in Computer Science \ PostDoc @ Univ. Paris 7
zack@{upsilon.cc,pps.jussieu.fr,debian.org} -<>- http://upsilon.cc/zack/
Dietro un grande uomo c'è ..|  .  |. Et ne m'en veux pas si je te tutoie
sempre uno zaino ...........| ..: |.... Je dis tu à tous ceux que j'aime

[Gunnar Wolf]

Stefano Zacchiroli dijo [Wed, Nov 04, 2009 at 05:50:01PM +0100]:

> > Uhm, why postpone this so long? I'd hope we could find a consensus quite soon.
> > Then, we might not be able to fix _all_ web apps until squeeze, but at least
> > tthose few with dir-or-file-in-var-www :-)
>
> I see it a tad more complicate than you, let's hope its me
> overestimating the task :-)
>
> - the agreement actually should not come among web app maintainers, but
>   rather among web *server* maintainers: they should agree over a
>   specific dir and change the default configuration of the web server so
>   that that dir is the document root (for the default vhost, for web
>   servers supporting vhosts)
>
>   * possibly, migrating to that would require offering migration paths
>     to package users

That's easy, as there is fewer of us than web app maintainers. And it
is a first step. We might even have a transitional symlink making
/var/www point to /var/lib/www or whatnot?

> - then you might start migrating web apps packages so that they install
>   (static) stuff under that dir, preserving the per-package path as
>   detailed in the webapps-common policy
>
> - then, the rule should go into policy (possibly under §9.1.1, has an
>   exception to FHS, not sure about the section though) and that can't
>   happen before due to the usual practice-should-predate-policy
>
> If it were me to try to achieve this, I would go for a DEP to keep track
> of consensus, ... but no, I'm not willing to drive this, at least not
> now :-)

It is a bit ambitious to have it _completely_ done by Squeeze. But we
can start pushing the right way. And anyway, I do not feel it is
asking too much. Yes, we cannot just go from using /var/www to
having its existence violate policy and making insta-RC, but as soon
as the (major at least) webserves change their defaults, we can start
filing wishlist bugs, pointing maintainers to this being a work in
progress and expecting it to be strictly enforced by policy for
squeeze+1.

--
Gunnar Wolf • gwolf@... • ([phone] / 1451-2244


--
To UNSUBSCRIBE, email to debian-policy-REQUEST@...
with a subject of "unsubscribe". Trouble? Contact listmaster@...

[Neil McGovern]

On Wed, Nov 04, 2009 at 05:50:01PM +0100, Stefano Zacchiroli wrote:
> - the agreement actually should not come among web app maintainers, but
>   rather among web *server* maintainers: they should agree over a
>   specific dir and change the default configuration of the web server so
>   that that dir is the document root (for the default vhost, for web
>   servers supporting vhosts)
>

For reference, when the initial RFH on a webapps policy was sent out,
all httpd maintainers were included.

> If it were me to try to achieve this, I would go for a DEP to keep track
> of consensus, ... but no, I'm not willing to drive this, at least not
> now :-)
>

I personally don't like the DEPs, but would be happy to accept reasoned
patches into the webapps policy submission.

Neil
--
<enrico> What is a sane place to look for washing machines around Manchester?
<mhy> enrico: the canals :-)


--
To UNSUBSCRIBE, email to debian-policy-REQUEST@...
with a subject of "unsubscribe". Trouble? Contact listmaster@...

[Giacomo A. Catenazzi]

Stefano Zacchiroli wrote:

> [ adding -policy to Cc: ]
>
> On Wed, Nov 04, 2009 at 04:08:02PM +0100, Holger Levsen wrote:
>> Uhm, why postpone this so long? I'd hope we could find a consensus quite soon.
>> Then, we might not be able to fix _all_ web apps until squeeze, but at least
>> tthose few with dir-or-file-in-var-www :-)
>
> I see it a tad more complicate than you, let's hope its me
> overestimating the task :-)
>
> - the agreement actually should not come among web app maintainers, but
>   rather among web *server* maintainers: they should agree over a
>   specific dir and change the default configuration of the web server so
>   that that dir is the document root (for the default vhost, for web
>   servers supporting vhosts)
>
>   * possibly, migrating to that would require offering migration paths
>     to package users
>
> - then you might start migrating web apps packages so that they install
>   (static) stuff under that dir, preserving the per-package path as
>   detailed in the webapps-common policy
>
> - then, the rule should go into policy (possibly under §9.1.1, has an
>   exception to FHS, not sure about the section though) and that can't
>   happen before due to the usual practice-should-predate-policy

Personally I would like to have a competently different approach:

- web server ask where to put the root (probably proposing default
   a /srv/www location).  But not further assumption about the location.
   Admins, per FHS, could choose other paths.

   This could be done by a new update-http-root application.
   (and ev. could handle multiple vhost).
   And possibly allowing no public location (thus forcing local only
   connections): we tend to forget about this, but IMHO more and more
   desktop computers are installed with webserver because of local
   convenience. Thus we really need to securely support this common
   cases.


- No webapp are installed "live" by default:

   We have too much crap web application, and some/most of our users
   don't realize that they are installing a public accessible crap.
   [the desktop users]

   Thus IMHO we need a "update-webappl" utility, which would
   list, ask and ev. install the just installed webapplication.

   This is not so far as the installation of apache modules, which
   ask for which apache (apache/apache2/apache2-ssl/...) to enable
   modules. We just list the possible web root.

   Naturally admins can skip this point (e.g. not allowing debian
   to handle webappl, but doing manually).

   Probably a webserver-specific support script will handle the
   generation of symlink (default) or via configuration (webserver
   specific) of the /usr/lib/cgi or /usr/share/* dir.


In short:

- no hardcoded default root location (only a default value for a
   real user question)
- not installing by default (without asking) web apps.


ciao
        cate

PS: first mail in debian-policy, so maybe I missed the point of
the discussion (which take place in the other mailing lists)


--
To UNSUBSCRIBE, email to debian-policy-REQUEST@...
with a subject of "unsubscribe". Trouble? Contact listmaster@...

[Stefano Zacchiroli]

On Thu, Nov 05, 2009 at 10:21:48AM +0100, Jan Hauke Rahm wrote:

> Okay, I understand. Now, I see two ways actually to solve this.
>
> 1. If we have a generic location for packages to drop their
>    html/php/whatever files, like /var/lib/www, all web servers can keep
>    their DocRoot as /var/www and provide an alias for /var/lib/www, for
>    instance /debian. That way webapp packages don't even have to care
>    about the web server in charge, we don't need a webapps-common, we
>    just need all web servers to provide that alias (or if they can't,
>    they have to symlink it). Every webapp would be available at
>    localhost/debian/webapp. Since it's either a symlink or a conffile
>    that makes this /debian alias, it can be changed by the local
>    sysadmin without any risks and without touching our packages data.
> 2. If however all packages put their files in different locations, we
>    need your suggested solution with scripts for each webserver. A
>    package like webapps-common could run
>    foreach server in /usr/share/webapps-common/webservers/*; do
>       ./$server webappPath webappName
>    done
>    and the web servers provide aliases/symlinks accordingly.
>
> So, what's wrong with (1) that we don't go this simple path?

Fair question.  As I can't came up with an answer, I guess that (1) is
indeed a better solution, due Occam's razor.

I only have a couple of remarks on some details:

- According to FHS, /var/lib/ is for "variable state information". As we
  are talking about static HTML content, which only change upon package
  upgrade, I believe it would not be appropriate. A better place would
  probably be /usr/share/www/PACKAGE/ (maybe some FHS guru can give us
  some insights here ...)

- Regarding the URL that would be mapped to that dir, I don't
  particularly like /debian/ (even though I've advanced it). However the
  alternative solutions I can come up with (e.g. /packages/) are
  actually uglier. So I guess /debian/ can stay. Some of the -webapps
  people can probably come up with wiser suggestions ...

Cheers.

--
Stefano Zacchiroli -o- PhD in Computer Science \ PostDoc @ Univ. Paris 7
zack@{upsilon.cc,pps.jussieu.fr,debian.org} -<>- http://upsilon.cc/zack/
Dietro un grande uomo c'è ..|  .  |. Et ne m'en veux pas si je te tutoie
sempre uno zaino ...........| ..: |.... Je dis tu à tous ceux que j'aime

[Jan Hauke Rahm-2]

On Thu, Nov 05, 2009 at 03:05:07PM +0100, Giacomo A. Catenazzi wrote:
> Personally I would like to have a competently different approach:
>
> - web server ask where to put the root (probably proposing default
>   a /srv/www location).  But not further assumption about the
>   location.
>   Admins, per FHS, could choose other paths.

This is done in a few webapps right now. But we (as in maintainers)
still need to provide a sane (FHS and policy comliant) default for
non-interactive installations and even for users who have no clue about
what they're doing and simply press enter.

>   This could be done by a new update-http-root application.
>   (and ev. could handle multiple vhost).
>   And possibly allowing no public location (thus forcing local only
>   connections): we tend to forget about this, but IMHO more and more
>   desktop computers are installed with webserver because of local
>   convenience. Thus we really need to securely support this common
>   cases.

Well, if we go for a solution that implements vhosts (in case of apache
and other web servers that even know about vhosts) we could include some
command to bind the vhost to localhost. I'd be okay with it. If,
however, we decide to have one debian specific DocRoot which all webapps
whould use, then we don't have vhosts for all packages but one generic
one. We can still have this per default bound to localhost but if one
webapp is public, all are.

> - No webapp are installed "live" by default:
>
>   We have too much crap web application, and some/most of our users
>   don't realize that they are installing a public accessible crap.
>   [the desktop users]

Isn't that what the topic was about? *confused*

>   Thus IMHO we need a "update-webappl" utility, which would
>   list, ask and ev. install the just installed webapplication.
>
>   This is not so far as the installation of apache modules, which
>   ask for which apache (apache/apache2/apache2-ssl/...) to enable
>   modules. We just list the possible web root.
>
>   Naturally admins can skip this point (e.g. not allowing debian
>   to handle webappl, but doing manually).
>
>   Probably a webserver-specific support script will handle the
>   generation of symlink (default) or via configuration (webserver
>   specific) of the /usr/lib/cgi or /usr/share/* dir.

What exactly should these scripts do then?

> In short:
>
> - no hardcoded default root location (only a default value for a
>   real user question)

As said above, we need a DocRoot that is FHS and policy compliant. If I
understand you correctly you want a DocRoot for each and every webapp
instead of one global DocRoot?

> - not installing by default (without asking) web apps.

You mean a debconf template that says: "I know you typed 'aptitude
install webapp' but we're smarter than you are and thus not doing it
unless you really want to and type 'yes' now." ? Remember Debian is not
only about a secure system but also about a working system. Don't make
life harder than necessary. I'd agree to binding to localhost, though.

Hauke

[Harald Braumann]

On Thu, 5 Nov 2009 16:39:06 +0100
Stefano Zacchiroli <zack@...> wrote:

> - Regarding the URL that would be mapped to that dir, I don't
>   particularly like /debian/ (even though I've advanced it). However
> the alternative solutions I can come up with (e.g. /packages/) are
>   actually uglier. So I guess /debian/ can stay. Some of the -webapps
>   people can probably come up with wiser suggestions ...

/debian/ seems to be the de facto standard for Debian archives. So I
guess it wouldn't be such a good idea to use it for something else
(sorry, can't really come up with a better name myself).

Cheers,
harry

[The Fungi]

On Fri, Nov 06, 2009 at 01:11:47PM +0100, Harald Braumann wrote:
> /debian/ seems to be the de facto standard for Debian archives. So I
> guess it wouldn't be such a good idea to use it for something else
> (sorry, can't really come up with a better name myself).

Something short, generic and distro-neutral like /app/ would be my
personal preference if I were developing a standard for my servers.
Unfortunately, going that direction as also increases the chances of
remapping a path the admin was already using...
--
{ IRL(Jeremy_Stanley); PGP(9E8DFF2E4F5995F8FEADDC5829ABF7441FB84657);
SMTP(fungi@...); IRC(fungi@...#ccl); ICQ(114362511);
AIM(dreadazathoth); YAHOO(crawlingchaoslabs); FINGER(fungi@...);
MUD(fungi@...:6669); WWW(http://fungi.yuggoth.org/); }


--
To UNSUBSCRIBE, email to debian-policy-REQUEST@...
with a subject of "unsubscribe". Trouble? Contact listmaster@...

[Jan Hauke Rahm-2]

On Thu, Nov 05, 2009 at 04:39:06PM +0100, Stefano Zacchiroli wrote:

> On Thu, Nov 05, 2009 at 10:21:48AM +0100, Jan Hauke Rahm wrote:
> > Okay, I understand. Now, I see two ways actually to solve this.
> >
> > 1. If we have a generic location for packages to drop their
> >    html/php/whatever files, like /var/lib/www, all web servers can keep
> >    their DocRoot as /var/www and provide an alias for /var/lib/www, for
> >    instance /debian. That way webapp packages don't even have to care
> >    about the web server in charge, we don't need a webapps-common, we
> >    just need all web servers to provide that alias (or if they can't,
> >    they have to symlink it). Every webapp would be available at
> >    localhost/debian/webapp. Since it's either a symlink or a conffile
> >    that makes this /debian alias, it can be changed by the local
> >    sysadmin without any risks and without touching our packages data.
> > 2. If however all packages put their files in different locations, we
> >    need your suggested solution with scripts for each webserver. A
> >    package like webapps-common could run
> >    foreach server in /usr/share/webapps-common/webservers/*; do
> >       ./$server webappPath webappName
> >    done
> >    and the web servers provide aliases/symlinks accordingly.
> >
> > So, what's wrong with (1) that we don't go this simple path?
>
> Fair question.  As I can't came up with an answer, I guess that (1) is
> indeed a better solution, due Occam's razor.
>
> I only have a couple of remarks on some details:
>
> - According to FHS, /var/lib/ is for "variable state information". As we
>   are talking about static HTML content, which only change upon package
>   upgrade, I believe it would not be appropriate. A better place would
>   probably be /usr/share/www/PACKAGE/ (maybe some FHS guru can give us
>   some insights here ...)

Full ack, and I even like /usr/share/www. It's easy to understand and
pretty unprobable that we'd have a package called www in the archive
some day needing this location.

> - Regarding the URL that would be mapped to that dir, I don't
>   particularly like /debian/ (even though I've advanced it). However the
>   alternative solutions I can come up with (e.g. /packages/) are
>   actually uglier. So I guess /debian/ can stay. Some of the -webapps
>   people can probably come up with wiser suggestions ...

Manoj suggested '/vendor-apps' and I like that. It says what it should
say and doesn't steal any probable path.

I still see a problem with the upgrade path for existing installations.
I might be wrong but I think the most difficult cases are very custom
setups with lots of changes by the local admin. I'm thinking of e.g.
webmail.domain.tld being a virtual host with DocRoot
/usr/share/squirrelmail. If the files there move to
/usr/share/www/squirrelmail we break a lot of setups. So, what about
shipping a symlink from the old location to the new one as a migration
path? This doesn't solve the very default (e.g. users accessing
squirrelmail via localhost/squirrelmail) but that is so easily solved
via alias directive or symlink that I suppose a NEWS.Debian entry would
fit best here.
What do you say?

Now, I'm willing to run this, i.e. file bugs against web servers, wait
for them to be fixed, then file bugs against web applications (if
needed, I'm right now looking into a way to make a lintian check for it,
e.g. package-with-section-web-but-no-files-in-canonical-docroot). But I
don't feel like we're having a clear consensus here, do we?

Attached is a suggestion for MBF as well as a dd-list of all relevant
web servers (if I did my job right).

Now critizise!
Hauke

Subject: Install alias '/vendor-apps' to /usr/share/www as canonical DocRoot for packaged web applications
Package: apache2
Severity: wishlist
User: jhr@...
Usertags: canonicaldocroot-server

Dear maintainer,

when discussing the appropriate location for web application files it
was suggested to have one canonical location for those in the file
system that all these web applications should use. All web servers in
the archive should support this decision by providing a default way to
access this location.

It is consensus in debian-devel@... that this location shall be
    /usr/share/www
and packages are supposed to drop their files in subdirectories, i.e.
    /usr/share/www/package

(If a web application additionally needs files that are auto-generated
and/or to be touched by local admins or users, the application itself
should cope with that and provide a proper location for that, i.e.
probably '/var/lib/package/html' or similar.)

As a default this location shall be accessable via
    http://localhost/vendor-apps/package

If this web server has an alias functionality it is best to use that as
conffiles are the easiest way for admins to adjust the setup to their
needs. If, however, such a functionality is not given, a symlink
    /var/www/vendor-apps -> /usr/share/www
may solve the problem.

If this web servers *needs* the target of a symlink or alias directive
to exist, it may install it as an empty directory. This should be
avoided if possible (a 404 response is no good reason).

Additionally to this new location it was brought up that web
applications should a) be able to run with default settings, but also b)
be somehow secure. Therefor this new location shall -- if anyhow
possible -- be bound to localhost per default.

This bug is filed against every httpd in the archive which is the first
step to solve the current situation. Please provide this new setup soon,
so we can start filing bugs on web applications to move their files.

Thanks!

Krzysztof Krzyżaniak (eloy) <eloy@...>
   lighttpd (U)

Jari Aalto <jari.aalto@...>
   micro-httpd

Henrik Andreasson <debian@...>
   caudium

Adam Conrad <adconrad@...>
   apache2 (U)

Debian Apache Maintainers <debian-apache@...>
   apache2

Debian Erlang Packagers <pkg-erlang-devel@...>
   yaws

Debian lighttpd maintainers <pkg-lighttpd-maintainers@...>
   lighttpd

Stefan Fritsch <sf@...>
   apache2 (U)

Sergei Golovan <sgolovan@...>
   yaws (U)

Francois-Denis Gonthier <neumann@...>
   boa

Debian QA Group <packages@...>
   thttpd

Steinar H. Gunderson <sesse@...>
   apache2 (U)

Pierre Habouzit <madcoder@...>
   lighttpd (U)

Tollef Fog Heen <tfheen@...>
   apache2 (U)

Francesco Paolo Lovergine <frankie@...>
   aolserver4

Torsten Marek <shlomme@...>
   lighttpd (U)

Thom May <thom@...>
   apache2 (U)

Jose M. Moya <josem@...>
   mathopd

Ryan Niebur <ryanryan52@...>
   dhttpd

Mattias Nordstrom <mnordstr@...>
   bozohttpd

Leonel Nunez <leonel@...>
   cherokee (U)

Alvaro Lopez Ortega <alvaro@...>
   cherokee (U)

Kari Pahula <kaol@...>
   tntnet

Gerrit Pape <pape@...>
   fnord

Jose Parrella <bureado@...>
   nginx

Franz Pletz <fpletz@...>
   lighttpd (U)

Iñaki Rodriguez <irodriguez@...>
   webfs

Peter Samuelson <peter@...>
   apache2 (U)

Thorsten Schmale <thorsten@...>
   monkey

Marvin Stark <marv@...>
   mini-httpd

Salvo 'LtWorf' Tomaselli <tiposchi@...>
   weborf

Fabio Tranchitella <kobold@...>
   nginx (U)

Gunnar Wolf <gwolf@...>
   cherokee

[Charles Plessy-12]

Le Sat, Nov 07, 2009 at 03:23:22PM +0100, Jan Hauke Rahm a écrit :

>
> I still see a problem with the upgrade path for existing installations.
> I might be wrong but I think the most difficult cases are very custom
> setups with lots of changes by the local admin. I'm thinking of e.g.
> webmail.domain.tld being a virtual host with DocRoot
> /usr/share/squirrelmail. If the files there move to
> /usr/share/www/squirrelmail we break a lot of setups. So, what about
> shipping a symlink from the old location to the new one as a migration
> path? This doesn't solve the very default (e.g. users accessing
> squirrelmail via localhost/squirrelmail) but that is so easily solved
> via alias directive or symlink that I suppose a NEWS.Debian entry would
> fit best here.
> What do you say?

Dear Hauke,

As a maintainer of a web application, I share your worries. I never had any
user request to make it work out of the box with alternative web servers, so I
guess that my users have nothing to gain and everything to lose in a change. I
strongly suggest that the transition is experimented on a few volunteer
packages before increasing the workload of many persons – users and developers.

For new packages, grouping everything in /usr/share/www sounds like a good
idea.  The alias name, « vendor », I find a bit disturbing because we do not
sell anything. But picking the name will be the priviledge of the Do-o-crat who
will lead the transition, I presume.

Still, having /usr/share/www as a document root does not prevent complex
packages to be fragmented between /usr/share, /usr/lib/cgi-bin/, /var/lib/,
/var/tmp, /var/run and /etc. Maybe you can double-check how many web servers
are able to cope with that before starting to invest a lot of time. Otherwise,
since shipping configuration files in /etc/webserver/conf.d will still be
necessary for these packages to work, there will little benefit in moving files
to /usr/share/www.

Anyway, thank you very much for your initiative. Exploring new directions is good !

Have a nice day,

--
Charles Plessy
Tsurumi, Kanagawa, Japan


--
To UNSUBSCRIBE, email to debian-policy-REQUEST@...
with a subject of "unsubscribe". Trouble? Contact listmaster@...

[Jan Hauke Rahm-2]

Thanks for your response, Charles!

On Sun, Nov 08, 2009 at 12:09:28AM +0900, Charles Plessy wrote:
> As a maintainer of a web application, I share your worries. I never had any
> user request to make it work out of the box with alternative web servers, so I
> guess that my users have nothing to gain and everything to lose in a change. I
> strongly suggest that the transition is experimented on a few volunteer
> packages before increasing the workload of many persons – users and developers.

I think this is a bit black or white. Users (ar rather admins) gain the
possibility to easily guess where a package is to be found. No looking
for /usr/share/package/html or something, just assume it's
/usr/share/www/package. And they don't lose everything, they just need
to read the chapter (yet to be written, of course) in the release notes
about web applications being moved; additionally NEWS.Debian should be
provided by any relevant package and so on.

Also, I said, a proper migration path has still to be figured out. The
question for now is: do we want this change at all and -- if so -- shall
I file bugs against the web servers. I'd suggest severity wishlist for
the start.

> For new packages, grouping everything in /usr/share/www sounds like a good
> idea.  The alias name, « vendor », I find a bit disturbing because we do not
> sell anything. But picking the name will be the priviledge of the Do-o-crat who
> will lead the transition, I presume.

Well, I find 'vendor' a bit confusing as well but I'm not a native
english speaker and I don't have better ideas. Shoot if you have any.
Something like 'debian', 'webapps', 'applications' seem way to generic
for this.

> Still, having /usr/share/www as a document root does not prevent complex
> packages to be fragmented between /usr/share, /usr/lib/cgi-bin/, /var/lib/,
> /var/tmp, /var/run and /etc. Maybe you can double-check how many web servers
> are able to cope with that before starting to invest a lot of time. Otherwise,
> since shipping configuration files in /etc/webserver/conf.d will still be
> necessary for these packages to work, there will little benefit in moving files
> to /usr/share/www.

And there isn't even a way we could (or want to) solve this.
Applications of many sorts have to deal with FHS, webapps are nothing
special in that matter. We can't allow an admin to fiddle aroung with
files in /usr/share, nor can we just pump all webapps into /var as this
will break (or at least bloat) many backup strategies and cause other
problems.

Until now I thought simple symlinks work with every webserver and thus
didn't see a problem for that. It's the application that sometimes needs
patching to deal with its being splattered around. Am I wrong here?

Hauke

[Henrik Andreasson]

On Sat, 7 Nov 2009, Jan Hauke Rahm wrote:

Caudium can and will adjust to any standard that the community agrees
upon and it can handle different directories without problem.

I really dont have that much input for how this should be done but leaving
it as it is now is worse.

> Thanks for your response, Charles!
>
> On Sun, Nov 08, 2009 at 12:09:28AM +0900, Charles Plessy wrote:
>> As a maintainer of a web application, I share your worries. I never had any
>> user request to make it work out of the box with alternative web servers, so I
>> guess that my users have nothing to gain and everything to lose in a change. I
>> strongly suggest that the transition is experimented on a few volunteer
>> packages before increasing the workload of many persons ? users and developers.
>
> I think this is a bit black or white. Users (ar rather admins) gain the
> possibility to easily guess where a package is to be found. No looking
> for /usr/share/package/html or something, just assume it's
> /usr/share/www/package. And they don't lose everything, they just need
> to read the chapter (yet to be written, of course) in the release notes
> about web applications being moved; additionally NEWS.Debian should be
> provided by any relevant package and so on.
>
> Also, I said, a proper migration path has still to be figured out. The
> question for now is: do we want this change at all and -- if so -- shall
> I file bugs against the web servers. I'd suggest severity wishlist for
> the start.
>
>> For new packages, grouping everything in /usr/share/www sounds like a good
>> idea.  The alias name, « vendor », I find a bit disturbing because we do not
>> sell anything. But picking the name will be the priviledge of the Do-o-crat who
>> will lead the transition, I presume.
>
> Well, I find 'vendor' a bit confusing as well but I'm not a native
> english speaker and I don't have better ideas. Shoot if you have any.
> Something like 'debian', 'webapps', 'applications' seem way to generic
> for this.
>
>> Still, having /usr/share/www as a document root does not prevent complex
>> packages to be fragmented between /usr/share, /usr/lib/cgi-bin/, /var/lib/,
>> /var/tmp, /var/run and /etc. Maybe you can double-check how many web servers
>> are able to cope with that before starting to invest a lot of time. Otherwise,
>> since shipping configuration files in /etc/webserver/conf.d will still be
>> necessary for these packages to work, there will little benefit in moving files
>> to /usr/share/www.
>
> And there isn't even a way we could (or want to) solve this.
> Applications of many sorts have to deal with FHS, webapps are nothing
> special in that matter. We can't allow an admin to fiddle aroung with
> files in /usr/share, nor can we just pump all webapps into /var as this
> will break (or at least bloat) many backup strategies and cause other
> problems.
>
> Until now I thought simple symlinks work with every webserver and thus
> didn't see a problem for that. It's the application that sometimes needs
> patching to deal with its being splattered around. Am I wrong here?
>
> Hauke
>

[Stefano Zacchiroli]

On Sat, Nov 07, 2009 at 03:23:22PM +0100, Jan Hauke Rahm wrote:

> > > 1. If we have a generic location for packages to drop their
> > >    html/php/whatever files, like /var/lib/www, all web servers can keep
> > >    their DocRoot as /var/www and provide an alias for /var/lib/www, for
> > >    instance /debian. That way webapp packages don't even have to care
> > >    about the web server in charge, we don't need a webapps-common, we
> > >    just need all web servers to provide that alias (or if they can't,
> > >    they have to symlink it). Every webapp would be available at
> > >    localhost/debian/webapp. Since it's either a symlink or a conffile
> > >    that makes this /debian alias, it can be changed by the local
> > >    sysadmin without any risks and without touching our packages data.
> > I only have a couple of remarks on some details:
> >
> > - According to FHS, /var/lib/ is for "variable state information". As we
> >   are talking about static HTML content, which only change upon package
> >   upgrade, I believe it would not be appropriate. A better place would
> >   probably be /usr/share/www/PACKAGE/ (maybe some FHS guru can give us
> >   some insights here ...)
>
> Full ack, and I even like /usr/share/www. It's easy to understand and
> pretty unprobable that we'd have a package called www in the archive
> some day needing this location.

Fine, it seems we're done with this aspect then.

> > - Regarding the URL that would be mapped to that dir, I don't
> >   particularly like /debian/ (even though I've advanced it). However the
> >   alternative solutions I can come up with (e.g. /packages/) are
> >   actually uglier. So I guess /debian/ can stay. Some of the -webapps
> >   people can probably come up with wiser suggestions ...
>
> Manoj suggested '/vendor-apps' and I like that. It says what it should
> say and doesn't steal any probable path.

Right, but it is possibly hard to type and a bit ugly, I've
counter-proposed "/vendor/" (see my separate mail on that).

> I still see a problem with the upgrade path for existing installations.
> I might be wrong but I think the most difficult cases are very custom
> setups with lots of changes by the local admin. I'm thinking of e.g.
> webmail.domain.tld being a virtual host with DocRoot
> /usr/share/squirrelmail. If the files there move to
> /usr/share/www/squirrelmail we break a lot of setups. So, what about
> shipping a symlink from the old location to the new one as a migration
> path? This doesn't solve the very default (e.g. users accessing
> squirrelmail via localhost/squirrelmail) but that is so easily solved
> via alias directive or symlink that I suppose a NEWS.Debian entry would
> fit best here.
> What do you say?

I think that migrations from complex setup to this new setup will stay
complex no matter what we do. Also, it is not really something we can
"standardize" upon as migrations are very specific to each involved
packages and will ultimately be dealed with by single maintainers. So
I'd refrain to propose a generic upgrade path and just describe the new
situation we want to obtain.

> Now, I'm willing to run this, i.e. file bugs against web servers, wait
> for them to be fixed, then file bugs against web applications (if
> needed, I'm right now looking into a way to make a lintian check for it,
> e.g. package-with-section-web-but-no-files-in-canonical-docroot). But I
> don't feel like we're having a clear consensus here, do we?

Well, defining consensus is always a tricky business :), but I haven't
heard significant voices against, am I wrong? I'd personally proceed as
follow: write a draft document (even a very brief one!) which summarizes
the proposal so that people do not need to dig into the thread to follow
the evolution. Once we have it, re-post it to the relevant lists (I'd
say -devel, -policy, -webapps) and ask for comments.

Actually, your suggested MBF text can pretty much be that document. If
it were me, I'd open a DEP on that just because I fear losing track of
it, but YMMV.

> Attached is a suggestion for MBF as well as a dd-list of all relevant
> web servers (if I did my job right).

Wow, thanks!

Cheers.

--
Stefano Zacchiroli -o- PhD in Computer Science \ PostDoc @ Univ. Paris 7
zack@{upsilon.cc,pps.jussieu.fr,debian.org} -<>- http://upsilon.cc/zack/
Dietro un grande uomo c'è ..|  .  |. Et ne m'en veux pas si je te tutoie
sempre uno zaino ...........| ..: |.... Je dis tu à tous ceux que j'aime

[Stefano Zacchiroli]

On Sun, Nov 08, 2009 at 12:09:28AM +0900, Charles Plessy wrote:
> For new packages, grouping everything in /usr/share/www sounds like a good
> idea.  The alias name, « vendor », I find a bit disturbing because we do not
> sell anything. But picking the name will be the priviledge of the Do-o-crat who
> will lead the transition, I presume.

Well, it is actually pretty common in cross-distro lingo, Debian is a
vendor as well as pretty much every other distro is. The advantage of
settling on such a name IMO would be a higher chance in making it
popular in other distros. Also, it is a name that I _think_ is pretty
unlike to be used by local admins.

> Still, having /usr/share/www as a document root does not prevent complex
> packages to be fragmented between /usr/share, /usr/lib/cgi-bin/, /var/lib/,
> /var/tmp, /var/run and /etc. Maybe you can double-check how many web servers
> are able to cope with that before starting to invest a lot of time. Otherwise,
> since shipping configuration files in /etc/webserver/conf.d will still be
> necessary for these packages to work, there will little benefit in moving files
> to /usr/share/www.

I don't understand this argument. Sure, complex packages will be split
in several dirs, our policy states the rule for that to happen. The
whole point of this standardization is to have a single URL prefix under
which _entry_points_ for shipped web applications can be found, no
matter how the applications are deployed on the filesystem. I found such
a goal worthwhile by itself and orthogonal to the other concern you
raise.

Cheers.

--
Stefano Zacchiroli -o- PhD in Computer Science \ PostDoc @ Univ. Paris 7
zack@{upsilon.cc,pps.jussieu.fr,debian.org} -<>- http://upsilon.cc/zack/
Dietro un grande uomo c'è ..|  .  |. Et ne m'en veux pas si je te tutoie
sempre uno zaino ...........| ..: |.... Je dis tu à tous ceux que j'aime

[Charles Plessy-12]

Le Mon, Nov 09, 2009 at 10:24:39AM +0100, Stefano Zacchiroli a écrit :

> On Sun, Nov 08, 2009 at 12:09:28AM +0900, Charles Plessy wrote:
>
> > Still, having /usr/share/www as a document root does not prevent complex
> > packages to be fragmented between /usr/share, /usr/lib/cgi-bin/, /var/lib/,
> > /var/tmp, /var/run and /etc. Maybe you can double-check how many web servers
> > are able to cope with that before starting to invest a lot of time. Otherwise,
> > since shipping configuration files in /etc/webserver/conf.d will still be
> > necessary for these packages to work, there will little benefit in moving files
> > to /usr/share/www.
>
> I don't understand this argument. Sure, complex packages will be split
> in several dirs, our policy states the rule for that to happen. The
> whole point of this standardization is to have a single URL prefix under
> which _entry_points_ for shipped web applications can be found, no
> matter how the applications are deployed on the filesystem. I found such
> a goal worthwhile by itself and orthogonal to the other concern you
> raise.

Hi Stefano,

the lintian error dir-or-file-in-var-www exists for a long time, and I believe
that most packages with active maintainers have already been split according to the
FHS. What I question is whether it is worth the effort to move the content of
/usr/share/<package> to /usr/share/www/<package>:

 - How many purely static websites do we distribute as Debian packages?
   (Note that /usr/share/doc/<package> is already served as http://localhost/doc/package/)

 - How many dynamic websites will start to work out of the box without
   the need for a specific configuration for each webserver?

I checked at the web application I maintain (emboss-explorer), and in its
particular case, it would still need an apache.conf file. That is not enough to
make statistics, so I am just asking if there will be many packages that can
take advantage of the proposed reorganisation. [And unfortunately source.debian.net
looks borken again…].

Of course, if the use of /usr/share/www/<package> is optional, everybody wins.

Cheers,

--
Charles Plessy
Debian Med packaging team,
http://www.debian.org/devel/debian-med
Tsurumi, Kanagawa, Japan


--
To UNSUBSCRIBE, email to debian-policy-REQUEST@...
with a subject of "unsubscribe". Trouble? Contact listmaster@...

[Jan Hauke Rahm-2]

On Mon, Nov 09, 2009 at 10:21:12AM +0100, Stefano Zacchiroli wrote:

> On Sat, Nov 07, 2009 at 03:23:22PM +0100, Jan Hauke Rahm wrote:
> > I still see a problem with the upgrade path for existing installations.
> > I might be wrong but I think the most difficult cases are very custom
> > setups with lots of changes by the local admin. I'm thinking of e.g.
> > webmail.domain.tld being a virtual host with DocRoot
> > /usr/share/squirrelmail. If the files there move to
> > /usr/share/www/squirrelmail we break a lot of setups. So, what about
> > shipping a symlink from the old location to the new one as a migration
> > path? This doesn't solve the very default (e.g. users accessing
> > squirrelmail via localhost/squirrelmail) but that is so easily solved
> > via alias directive or symlink that I suppose a NEWS.Debian entry would
> > fit best here.
> > What do you say?
>
> I think that migrations from complex setup to this new setup will stay
> complex no matter what we do. Also, it is not really something we can
> "standardize" upon as migrations are very specific to each involved
> packages and will ultimately be dealed with by single maintainers. So
> I'd refrain to propose a generic upgrade path and just describe the new
> situation we want to obtain.

I agree, I was just pointing out that common setups can have a proper
migration path. We could give a hint when we're at it but the maintainer
needs to think of something him/herself after all.

> > Now, I'm willing to run this, i.e. file bugs against web servers, wait
> > for them to be fixed, then file bugs against web applications (if
> > needed, I'm right now looking into a way to make a lintian check for it,
> > e.g. package-with-section-web-but-no-files-in-canonical-docroot). But I
> > don't feel like we're having a clear consensus here, do we?
>
> Well, defining consensus is always a tricky business :), but I haven't
> heard significant voices against, am I wrong? I'd personally proceed as
> follow: write a draft document (even a very brief one!) which summarizes
> the proposal so that people do not need to dig into the thread to follow
> the evolution. Once we have it, re-post it to the relevant lists (I'd
> say -devel, -policy, -webapps) and ask for comments.

I'll try to come up with something within the next 24 hours. Don't know
yet if it's gonna be a DEP or just another mail but summarizing what we
got so far sounds like a plan.

Hauke

[Stefano Zacchiroli]

On Mon, Nov 09, 2009 at 07:04:22PM +0900, Charles Plessy wrote:

> the lintian error dir-or-file-in-var-www exists for a long time, and I
> believe that most packages with active maintainers have already been
> split according to the FHS. What I question is whether it is worth the
> effort to move the content of /usr/share/<package> to
> /usr/share/www/<package>:
>
>  - How many purely static websites do we distribute as Debian packages?
>    (Note that /usr/share/doc/<package> is already served as http://localhost/doc/package/)
>
>  - How many dynamic websites will start to work out of the box without
>    the need for a specific configuration for each webserver?

I now understand better your argument, thanks for rephrasing. I don't
have an answer, because I haven't done the test, but I do agree that it
would be interesting to know in advance. Still, it is not exactly clear
to me how to test this, how would you automatically discover whether a
package has a static splash screen (note indeed that it is not only
about "purely static" web applications, but also about "regular"
webapps, with a static splash screen).
>
> I checked at the web application I maintain (emboss-explorer), and in its
> particular case, it would still need an apache.conf file. That is not enough to
> make statistics, so I am just asking if there will be many packages that can
> take advantage of the proposed reorganisation. [And unfortunately source.debian.net
> looks borken again…].

Can you perhaps explain why so?

I frankly hope that with /vendor/ + /usr/lib/cgi-bin/ (which we already
have), and maybe with some symlinks under /vendor/ we will be able to
address quite a lot of issues. It would be interesting to known which
one we can't.

Now that I think of it, probably a per-package data dir would help, but
that can be a tad more tricky due to single-instance of
multiple-instance nature of the webapp in question ...

> Of course, if the use of /usr/share/www/<package> is optional,
> everybody wins.

It is forcibly optional: if you have some static content you should use,
otherwise not.

Cheers.

--
Stefano Zacchiroli -o- PhD in Computer Science \ PostDoc @ Univ. Paris 7
zack@{upsilon.cc,pps.jussieu.fr,debian.org} -<>- http://upsilon.cc/zack/
Dietro un grande uomo c'è ..|  .  |. Et ne m'en veux pas si je te tutoie
sempre uno zaino ...........| ..: |.... Je dis tu à tous ceux que j'aime

[sean finney]

On Mon, Nov 09, 2009 at 06:15:42PM +0100, Stefano Zacchiroli wrote:
> I frankly hope that with /vendor/ + /usr/lib/cgi-bin/ (which we already
> have), and maybe with some symlinks under /vendor/ we will be able to
> address quite a lot of issues. It would be interesting to known which
> one we can't.

objection, you honor!  :)

something that hasn't really been brought up (i mentioned it on the
non-webapps thread in -devel already) is that this makes packages
potentially opened in an unconfigured state.  unless you can ensure that
the system is only running on localhost, it has some significant security
implications.  personally i'd rather that /usr/lib/cgi-bin goes the way
of the dodo, and that packages are required to ship/generate webserver
config files if they want to function out of the box.


sean

[Russ Allbery-2]

sean finney <seanius@...> writes:

> something that hasn't really been brought up (i mentioned it on the
> non-webapps thread in -devel already) is that this makes packages
> potentially opened in an unconfigured state.  unless you can ensure that
> the system is only running on localhost, it has some significant
> security implications.  personally i'd rather that /usr/lib/cgi-bin goes
> the way of the dodo, and that packages are required to ship/generate
> webserver config files if they want to function out of the box.

Wholeheartedly agreed, particularly if we can put a management system in
place similar to the (really nice) Apache module management system that
lets admins selectively enable specific applications, which installing
everything into a default CGI-active directory doesn't permit as easily.

--
Russ Allbery (rra@...)               <http://www.eyrie.org/~eagle/>


--
To UNSUBSCRIBE, email to debian-policy-REQUEST@...
with a subject of "unsubscribe". Trouble? Contact listmaster@...

[Jan Hauke Rahm-2]

On Mon, Nov 09, 2009 at 03:55:58PM -0800, Russ Allbery wrote:

> sean finney <seanius@...> writes:
>
> > something that hasn't really been brought up (i mentioned it on the
> > non-webapps thread in -devel already) is that this makes packages
> > potentially opened in an unconfigured state.  unless you can ensure that
> > the system is only running on localhost, it has some significant
> > security implications.  personally i'd rather that /usr/lib/cgi-bin goes
> > the way of the dodo, and that packages are required to ship/generate
> > webserver config files if they want to function out of the box.
>
> Wholeheartedly agreed, particularly if we can put a management system in
> place similar to the (really nice) Apache module management system that
> lets admins selectively enable specific applications, which installing
> everything into a default CGI-active directory doesn't permit as easily.

Not that I'm opposing to what you're saying but... every application in
the archive is configured during the installation process, possibly
asking debconf questions, providing defaults etc. After the installation
it should run in a mode that suites most use cases and is secure. We (or
at least I) always expected that.

Now with web applications, if I read you suggestions correctly, you want
to just throw the files in the system, leave it unconfigured without
meaningfull defaults, even leading to an unsecure state, and then blame
the web server for not securing the application?

Or am I misunderstanding you?
Hauke