Deprecating /usr/lib/site-python in python policy

[Raphael Hertzog-3]

Hi,

as part of the new policy we should also deprecate /usr/lib/site-python.

Since this directory is in sys.path of all python versions, if we
byte-compile those in-place for the current version, then the modules
won't work with other python versions.

Most packages using this directory (like linda) use this directory for
private python modules that should be moved in /usr/share/$package
for examples.
   
This principle has been agreed by Matthias and Josselin.

Someone should file a wishlist bug on lintian to check that packages don't
use /usr/lib/site-python any more.

Cheers,
--
Raphaël Hertzog

Premier livre français sur Debian GNU/Linux :
http://www.ouaza.com/livre/admin-debian/


--
To UNSUBSCRIBE, email to debian-python-REQUEST@...
with a subject of "unsubscribe". Trouble? Contact listmaster@...

[Dafydd Harries-4]

Ar 11/06/2006 am 14:35, ysgrifennodd Raphael Hertzog:
> Someone should file a wishlist bug on lintian to check that packages don't
> use /usr/lib/site-python any more.

Done: http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=372748

--
Dafydd


--
To UNSUBSCRIBE, email to debian-python-REQUEST@...
with a subject of "unsubscribe". Trouble? Contact listmaster@...

[Thomas Viehmann]

tag 372748 +patch
thanks

Dafydd Harries wrote:
> Ar 11/06/2006 am 14:35, ysgrifennodd Raphael Hertzog:
>> Someone should file a wishlist bug on lintian to check that packages don't
>> use /usr/lib/site-python any more.
> Done: http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=372748
Maybe the attached patch is useful.

Kind regards

T.
--
Thomas Viehmann, http://thomas.viehmann.net/

diff -burN lintian-1.23.21/checks/files lintian-1.23.21.tv0/checks/files
--- lintian-1.23.21/checks/files 2006-05-04 05:37:21.000000000 +0200
+++ lintian-1.23.21.tv0/checks/files 2006-06-11 21:13:33.000000000 +0200
@@ -448,6 +448,10 @@
  tag "package-installs-python-pyc", "$file"
     }
 
+    # ---------------- /usr/lib/site-python
+    if ($file =~ m,^usr/lib/site-python/,) {
+ tag "file-in-usr-lib-site-python", $file
+    }
     # ---------------- pythonX.Y extensions
     if ($file =~ m,^usr/lib/python\d\.\d/\S,
  and not $file =~ m,^usr/lib/python\d\.\d/site-packages/,) {
diff -burN lintian-1.23.21/checks/files.desc lintian-1.23.21.tv0/checks/files.desc
--- lintian-1.23.21/checks/files.desc 2006-05-04 05:37:21.000000000 +0200
+++ lintian-1.23.21.tv0/checks/files.desc 2006-06-11 18:36:34.000000000 +0200
@@ -360,6 +360,12 @@
 Info: Debian packages should not install into <tt>/opt</tt>, because it
  is reserved for add-on software.
 
+Tag: file-in-usr-lib-site-python
+Type: error
+Info: The directory /usr/lib/site-python has been deprecated as a
+ location for installing Python packages. Packages should use
+ /usr/share/$package instead.
+
 Tag: third-party-package-in-python-dir
 Type: warning
 Info: According to <a href="http://www.python.org/doc/essays/packages.html">

[Raphael Hertzog-3]

On Sun, 11 Jun 2006, Thomas Viehmann wrote:

> --- lintian-1.23.21/checks/files.desc 2006-05-04 05:37:21.000000000 +0200
> +++ lintian-1.23.21.tv0/checks/files.desc 2006-06-11 18:36:34.000000000 +0200
> @@ -360,6 +360,12 @@
>  Info: Debian packages should not install into <tt>/opt</tt>, because it
>   is reserved for add-on software.
>  
> +Tag: file-in-usr-lib-site-python
> +Type: error
> +Info: The directory /usr/lib/site-python has been deprecated as a
> + location for installing Python packages. Packages should use
> + /usr/share/$package instead.

I suggest this:
Info: The directory /usr/lib/site-python has been deprecated as a
 location for installing Python modules. Please check the Python policy.
 Most likely the modules are private modules and should be
 packaged in a directory outside of Python's default search path.

Cheers,
--
Raphaël Hertzog

Premier livre français sur Debian GNU/Linux :
http://www.ouaza.com/livre/admin-debian/


--
To UNSUBSCRIBE, email to debian-python-REQUEST@...
with a subject of "unsubscribe". Trouble? Contact listmaster@...

[Alexandre Fayolle]

On Sun, Jun 11, 2006 at 02:35:05PM +0200, Raphael Hertzog wrote:
> Hi,
>
> as part of the new policy we should also deprecate /usr/lib/site-python.

I'll have to reread the new policy, but I think this is not a very good
idea. /usr/lib/site-python exists for modules which do not care about
which version of the python interpreter is used and is therefore very
useful.

>
> Since this directory is in sys.path of all python versions, if we
> byte-compile those in-place for the current version, then the modules
> won't work with other python versions.

This just not true. Python is smart enough not to use the .pyc if the
version used to produce it is not the one which is used. The problem
only exists for python extensions, but pure python modules have no
problem with this.


 

--
Alexandre Fayolle                              LOGILAB, Paris (France)
Formations Python, Zope, Plone, Debian:  http://www.logilab.fr/formations
Développement logiciel sur mesure:       http://www.logilab.fr/services
Informatique scientifique:               http://www.logilab.fr/science

[Raphael Hertzog-3]

On Mon, 12 Jun 2006, Alexandre Fayolle wrote:
> > Since this directory is in sys.path of all python versions, if we
> > byte-compile those in-place for the current version, then the modules
> > won't work with other python versions.
>
> This just not true. Python is smart enough not to use the .pyc if the
> version used to produce it is not the one which is used. The problem
> only exists for python extensions, but pure python modules have no
> problem with this.

/usr/lib/site-python is on the sys.path of all python versions so we must
make sure to provide the best (ie .py with working .pyc) for all python
versions. We can't do that within /usr/lib/site-python so we must do it
outside.

python-support and python-central provides convenient support for that.
Any package using /usr/lib/site-python shoud be changed to use one of
those ... or if the modules shouldn't be public, then it should use a
private directory that the application manually adds to the sys.path.

Cheers,
--
Raphaël Hertzog

Premier livre français sur Debian GNU/Linux :
http://www.ouaza.com/livre/admin-debian/


--
To UNSUBSCRIBE, email to debian-python-REQUEST@...
with a subject of "unsubscribe". Trouble? Contact listmaster@...

[Juha-Matti Tapio]

On Mon, Jun 12, 2006 at 09:20:13AM +0200, Raphael Hertzog wrote:
> /usr/lib/site-python is on the sys.path of all python versions so we must
> make sure to provide the best (ie .py with working .pyc) for all python
> versions.

Why "must"?

Has this for some reason become a problem just recently or has this always
been a problem? I do not think missing or invalid .pyc's there could cause
noticable performance problems (especially since it is at the end of the
search path or at least near it) and certainly the modules are importable
from multiple python versions already.

As a user I am greatful for all the people making an effort to improve
Debian Python, but I am beginning to worry if you people are taking too many
steps at once this close to the freeze.

[Raphael Hertzog-3]

On Mon, 12 Jun 2006, Juha-Matti Tapio wrote:
> On Mon, Jun 12, 2006 at 09:20:13AM +0200, Raphael Hertzog wrote:
> > /usr/lib/site-python is on the sys.path of all python versions so we must
> > make sure to provide the best (ie .py with working .pyc) for all python
> > versions.
>
> Why "must"?

Well "should".

> Has this for some reason become a problem just recently or has this always
> been a problem? I do not think missing or invalid .pyc's there could cause

It's not a major problem, we won't be pushing hard for it but it's good
to update the policy accordingly however.

We'll concentrate on fixing the modules first. The application won't stop
working whereas modules could.

Cheers,
--
Raphaël Hertzog

Premier livre français sur Debian GNU/Linux :
http://www.ouaza.com/livre/admin-debian/


--
To UNSUBSCRIBE, email to debian-python-REQUEST@...
with a subject of "unsubscribe". Trouble? Contact listmaster@...

[Joe Wreschnig-2]

On Mon, 2006-06-12 at 17:08 +0300, Juha-Matti Tapio wrote:

> On Mon, Jun 12, 2006 at 09:20:13AM +0200, Raphael Hertzog wrote:
> > /usr/lib/site-python is on the sys.path of all python versions so we must
> > make sure to provide the best (ie .py with working .pyc) for all python
> > versions.
>
> Why "must"?
>
> Has this for some reason become a problem just recently or has this always
> been a problem? I do not think missing or invalid .pyc's there could cause
> noticable performance problems (especially since it is at the end of the
> search path or at least near it) and certainly the modules are importable
> from multiple python versions already.
>
> As a user I am greatful for all the people making an effort to improve
> Debian Python, but I am beginning to worry if you people are taking too many
> steps at once this close to the freeze.

Only a dozen or so packages bother with /usr/lib/site-python, and almost
all of them are buggy for placing unstable APIs in a public module
directory anyway. Note that it's *not* being removed from sys.path
(yet), it's just being deprecated. We deprecated /usr/doc with far more
than a dozen packages still using it.
--
Joe Wreschnig <piman@...>