Report on the situation of python2.5 in Debian

[Josselin Mouette]

Hi,

I have conducted a thorough analysis of all packages preventing us from
migrating to python 2.5.

I cannot say that the situation looks good. Since the last transition,
many new packages have been introduced, and a large part of them don't
support the current policy. Another bad news is that it isn't possible
to rely on the packages' metadata, as many maintainers don't understand
what the X?-Python-Version fields mean.


Let's start with the good news. The following packages are included in
python2.5, and will disappear or become legacy.
        celementtree
        ctypes
        python-pysqlite2
        python-wsgiref


The following packages need a round of binNMUs as soon as possible, to
build extensions for python2.5, after which they shouldn't bother us.
Can anyone schedule the binNMUs please?
        bitpim
        cx-bsdiff
        duplicity
        elementtidy
        eunuchs
        gamin
        mmpython
        pyalsaaudio
        pyao
        pychm
        pymad
        pyme
        pyopenssl
        pypoker-eval
        pystatgrab
        python-adns
        python-biggles
        python-cddb
        python-crack
        python-daap
        python-extclass
        python-fam
        python-gd
        python-geoip
        python-omniorb2
        python-osd
        python-pgsql
        python-pqueue
        python-pylibacl
        python-scientific
        python-smbpasswd
        python-tcpwrap
        python-utmp
        python-xattr
        python-yenc
        pyvorbis
        pyx
        pyxmpp
        qm
        quixote1
        reportlab-accel
        zodb
        zope-textindexng2

The following packages should also work with python2.5 after a round of
binNMUs, as some dependencies were relaxed in python-support 0.7.4:
        dia
        exaile
        k3d


Now, for the bad news. The following packages have various kinds of
issues that prevent them from working with python2.5. Some of them are
trivial, some are much more tricky. In all cases, we need to focus on
these bugs if we want to see those packages in lenny:
        buxon #445247
        decompyle #445248
        gajim #445275
        gaphas #445397
        gaphor #445277
        gpodder #445278
        hamlib #445382
        hk-classes #445372
        imgsizer #445280
        jppy #445379
        kde-guidance #445281
        lcms #436541
        ldaptor #445285
        libsvm #445386
        londonlaw #445288
        maxdb-7.5.00 #445289
        moosic #445400
        musiclibrarian #445399
        newt #445392
        opensync #445394
        postr #445403
        pykdeextensions #445292
        pyroman #445408
        python-gammu #445370
        pyvtk #445396
        quantlib-swig #445416
        quixote #445405
        revelation #445415
        scanerrlog #406729
        swaml #445422
        urlscan #445424
        xmms2 #445409
I request a zero-day NMU policy for these bugs.

The following packages are also buggy, but they are nevertheless too
buggy for other reasons; let's ignore them.
        diacanvas2
        pyspi
        schooltool


The following packages do not support multiple versions of python at
once. This is where we have the most serious regression compared to the
situation of the python2.4 transition. It is understandable not to
rebuild the gimp or OpenOffice.org packages for several python versions,
but many of these packages are using distutils and are therefore
*trivial* to get to work with several versions.

Please note that they can all be binNMUed after python2.5 has become the
default, but all of them will have to migrate to testing at once. We
must make this list shorter unless we want this transition to recall bad
memories to the release team.

Here is the list:
        adesklets
        aubio
        audit
        beagle
        brltty
        capisuite
        comedilib
        cwiid
        dballe
        dds
        deskbar-applet
        eikazo
        empathy
        galago-python
        galago-python-gtk
        gimmie
        gimp
        gnome-orca
        gnuradio
        hocr
        hplip
        imgseek
        jppy
        kdebindings
        libapache2-mod-python
        libbtctl
        libhdate
        libhid
        libipod
        libmetakit2.4.9.3
        libphidgets
        memaid-pyqt
        mirage
        mod-wsgi
        music-applet
        ninix-aya
        notify-python
        nufw
        ocempgui
        ocfs2-tools
        omniidl4
        opencv
        openoffice.org
        pida
        player
        pyclutter
        pyg
        pygoocanvas
        pykaraoke
        pymol
        pynjb
        pypanel
        pyqonsole
        pyrite-publisher
        pyslide
        python-fuse
        python-libpcap
        pytone
        pyxine
        rdiff-backup
        renpy
        sabayon
        skencil
        smart
        snappea
        solfege
        sonata
        sqlrelay
        streamtuner
        subterfugue
        subversion
        synopsis
        vtk
        wxwidgets2.6
        xmldiff
        xulrunner
        zeroc-ice-python
I also request a zero-day NMU policy for these issues, at least for
packages using distutils.


Finally, for the following packages, I'm waiting for the situation to
clarify in libboost. It is being discussed in bug #445381.
        democracyplayer #445249
        libavg
        miro
        python-visual
        pythonmagick #445395


A status of all opened bugs can be found here:
http://bugs.debian.org/cgi-bin/pkgreport.cgi?tag=python2.5;users=joss@...

Thanks for reading.
--
 .''`.           Josselin Mouette        /\./\
: :' :           josselin.mouette@...
`. `'                        joss@...
  `-  Debian GNU/Linux -- The power of freedom

[Bernd Zeimetz]

Hi,

> Now, for the bad news. The following packages have various kinds of
> issues that prevent them from working with python2.5. Some of them are
> trivial, some are much more tricky. In all cases, we need to focus on
> these bugs if we want to see those packages in lenny:

You have missed Zope.
It is not possible to run Zope 2.X with Python 2.5 yet, same for Zope 3.
The problems in there are nothing a maintainer could fix, except
somebody is willing to pay several days (weeks!?) of work.
Changing the default python version also means to go trough all Zope
packages and replace /usr/bin/python by /usr/bin/python2.4.

> The following packages do not support multiple versions of python at
> once. This is where we have the most serious regression compared to the
> situation of the python2.4 transition. It is understandable not to
> rebuild the gimp or OpenOffice.org packages for several python versions,
> but many of these packages are using distutils and are therefore
> *trivial* to get to work with several versions.

Does the list include those packages which are not conform to the new
Python policy? There're still several of them out there. A zero-day NMU
policy would be good to have here, too.
http://bugs.debian.org/from:madcoder-python-transition@...;pend-exc=done;exclude=fixed


> Please note that they can all be binNMUed after python2.5 has become the
> default, but all of them will have to migrate to testing at once. We
> must make this list shorter unless we want this transition to recall bad
> memories to the release team.

>         mod-wsgi

It's not possible to use more than one python version with mod-wsgi,
therefore it will only work and be build against the default python
version. A binNMU after changing the version should work well.


Cheers,

Bernd

--
Bernd Zeimetz
<bernd@...>                         <http://bzed.de/>


--
To UNSUBSCRIBE, email to debian-python-REQUEST@...
with a subject of "unsubscribe". Trouble? Contact listmaster@...

[Pierre Habouzit-2]

On Fri, Oct 05, 2007 at 06:42:46PM +0000, Bernd Zeimetz wrote:
> Hi,
>
> > Now, for the bad news. The following packages have various kinds of
> > issues that prevent them from working with python2.5. Some of them are
> > trivial, some are much more tricky. In all cases, we need to focus on
> > these bugs if we want to see those packages in lenny:
>
> You have missed Zope.

  Zope is kind of outside the python policy because it always lags
behind, and is dealt with in its own round, I presume that's why Joss
didn't talked about it.

> Does the list include those packages which are not conform to the new
> Python policy? There're still several of them out there. A zero-day NMU
> policy would be good to have here, too.
> http://bugs.debian.org/from:madcoder-python-transition@...;pend-exc=done;exclude=fixed

  Last time I checked, most of them weren't in testing for ages. But
this wasn't thorough.

> > Please note that they can all be binNMUed after python2.5 has become the
> > default, but all of them will have to migrate to testing at once. We
> > must make this list shorter unless we want this transition to recall bad
> > memories to the release team.
>
> >         mod-wsgi
>
> It's not possible to use more than one python version with mod-wsgi,
> therefore it will only work and be build against the default python
> version. A binNMU after changing the version should work well.

  yes, for some packages it makes sense, Joss didn't asked for an empty
list, merely a shorter one, and I agree with him.

--
·O·  Pierre Habouzit
··O                                                madcoder@...
OOO                                                http://www.madism.org

[Bernd Zeimetz]

>>> Please note that they can all be binNMUed after python2.5 has become the
>>> default, but all of them will have to migrate to testing at once. We
>>> must make this list shorter unless we want this transition to recall bad
>>> memories to the release team.
>>>         mod-wsgi
>> It's not possible to use more than one python version with mod-wsgi,
>> therefore it will only work and be build against the default python
>> version. A binNMU after changing the version should work well.
>
>   yes, for some packages it makes sense, Joss didn't asked for an empty
> list, merely a shorter one, and I agree with him.

I agree with him, too - just wanted to make sure nobody tries to "fix"
mod-wsgi.

--
Bernd Zeimetz
<bernd@...>                         <http://bzed.de/>


--
To UNSUBSCRIBE, email to debian-python-REQUEST@...
with a subject of "unsubscribe". Trouble? Contact listmaster@...

[Pierre Habouzit-2]

On Fri, Oct 05, 2007 at 06:04:27PM +0000, Josselin Mouette wrote:
> Hi,
>
> I have conducted a thorough analysis of all packages preventing us from
> migrating to python 2.5.

  Thanks a lot for the work !


> Now, for the bad news. The following packages have various kinds of
> issues that prevent them from working with python2.5. Some of them are
> trivial, some are much more tricky. In all cases, we need to focus on
> these bugs if we want to see those packages in lenny:
[...]
> I request a zero-day NMU policy for these bugs.

  I second this.

> The following packages do not support multiple versions of python at
> once. This is where we have the most serious regression compared to the
> situation of the python2.4 transition. It is understandable not to
> rebuild the gimp or OpenOffice.org packages for several python versions,
> but many of these packages are using distutils and are therefore
> *trivial* to get to work with several versions.
>
> Please note that they can all be binNMUed after python2.5 has become the
> default, but all of them will have to migrate to testing at once. We
> must make this list shorter unless we want this transition to recall bad
> memories to the release team.
>
> Here is the list:

[...]
> I also request a zero-day NMU policy for these issues, at least for
> packages using distutils.

  This is a tad aggressive as for some of the packages (see wcgi) it
make sense to stay like that, and an NMUer that does 10 of those in a
round may miss the reasons. I'd rather see a mass bug filling on those
packages, and see the maintainers that feel their package should be
built for one version only say it loud first.

  Then maybe we will consider making it RC.

--
·O·  Pierre Habouzit
··O                                                madcoder@...
OOO                                                http://www.madism.org

[Mike Hommey]

On Fri, Oct 05, 2007 at 08:04:27PM +0200, Josselin Mouette wrote:
> Please note that they can all be binNMUed after python2.5 has become the
> default, but all of them will have to migrate to testing at once. We
> must make this list shorter unless we want this transition to recall bad
> memories to the release team.

Well, then, that's not going to happen before the toolchain gets fixed
on mips, because of

> Here is the list:
(...)
>         xulrunner

Mike


--
To UNSUBSCRIBE, email to debian-python-REQUEST@...
with a subject of "unsubscribe". Trouble? Contact listmaster@...

[Josselin Mouette]

Le vendredi 05 octobre 2007 à 20:42 +0200, Bernd Zeimetz a écrit :
> You have missed Zope.
> It is not possible to run Zope 2.X with Python 2.5 yet, same for Zope 3.
> The problems in there are nothing a maintainer could fix, except
> somebody is willing to pay several days (weeks!?) of work.
> Changing the default python version also means to go trough all Zope
> packages and replace /usr/bin/python by /usr/bin/python2.4.

My guess is that must have already been done, because all zope packages
(except those I have listed) already depend on python2.4.

> Does the list include those packages which are not conform to the new
> Python policy? There're still several of them out there. A zero-day NMU
> policy would be good to have here, too.
> http://bugs.debian.org/from:madcoder-python-transition@...;pend-exc=done;exclude=fixed

AFAICT the remaining packages shouldn't prevent python2.5 to migrate to
testing. Still, these bugs also deserve a 0-day NMU policy indeed.

> >         mod-wsgi
>
> It's not possible to use more than one python version with mod-wsgi,
> therefore it will only work and be build against the default python
> version. A binNMU after changing the version should work well.

I think it should be possible, by building several versions and using a
rtupdate hook to change a symbolic link pointing to one of them. Still,
that makes the package much harder to change than those using distutils,
for which there is no excuse.

A 0-day NMU policy is indeed a bit too much for such packages; it would
be nice to have them fixed, but it should be done in agreement with the
maintainer.

--
 .''`.           Josselin Mouette        /\./\
: :' :           josselin.mouette@...
`. `'                        joss@...
  `-  Debian GNU/Linux -- The power of freedom

[Bernd Zeimetz]

Josselin Mouette wrote:

> Le vendredi 05 octobre 2007 à 20:42 +0200, Bernd Zeimetz a écrit :
>> You have missed Zope.
>> It is not possible to run Zope 2.X with Python 2.5 yet, same for Zope 3.
>> The problems in there are nothing a maintainer could fix, except
>> somebody is willing to pay several days (weeks!?) of work.
>> Changing the default python version also means to go trough all Zope
>> packages and replace /usr/bin/python by /usr/bin/python2.4.
>
> My guess is that must have already been done, because all zope packages
> (except those I have listed) already depend on python2.4.

They depend on python2.4, but I'm pretty sure a lot of them use
#!/usr/bin/python, unfortunately. I'm forwading the mail to the Zope
lsit therefore.



>> Does the list include those packages which are not conform to the new
>> Python policy? There're still several of them out there. A zero-day NMU
>> policy would be good to have here, too.
>> http://bugs.debian.org/from:madcoder-python-transition@...;pend-exc=done;exclude=fixed
>
> AFAICT the remaining packages shouldn't prevent python2.5 to migrate to
> testing. Still, these bugs also deserve a 0-day NMU policy indeed.

As far as I remember several of them were supposed to be orphaned
anyway, removing instead of fixing would be the way for them.


>>>         mod-wsgi
>> It's not possible to use more than one python version with mod-wsgi,
>> therefore it will only work and be build against the default python
>> version. A binNMU after changing the version should work well.
>
> I think it should be possible, by building several versions and using a
> rtupdate hook to change a symbolic link pointing to one of them.

Probably I'll implement that, and provide a module for every python
version. That's something I have to talk trough with upstream first, though.



--
Bernd Zeimetz
<bernd@...>                         <http://bzed.de/>


--
To UNSUBSCRIBE, email to debian-python-REQUEST@...
with a subject of "unsubscribe". Trouble? Contact listmaster@...

[Josselin Mouette]

Le vendredi 05 octobre 2007 à 21:21 +0200, Mike Hommey a écrit :
> Well, then, that's not going to happen before the toolchain gets fixed
> on mips, because of
>
> > Here is the list:
> (...)
> >         xulrunner

Well, I don't think this transition will be done in a day. If, by the
time that is required to come to switch python 2.5, the toolchain still
isn't fixed, that will say much about the state of the mips port.

--
 .''`.           Josselin Mouette        /\./\
: :' :           josselin.mouette@...
`. `'                        joss@...
  `-  Debian GNU/Linux -- The power of freedom

[Josselin Mouette]

Le vendredi 05 octobre 2007 à 20:04 +0200, Josselin Mouette a écrit :
> The following packages should also work with python2.5 after a round of
> binNMUs, as some dependencies were relaxed in python-support 0.7.4:
>         dia
>         exaile
>         k3d

You can add:
        mod-wsgi
which in fact enters in this category.

--
 .''`.
: :' :      We are debian.org. Lower your prices, surrender your code.
`. `'       We will add your hardware and software distinctiveness to
  `-        our own. Resistance is futile.

[Josselin Mouette]

Le samedi 06 octobre 2007 à 09:07 +0200, Josselin Mouette a écrit :

> Le vendredi 05 octobre 2007 à 20:04 +0200, Josselin Mouette a écrit :
> > The following packages should also work with python2.5 after a round of
> > binNMUs, as some dependencies were relaxed in python-support 0.7.4:
> >         dia
> >         exaile
> >         k3d
>
> You can add:
> mod-wsgi
> which in fact enters in this category.

Or not. These dependencies are explicitly added by debian/rules for no
reason.
Sorry, I wasn't awake.

--
 .''`.
: :' :      We are debian.org. Lower your prices, surrender your code.
`. `'       We will add your hardware and software distinctiveness to
  `-        our own. Resistance is futile.

[Bernd Zeimetz]

Josselin Mouette wrote:

> Le samedi 06 octobre 2007 à 09:07 +0200, Josselin Mouette a écrit :
>> Le vendredi 05 octobre 2007 à 20:04 +0200, Josselin Mouette a écrit :
>>> The following packages should also work with python2.5 after a round of
>>> binNMUs, as some dependencies were relaxed in python-support 0.7.4:
>>>         dia
>>>         exaile
>>>         k3d
>> You can add:
>> mod-wsgi
>> which in fact enters in this category.
>
> Or not. These dependencies are explicitly added by debian/rules for no
> reason.

Sorry , but there is a very good reason to do so: If the default python
version changes running applications with mod-wsgi will badly fail.
Therefore it depends in a relaxed way (python <<2.5, but >=2.4) on the
actual default python version which was used to build the package. If
you rebuild it with python2.5 as default python version, it'll just add
the right dependencies. This was tested on Ubuntu and works well. It's
nothing but an extra safety to stop people from upgrading python or
mod-wsgi, they need to be upgraded both.


--
Bernd Zeimetz
<bernd@...>                         <http://bzed.de/>


--
To UNSUBSCRIBE, email to debian-python-REQUEST@...
with a subject of "unsubscribe". Trouble? Contact listmaster@...

[Bernd Zeimetz]

Bernd Zeimetz wrote:

> Josselin Mouette wrote:
>> Le samedi 06 octobre 2007 à 09:07 +0200, Josselin Mouette a écrit :
>>> Le vendredi 05 octobre 2007 à 20:04 +0200, Josselin Mouette a écrit :
>>>> The following packages should also work with python2.5 after a round of
>>>> binNMUs, as some dependencies were relaxed in python-support 0.7.4:
>>>>         dia
>>>>         exaile
>>>>         k3d
>>> You can add:
>>> mod-wsgi
>>> which in fact enters in this category.
>> Or not. These dependencies are explicitly added by debian/rules for no
>> reason.
>
> Sorry , but there is a very good reason to do so: If the default python
> version changes running applications with mod-wsgi will badly fail.
> Therefore it depends in a relaxed way (python <<2.5, but >=2.4) on the
> actual default python version which was used to build the package. If
> you rebuild it with python2.5 as default python version, it'll just add
> the right dependencies. This was tested on Ubuntu and works well. It's
> nothing but an extra safety to stop people from upgrading python or
> mod-wsgi, they need to be upgraded both.

-> it needs to be binNMUed AFTER changing the default python version.


--
Bernd Zeimetz
<bernd@...>                         <http://bzed.de/>


--
To UNSUBSCRIBE, email to debian-python-REQUEST@...
with a subject of "unsubscribe". Trouble? Contact listmaster@...

[Josselin Mouette]

Le samedi 06 octobre 2007 à 12:21 +0200, Bernd Zeimetz a écrit :

> > Sorry , but there is a very good reason to do so: If the default python
> > version changes running applications with mod-wsgi will badly fail.
> > Therefore it depends in a relaxed way (python <<2.5, but >=2.4) on the
> > actual default python version which was used to build the package. If
> > you rebuild it with python2.5 as default python version, it'll just add
> > the right dependencies. This was tested on Ubuntu and works well. It's
> > nothing but an extra safety to stop people from upgrading python or
> > mod-wsgi, they need to be upgraded both.
>
> -> it needs to be binNMUed AFTER changing the default python version.

Which is quite annoying for the transition of python to testing.

I have sent you a multi-build patch for mod-wsgi. As you can see, it
isn't that complicated. And for most packages in the list, the required
changes are much simpler.

--
 .''`.
: :' :      We are debian.org. Lower your prices, surrender your code.
`. `'       We will add your hardware and software distinctiveness to
  `-        our own. Resistance is futile.

[Bernd Zeimetz]

> Which is quite annoying for the transition of python to testing.
>
> I have sent you a multi-build patch for mod-wsgi. As you can see, it
> isn't that complicated. And for most packages in the list, the required
> changes are much simpler.

I indeed didn't know about rtupdate - it seems to be missing on
http://www.debian.org/doc/packaging-manuals/python-policy/
Google found it in
http://people.debian.org/~srivasta/manoj-policy/index.html
which is not a quite obvious place for a documentation.

Thanks for the patch,

Bernd

--
Bernd Zeimetz
<bernd@...>                         <http://bzed.de/>


--
To UNSUBSCRIBE, email to debian-python-REQUEST@...
with a subject of "unsubscribe". Trouble? Contact listmaster@...

[Josselin Mouette]

Le samedi 06 octobre 2007 à 13:26 +0200, Bernd Zeimetz a écrit :
> I indeed didn't know about rtupdate - it seems to be missing on
> http://www.debian.org/doc/packaging-manuals/python-policy/

As some changes that were accepted 7 months ago have still not been
integrated into the policy, I wouldn't expect it to be up-to-date
anyway.

--
 .''`.
: :' :      We are debian.org. Lower your prices, surrender your code.
`. `'       We will add your hardware and software distinctiveness to
  `-        our own. Resistance is futile.

[Aaron M. Ucko]

Josselin Mouette <joss@...> writes:

> The following packages need a round of binNMUs as soon as possible, to
> build extensions for python2.5, after which they shouldn't bother us.
> Can anyone schedule the binNMUs please?
>         bitpim

Please don't bother with this one; BitPim actually builds its
extensions only for whichever Python version is currently default, as
they're private and the app needs wxPython anyway.

--
Aaron M. Ucko, KB1CJC (amu at alum.mit.edu, ucko at debian.org)
http://www.mit.edu/~amu/ | http://stuff.mit.edu/cgi/finger/?amu@...


--
To UNSUBSCRIBE, email to debian-python-REQUEST@...
with a subject of "unsubscribe". Trouble? Contact listmaster@...

[Josselin Mouette]

Le lundi 08 octobre 2007 à 15:59 -0400, Aaron M. Ucko a écrit :
> > The following packages need a round of binNMUs as soon as possible, to
> > build extensions for python2.5, after which they shouldn't bother us.
> > Can anyone schedule the binNMUs please?
> >         bitpim
>
> Please don't bother with this one; BitPim actually builds its
> extensions only for whichever Python version is currently default, as
> they're private and the app needs wxPython anyway.

Indeed, these are private extensions; that moves bitpim to the other
list.

Still, it would be nice to build extensions for all python versions, but
the fact they are private makes this process more complicated.

--
 .''`.           Josselin Mouette        /\./\
: :' :           josselin.mouette@...
`. `'                        joss@...
  `-  Debian GNU/Linux -- The power of freedom

[Aaron M. Ucko]

Josselin Mouette <joss@...> writes:

> Still, it would be nice to build extensions for all python versions, but
> the fact they are private makes this process more complicated.

... particularly given that neither python-support (which BitPim
currently uses) nor python-central actually supports that case
AFAICT. :-/  As such, I'll leave the (binNMU-friendly) packaging as is.

BTW, do you have an ETA for the transition?

--
Aaron M. Ucko, KB1CJC (amu at alum.mit.edu, ucko at debian.org)
http://www.mit.edu/~amu/ | http://stuff.mit.edu/cgi/finger/?amu@...


--
To UNSUBSCRIBE, email to debian-python-REQUEST@...
with a subject of "unsubscribe". Trouble? Contact listmaster@...

[Josselin Mouette]

Le jeudi 11 octobre 2007 à 14:44 -0400, Aaron M. Ucko a écrit :
> Josselin Mouette <joss@...> writes:
>
> > Still, it would be nice to build extensions for all python versions, but
> > the fact they are private makes this process more complicated.
>
> ... particularly given that neither python-support (which BitPim
> currently uses) nor python-central actually supports that case
> AFAICT. :-/  

If anyone has ideas about how to deal with such cases, I'm all open for
implementing them. It would be easy for python-support to maintain a
list of files that should be symbolic links to the version linked
against the current python. The difficulty is about where to install
those files in the packaging process so that dh_pysupport sees them.

--
 .''`.
: :' :      We are debian.org. Lower your prices, surrender your code.
`. `'       We will add your hardware and software distinctiveness to
  `-        our own. Resistance is futile.