Re: [[e-mail]: The future of Zope{2,3} and Plone in Debian and Ubuntu]

[Erik Rose]

I'm sad to see Plone support go, as I have a lot of reservations about  
how Plone is distributed these days.

> The suggested upstream way to install plone, for example, is the  
> unified
> installer. ZTK developers suggest the use of zc.buildout. These
> tools create an isolated environment where it is possible to develop  
> and
> run your software with a very limited interactions with the rest of  
> the
> system.

Buildout is really a development tool and not universally lauded as a  
deployment solution (though it's ubiquitous right now simply because  
it's the only thing that works). It suffers many reliability issues in  
both its design and its execution that make it unsuitable for our  
production environments, and it routinely confounds new users with the  
very build system concept, with its config syntax, and with its opaque  
modes of failure. Its goal of isolation from the base system is also  
both a strength and weakness: at some point, it either has to admit a  
dependency on system libraries (e.g. PIL) or else become a (less  
mature) package management system in its own right. By bundling zipped  
copies of the necessary packages and not exposing buildout's config  
file during installation, Steve McMahon has done an incredible job  
making the Unified Installer approachable and reliable for initial  
installs, but one is still left with raw buildout for updates and  
managing third-party add-ons.

For years, I've enjoyed and admired your packages as a refreshingly  
mature alternative. Leveraging Debian's superior QA and aptitude's  
fail-safety, they have been the most dependable solution for the  
unattended deployments that comprise WebLion's Plone hosting service.  
We will certainly miss your excellent work!

> Zope 2 and Plone are obviously related, so the future of one of the  
> two
> influences the other one.
>
> The main problem for Zope2 is that the current stable upstream branch
> (2.12) still requires pthon2.4.

Actually not; it works in 2.5 and 2.6. 2.4 is unsupported by 2.12,  
though it "should work". http://docs.zope.org/zope2/releases/2.12/WHATSNEW.html#support-for-newer-python-versions

> This is not acceptable in Debian and
> Ubuntu, and Zope 2 is right now the only stopper for the removal of
> python2.4 from both Debian and Ubuntu.
>
> Even worse, the current stable Plone releases requires Zope 2.10,  
> which we
> suppose will never support anything but python2.4 in the foreseeable
> future. The new major upstream branch (Plone 4) is still far from  
> being released, which means
> that the only way to support Plone and Zope 2.x in Debian and Ubuntu  
> is to
> keep python2.4 in the distribution.

Were you aware that we've renumbered the releases and inserted a less  
ambitious Plone 4, which should be in beta by the end of the year? It  
will run on (and require) Zope 2.12. Plone is finally joining the  
modern Python world. :-)

Best,
Erik


--
To UNSUBSCRIBE, email to debian-python-REQUEST@...
with a subject of "unsubscribe". Trouble? Contact listmaster@...

[Fabio Tranchitella]

(filtering out some of the interested mailing lists)

* 2009-06-23 20:19, Erik Rose wrote:
> I'm sad to see Plone support go, as I have a lot of reservations about  
> how Plone is distributed these days.

FWIW, I'm sad too and I share your same reservations about how Plone is
distributed.

> Actually not; it works in 2.5 and 2.6. 2.4 is unsupported by 2.12,  
> though it "should work".
> http://docs.zope.org/zope2/releases/2.12/WHATSNEW.html#support-for-newer-python-versions

My fault, I wanted to write 2.11 (which is the current stable release,
as today). Sorry for the wrong number.

> Were you aware that we've renumbered the releases and inserted a less
> ambitious Plone 4, which should be in beta by the end of the year? It
> will run on (and require) Zope 2.12. Plone is finally joining the  modern
> Python world. :-)

I don't exclude to support Zope 2.x again in Debian and Ubuntu, but I
really think that in this moment dropping the packages is the best
solution: we will finally be able to drop python2.4.

For Plone, after 5 years of maintenance in Debian, I'm sure that *not*
having an official package (eg. included in Debian stable) is the best
option for our users.

--
Fabio Tranchitella <kobold@...>                        .''`.
Proud Debian GNU/Linux developer, admin and user.            : :'  :
                                                             `. `'`
   http://people.debian.org/~kobold/                           `-
_____________________________________________________________________
1024D/7F961564, fpr 5465 6E69 E559 6466 BF3D 9F01 2BF8 EE2B 7F96 1564

[Brian Sutherland-3]

On Tue, Jun 23, 2009 at 01:48:41PM -0400, Erik Rose wrote:
> Buildout is really a development tool and not universally lauded as a  
> deployment solution (though it's ubiquitous right now simply because  
> it's the only thing that works). It suffers many reliability issues in  
> both its design and its execution that make it unsuitable for our  
> production environments

There's also the option of converting a built buildout into a package
for deployment on production environments. I don't use it myself, but
some do use zc.sourcerelease for this:

    http://pypi.python.org/pypi/zc.sourcerelease/

Personally, I prefer to manage Debian repositories and mirror them into
PYPI style indexes of eggs (or a set of buildout versions). Development
buildouts then use the PYPI index:

   http://pypi.python.org/pypi/van.reposync 

--
Brian Sutherland


--
To UNSUBSCRIBE, email to debian-python-REQUEST@...
with a subject of "unsubscribe". Trouble? Contact listmaster@...

[Jonas Meurer]

On 23/06/2009 Fabio Tranchitella wrote:
> > Were you aware that we've renumbered the releases and inserted a less
> > ambitious Plone 4, which should be in beta by the end of the year? It
> > will run on (and require) Zope 2.12. Plone is finally joining the  modern
> > Python world. :-)
>
> I don't exclude to support Zope 2.x again in Debian and Ubuntu, but I
> really think that in this moment dropping the packages is the best
> solution: we will finally be able to drop python2.4.

why not wait for zope2.12 with python2.5/2.6 support, upload that one to
debian/unstable and afterwards file a request for removal for
zope2.10/zope2.11/python2.4? I believe that a zope2.12 release candidate
will be published within the next month, given that a beta2 has already
been published on 27. of may.
That way we would have a zope2 release available in debian/unstable all
the time would.

greetings,
 jonas

[Fabio Tranchitella-2]

Hello,

* 2009-07-02 14:05, Jonas Meurer wrote:
> why not wait for zope2.12 with python2.5/2.6 support, upload that one to
> debian/unstable and afterwards file a request for removal for
> zope2.10/zope2.11/python2.4? I believe that a zope2.12 release candidate
> will be published within the next month, given that a beta2 has already
> been published on 27. of may.  That way we would have a zope2 release
> available in debian/unstable all the time would.

Zope2.12 is a different source/binary package: why can't we just drop it
now, and when the 2.12 release will be out upload it to testing/unstable?

I don't see the point of keeping zope2.10 around just because zope2.12 is
not ready: I really want to avoid releasing a new stable release of Debian
or Ubuntu with zope2.10.

--
Fabio Tranchitella                         http://www.kobold.it
Free Software Developer and Consultant     http://www.tranchitella.it
_____________________________________________________________________
1024D/7F961564, fpr 5465 6E69 E559 6466 BF3D 9F01 2BF8 EE2B 7F96 1564


--
To UNSUBSCRIBE, email to debian-python-REQUEST@...
with a subject of "unsubscribe". Trouble? Contact listmaster@...

[Matthias Klose]

On 02.07.2009 13:05, Jonas Meurer wrote:

> On 23/06/2009 Fabio Tranchitella wrote:
>>> Were you aware that we've renumbered the releases and inserted a less
>>> ambitious Plone 4, which should be in beta by the end of the year? It
>>> will run on (and require) Zope 2.12. Plone is finally joining the  modern
>>> Python world. :-)
>>
>> I don't exclude to support Zope 2.x again in Debian and Ubuntu, but I
>> really think that in this moment dropping the packages is the best
>> solution: we will finally be able to drop python2.4.
>
> why not wait for zope2.12 with python2.5/2.6 support, upload that one to
> debian/unstable and afterwards file a request for removal for
> zope2.10/zope2.11/python2.4? I believe that a zope2.12 release candidate
> will be published within the next month, given that a beta2 has already
> been published on 27. of may.
> That way we would have a zope2 release available in debian/unstable all
> the time would.

The zope2.12 release candidate was released last week. I updated the packaging
in the zope team repository. An upload still requires some work, because some
modules still need to be packaged. These are:

   Acquisition DateTime ExtensionClass
   Persistence RestrictedPython tempstorage zLOG zope.container
   zope.contentprovider zope.contenttype zope.deferredimport zope.formlib
   zope.lifecycleevent zope.pagetemplate zope.processlifetime zope.sendmail
   zope.sequencesort zope.site zope.size zope.structuredtext zope.tal
   zope.tales zope.testbrowser [zope-functional-testing] (UNRELEASED?)
   zope.viewlet zope.app.form zope.app.publication zope.app.publisher
   zope.app.schema

All other zope dependencies are available as modular zope packages in unstable.
Please have a look how these are packaged, and start packaging. As an interim
solution it could be useful to include the still missing modules in the zope2.12
package until all these are packaged.

The good news is that the schooltool project already did package a lot of these,
so you "only" need updates to recent upstream versions, and an update from
python-vanguardistas to python-van.pydeb (Brian might give more help on this).

I know that not the whole zope team is interested in these additional modules,
so I'm CCing the zope2.x uploaders directly to get involved with the packaging.

I do not want to wait with the removal of python2.4 from unstable for too long,
I think a short time without zope2.x in unstable is ok, while having three
python2.x versions is too much. But it looks like zope2.12 based on python2.5 or
python2.6 is doable for squeeze.

   Matthias

[1]
https://edge.launchpad.net/~schooltool-owners/+archive/ppa/+index?field.series_filter=jaunty&batch=200


--
To UNSUBSCRIBE, email to debian-python-REQUEST@...
with a subject of "unsubscribe". Trouble? Contact listmaster@...

[Jonas Meurer]

hey,

On 20/09/2009 Matthias Klose wrote:

> The zope2.12 release candidate was released last week. I updated the
> packaging in the zope team repository. An upload still requires some
> work, because some modules still need to be packaged. These are:
>
>   Acquisition DateTime ExtensionClass
>   Persistence RestrictedPython tempstorage zLOG zope.container
>   zope.contentprovider zope.contenttype zope.deferredimport zope.formlib
>   zope.lifecycleevent zope.pagetemplate zope.processlifetime zope.sendmail
>   zope.sequencesort zope.site zope.size zope.structuredtext zope.tal
>   zope.tales zope.testbrowser [zope-functional-testing] (UNRELEASED?)
>   zope.viewlet zope.app.form zope.app.publication zope.app.publisher
>   zope.app.schema
>
> All other zope dependencies are available as modular zope packages
> in unstable. Please have a look how these are packaged, and start
> packaging. As an interim solution it could be useful to include the
> still missing modules in the zope2.12 package until all these are
> packaged.

if i got it right then packaging the dependencies as seperate packages
isn't an option for zope2.12, we'll have to include them within the
zope2.10 source tarball. the reason for that is, that zope2.12 requires
particular versions of the dependencies, and doesn't build even if minor
versions aren't correct.

the reason why i didn't finish packaging zope2.12 is the new build
system. starting with zope2.12, upstream stops support for makefiles and
tradiditional build system, and requires to use buildout for building
instead. i didn't have time to dig into buildout yet. but as far as i
know we'll have to use something like zc.sourcerelease and a custom
buildout.cfg in order to build zope2.12 with local copies of the
depencency zope/python libraries/modules.

> I know that not the whole zope team is interested in these
> additional modules, so I'm CCing the zope2.x uploaders directly to
> get involved with the packaging.

i removed Cc to private adresses as i guess that anybody interested is
subscribed to one of the lists.

> I do not want to wait with the removal of python2.4 from unstable
> for too long, I think a short time without zope2.x in unstable is
> ok, while having three python2.x versions is too much. But it looks
> like zope2.12 based on python2.5 or python2.6 is doable for squeeze.

i didn't know that packaging zope2.12 is that timeconsuming at the time
that i proposed to wait with removing python2.4 from unstable. so no
objections against removal of pyhton2.4/zope2.10/zope2.11 from my side
any longer.

greetings,
 jonas


--
To UNSUBSCRIBE, email to debian-python-REQUEST@...
with a subject of "unsubscribe". Trouble? Contact listmaster@...

[Matthias Klose]

On 20.09.2009 16:45, Jonas Meurer wrote:
> if i got it right then packaging the dependencies as seperate packages
> isn't an option for zope2.12, we'll have to include them within the
> zope2.10 source tarball. the reason for that is, that zope2.12 requires
> particular versions of the dependencies, and doesn't build even if minor
> versions aren't correct.

this is the usual answer from an upstream with more than 50 dependencies. From
my experience this based on the fact that upstream only wants to test and
certify one configuration, and doesn't take responsibility for anything else. On
the other hand a distribution tries to minimize the duplicate code in its
distribution, and applies patches to packages to make these work. Look at
OpenOffice, eclipse, etc. zope is not different. It's up to you as a packager to
decide what you can maintain, and where you do want to duplicate sources.

>> I do not want to wait with the removal of python2.4 from unstable
>> for too long, I think a short time without zope2.x in unstable is
>> ok, while having three python2.x versions is too much. But it looks
>> like zope2.12 based on python2.5 or python2.6 is doable for squeeze.
>
> i didn't know that packaging zope2.12 is that timeconsuming at the time
> that i proposed to wait with removing python2.4 from unstable. so no
> objections against removal of pyhton2.4/zope2.10/zope2.11 from my side
> any longer.

ok, I'll file a request for removal next week; zope2.x was the last package
absolutely needing python2.4.

   Matthias


--
To UNSUBSCRIBE, email to debian-python-REQUEST@...
with a subject of "unsubscribe". Trouble? Contact listmaster@...

[Jonas Meurer]

hey,

On 20/09/2009 Matthias Klose wrote:

> On 20.09.2009 16:45, Jonas Meurer wrote:
> >if i got it right then packaging the dependencies as seperate packages
> >isn't an option for zope2.12, we'll have to include them within the
> >zope2.10 source tarball. the reason for that is, that zope2.12 requires
> >particular versions of the dependencies, and doesn't build even if minor
> >versions aren't correct.
>
> this is the usual answer from an upstream with more than 50
> dependencies. From my experience this based on the fact that
> upstream only wants to test and certify one configuration, and
> doesn't take responsibility for anything else. On the other hand a
> distribution tries to minimize the duplicate code in its
> distribution, and applies patches to packages to make these work.
> Look at OpenOffice, eclipse, etc. zope is not different. It's up to
> you as a packager to decide what you can maintain, and where you do
> want to duplicate sources.

while i agree with you that duplicated code is bad security wise i fear
that we would have to hack upstream zope2.12 code in order to build with
different versions of the dependencies than requested upstream.
i already discussed that with kobold some weeks ago on irc, and he said
that this is different for zope3, where no particular versions of the
dependencies are required.

i would be ok with seperate source packages for all zope2.12
dependencies in case that this doesn't cause more harm than good. on the
other hand much dependencies are only required for zope2.12 so far, so
it might not be worth the effort to package them as seperate source
package.
maybe we could distribute these only-zope2.12-dependencies within the
zope2.12 tarball, and build-depend on seperate source packages for
dependencies that are useful for zope3 etc. as well.

> >>I do not want to wait with the removal of python2.4 from unstable
> >>for too long, I think a short time without zope2.x in unstable is
> >>ok, while having three python2.x versions is too much. But it looks
> >>like zope2.12 based on python2.5 or python2.6 is doable for squeeze.
> >
> >i didn't know that packaging zope2.12 is that timeconsuming at the time
> >that i proposed to wait with removing python2.4 from unstable. so no
> >objections against removal of pyhton2.4/zope2.10/zope2.11 from my side
> >any longer.
>
> ok, I'll file a request for removal next week; zope2.x was the last
> package absolutely needing python2.4.

great, thanks for your work.

greetings,
 jonas

[Brian Sutherland-3]

On Sun, Sep 20, 2009 at 01:34:24PM +0200, Matthias Klose wrote:

> On 02.07.2009 13:05, Jonas Meurer wrote:
>> why not wait for zope2.12 with python2.5/2.6 support, upload that one to
>> debian/unstable and afterwards file a request for removal for
>> zope2.10/zope2.11/python2.4? I believe that a zope2.12 release candidate
>> will be published within the next month, given that a beta2 has already
>> been published on 27. of may.
>> That way we would have a zope2 release available in debian/unstable all
>> the time would.
>
> The zope2.12 release candidate was released last week. I updated the
> packaging in the zope team repository. An upload still requires some
> work, because some modules still need to be packaged. These are:
>
>   Acquisition DateTime ExtensionClass
>   Persistence RestrictedPython tempstorage zLOG zope.container

Note: Persistence and ExtensionClass (I think) are packaged in python-zodb already

>   zope.contentprovider zope.contenttype zope.deferredimport zope.formlib
>   zope.lifecycleevent zope.pagetemplate zope.processlifetime zope.sendmail
>   zope.sequencesort zope.site zope.size zope.structuredtext zope.tal
>   zope.tales zope.testbrowser [zope-functional-testing] (UNRELEASED?)
>   zope.viewlet zope.app.form zope.app.publication zope.app.publisher
>   zope.app.schema
>
> All other zope dependencies are available as modular zope packages in
> unstable. Please have a look how these are packaged, and start packaging.

The current packaging of the zope.* packages is the way it is because
they are required to be easily backportable.

I'm working on a few van.pydeb plugins for debhelper 7 that'll make the
packaging much more elegant. But we'll only use that for the packages the
zope team maintains after the current release.

However, if you're interested in using this for the any of the above
packages, please send me a private mail.

> As an interim solution it could be useful to include the still missing
> modules in the zope2.12 package until all these are packaged.

+1, especially for things like zope.app.* or zLOG that Zope2 is looking
to remove as dependencies.

> The good news is that the schooltool project already did package a lot of
> these, so you "only" need updates to recent upstream versions, and an
> update from python-vanguardistas to python-van.pydeb (Brian might give
> more help on this).

Honestly, these packages are so simple that it's probably easier to just
remake them by hand.

> I know that not the whole zope team is interested in these additional
> modules, so I'm CCing the zope2.x uploaders directly to get involved with
> the packaging.

Of the list presented, we are only currently interested in
packaging zope.sendmail (See an earlier mail from me). The others have
very difficult dependencies or are semi-deprecated.

--
Brian Sutherland


--
To UNSUBSCRIBE, email to debian-python-REQUEST@...
with a subject of "unsubscribe". Trouble? Contact listmaster@...

[Jonas Meurer]

hey again,

On 20/09/2009 Matthias Klose wrote:
> The zope2.12 release candidate was released last week. I updated the
> packaging in the zope team repository. An upload still requires some
> work, because some modules still need to be packaged. These are:

zope2.12.0 final was released some days ago, and i uploaded packaging
accordingly. unfortunately the zope2.12 packages are far from building
succuessfully ...

>   Acquisition DateTime ExtensionClass
>   Persistence RestrictedPython tempstorage zLOG zope.container
>   zope.contentprovider zope.contenttype zope.deferredimport zope.formlib
>   zope.lifecycleevent zope.pagetemplate zope.processlifetime zope.sendmail
>   zope.sequencesort zope.site zope.size zope.structuredtext zope.tal
>   zope.tales zope.testbrowser [zope-functional-testing] (UNRELEASED?)
>   zope.viewlet zope.app.form zope.app.publication zope.app.publisher
>   zope.app.schema
>
> All other zope dependencies are available as modular zope packages
> in unstable. Please have a look how these are packaged, and start
> packaging. As an interim solution it could be useful to include the
> still missing modules in the zope2.12 package until all these are
> packaged.

as mentioned earlier, the upstream build system (buildout) doesn't
accept other versions than given in versions.cfg for dependencies. this
is different for zope3, but for zope2.12 they seem to keep strict
versioned dependencies. you may be able to build zope2.12 with
dependency versions different from versions.cfg by building in another
way than recommended upstream, but future versions of the dependencies
might break compability, and at that time we're lost and left alone.

thus i would like to build zope2.12 packages the recommended way where
possible. that means to ship python/zope eggs within the zope2.12
package and use download-cache + install-from-cache in buildout.cfg

the current debian/rules from zope2.12 packaging still uses the old and
obsolete autotools build process. this is depreciated and not supported
any longer with the release of zope2.12. i guess it's only included into
the zope2.12 tarballs by mistake and might be removed with a subsequent
release.

thus we need to migrate our debian/rules to build zope2.12 with the help
of buildout, whether we like it or not. unfortunately buildout seems to
be a real pain for building distribution packages. it hardcodes absolute
pathnames into scripts everywhere, etc.

i didn't manage to roll a sufficient buildout.cfg for zope2.12 debian
packages yet, and would really like to discuss this with you before
commiting any further changes to the zope2.12 packaging repository.

following is a list of thoughts and problems i ran into:

- we have to ship copies of dependency eggs within a cache dir inside
  the source tarball.
- this egg-cache can be used by adding download-cache and
  install-from-cache options into buildout.cfg
- all eggs are installed into eggs/ subdir by default. i don't like the
  idea of installing this subdir within the zope2.12 package, but don't
  see another solution either yet. (/usr/lib/zope2.12/python/eggs)?
- absolut pathnames are hardcoded into bin/runzope, bin/zopepy, etc.

finally i get the feeling that building FHS and debian policy compliant
zope2.12 is a real challenge with the new build system. maybe someone
with better python/zope/buildout skills could give advice ;-)

greetings,
 jonas

[Toni Mueller-5]

Hi,

On Sun, 04.10.2009 at 00:00:16 +0200, Jonas Meurer <jonas@...> wrote:
> thus we need to migrate our debian/rules to build zope2.12 with the help
> of buildout, whether we like it or not. unfortunately buildout seems to
> be a real pain for building distribution packages. it hardcodes absolute
> pathnames into scripts everywhere, etc.

I'm not sure, but maybe you'll be able to replace 'buildout' with any
of the incumbent successors, eg 'distribute' and/or eggbasket (no
advertising intended).

Another idea could be to defer the buildout process to the postinst
phase, so that shipping the 'raw' eggs and installing them at postinst
time could "work around", or alleviate, the hardcoded-paths issue.


Just my 0.02 cents - it's now been a while that I've actively looked at
such packaging tools.

> - we have to ship copies of dependency eggs within a cache dir inside
>   the source tarball.

Sounds about as good as it gets to me. Otherwise, offline installation
will not be possible.

> - this egg-cache can be used by adding download-cache and
>   install-from-cache options into buildout.cfg

Right.

> - all eggs are installed into eggs/ subdir by default. i don't like the
>   idea of installing this subdir within the zope2.12 package, but don't
>   see another solution either yet. (/usr/lib/zope2.12/python/eggs)?

The latter sounds better to me, if re-using these eggs should be
possible. I also wonder how to individually upgrade them, in case of a
problem (eg. a CVE).

> - absolut pathnames are hardcoded into bin/runzope, bin/zopepy, etc.

There's imho not much one can do to work around that. For me
personally, I recommend and use appropriate virtualenv-s for such
purposes to avoid cluttering the system. But of course, this completely
bypasses all FHS issues and, at the same time, prevents all kinds of
re-use or other system integration, too.


--
Kind regards,
--Toni++


--
To UNSUBSCRIBE, email to debian-python-REQUEST@...
with a subject of "unsubscribe". Trouble? Contact listmaster@...