grokproject pins setuptools, zc.buildout et.al.

[Jan-Wijbrand Kolman-3]

Hi,

I'm ambivalent about this topic: pinning the versions of zc.buildout and
setuptools and related packages in the version list of a project.

But this is the situation:

A project built with grokproject will have a versions.cfg file that is a
copy of the latests grok "kgs" file found online and in addition will
have list the following packages:

Paste = 1.7.2
PasteDeploy = 1.3.2
PasteScript = 1.7.3
setuptools = 0.6c9
collective.recipe.template = 1.4.0
z3c.evalexception = 2.0
z3c.recipe.dev = 0.5.3
z3c.recipe.i18n = 0.5.3
z3c.recipe.mkdir = 0.3.1
zc.buildout = 1.1.1
zc.lockfile = 1.0.0
zc.recipe.egg = 1.1.0
zc.recipe.filestorage = 1.0.1

Now we have the situation that there's a newer setuptools package
released, that I have installed in my virtualenv (as it fixes for one an
ugly bug related to subversion 1.6 and up). Having installed
setuptools0.6c11 in my virtualenv conflicts during buildout with the
version listed in my newly created project. Ugh.

So there's my "ambivalence": should we really pin versions of
buildout-related infrastructure and even setuptools in projects or not?
If it were not pinned, I would not have had a problem, where the
intention of pinning these packages was to prevent any problems from
happening in the first place.

Opinions?


regards,
jw

_______________________________________________
Grok-dev mailing list
Grok-dev@...
https://mail.zope.org/mailman/listinfo/grok-dev

[Reinout van Rees]

On 2009-11-06, Jan-Wijbrand Kolman <janwijbrand@...> wrote:
>
> So there's my "ambivalence": should we really pin versions of
> buildout-related infrastructure and even setuptools in projects or not?

I don't think it is needed to pin them here.  They don't influence the
run-time behaviour and don't conflict with something else.  (Well, setuptools
could have influence with entry points).

Minimal versions are handy.  Require at least setuptools 0.6c11 to prevent
subversion 1.6 related errors.  (Technical note: distribute has no problem
with such a setuptools minimal dependency: it will just say "yep, that's me"
in the places that matter).


Reinout

--
Reinout van Rees - reinout@... - http://reinout.vanrees.org
Software developer at http://www.thehealthagency.com
"Military engineers build missiles. Civil engineers build targets"

_______________________________________________
Grok-dev mailing list
Grok-dev@...
https://mail.zope.org/mailman/listinfo/grok-dev

[Hanno Schlichting-4]

On Fri, Nov 6, 2009 at 11:45 AM, Jan-Wijbrand Kolman
<janwijbrand@...> wrote:
> I'm ambivalent about this topic: pinning the versions of zc.buildout and
> setuptools and related packages in the version list of a project.

I think the correct solution to this problem is to release a new
bugfix release of Grok (grokproject?), which has an updated setuptools
version in it ;-)

At least in the Plone-world we depend quite heavily on buildout and
have had several times where changes in buildout broke recipes or
where newer versions of recipes only work with newer versions of Plone
itself.

We adopted a strict "pin everything" policy that includes any
packages, even the build infrastructure. This works pretty well for
us.

Just my 2cent :)
Hanno
_______________________________________________
Grok-dev mailing list
Grok-dev@...
https://mail.zope.org/mailman/listinfo/grok-dev

[Maurits van Rees-3]

Hanno Schlichting, on 2009-11-06:

> On Fri, Nov 6, 2009 at 11:45 AM, Jan-Wijbrand Kolman
><janwijbrand@...> wrote:
>> I'm ambivalent about this topic: pinning the versions of zc.buildout and
>> setuptools and related packages in the version list of a project.
>
> I think the correct solution to this problem is to release a new
> bugfix release of Grok (grokproject?), which has an updated setuptools
> version in it ;-)
>
> At least in the Plone-world we depend quite heavily on buildout and
> have had several times where changes in buildout broke recipes or
> where newer versions of recipes only work with newer versions of Plone
> itself.
>
> We adopted a strict "pin everything" policy that includes any
> packages, even the build infrastructure. This works pretty well for
> us.

+1

Minimum versions in setup.py is fine, exact versions not.

Pinning exact versions in a buildout config file still allows anyone
to override them when problems like these occur, so that is still fine.

--
Maurits van Rees | http://maurits.vanrees.org/
            Work | http://zestsoftware.nl/
"This is your day, don't let them take it away." [Barlow Girl]

_______________________________________________
Grok-dev mailing list
Grok-dev@...
https://mail.zope.org/mailman/listinfo/grok-dev

[Sebastien Douche]

On Fri, Nov 6, 2009 at 16:30, Maurits van Rees
<m.van.rees@...> wrote:
>> We adopted a strict "pin everything" policy that includes any
>> packages, even the build infrastructure. This works pretty well for
>> us.
>
> +1
>
> Minimum versions in setup.py is fine, exact versions not.

It could be great to use buildout to pin minimum versions and let
setup.py empty.


--
Sebastien Douche <sdouche@...>
Twitter: http://bit.ly/afkrK (agile, python, open source)
_______________________________________________
Grok-dev mailing list
Grok-dev@...
https://mail.zope.org/mailman/listinfo/grok-dev

[Maurits van Rees-3]

Sebastien Douche, on 2009-11-06:

> On Fri, Nov 6, 2009 at 16:30, Maurits van Rees
><m.van.rees@...> wrote:
>>> We adopted a strict "pin everything" policy that includes any
>>> packages, even the build infrastructure. This works pretty well for
>>> us.
>>
>> +1
>>
>> Minimum versions in setup.py is fine, exact versions not.
>
> It could be great to use buildout to pin minimum versions and let
> setup.py empty.

-1.  That would just mean you allow a user to pick a version that you
know is not going to work.

--
Maurits van Rees | http://maurits.vanrees.org/
            Work | http://zestsoftware.nl/
"This is your day, don't let them take it away." [Barlow Girl]

_______________________________________________
Grok-dev mailing list
Grok-dev@...
https://mail.zope.org/mailman/listinfo/grok-dev

[Roger Erens-2]

On Fri, Nov 6, 2009 at 19:17, Maurits van Rees
<m.van.rees@...> wrote:

> Sebastien Douche, on 2009-11-06:
>> On Fri, Nov 6, 2009 at 16:30, Maurits van Rees
>><m.van.rees@...> wrote:
>>>> We adopted a strict "pin everything" policy that includes any
>>>> packages, even the build infrastructure. This works pretty well for
>>>> us.
>>>
>>> +1
>>>
>>> Minimum versions in setup.py is fine, exact versions not.
>>
>> It could be great to use buildout to pin minimum versions and let
>> setup.py empty.
>
> -1.  That would just mean you allow a user to pick a version that you
> know is not going to work.

I don't understand; can you elaborate? I mean, someone who can edit
buildout.cfg is also able to edit setup.py. Or isn't that the issue?

Roger
_______________________________________________
Grok-dev mailing list
Grok-dev@...
https://mail.zope.org/mailman/listinfo/grok-dev

[Maurits van Rees-3]

Roger Erens, on 2009-11-10:

> On Fri, Nov 6, 2009 at 19:17, Maurits van Rees
><m.van.rees@...> wrote:
>> Sebastien Douche, on 2009-11-06:
>>> On Fri, Nov 6, 2009 at 16:30, Maurits van Rees
>>><m.van.rees@...> wrote:
>>>> Minimum versions in setup.py is fine, exact versions not.
>>>
>>> It could be great to use buildout to pin minimum versions and let
>>> setup.py empty.
>>
>> -1.  That would just mean you allow a user to pick a version that you
>> know is not going to work.
>
> I don't understand; can you elaborate? I mean, someone who can edit
> buildout.cfg is also able to edit setup.py. Or isn't that the issue?

If the setup.py is in your own package, say a package you wrote for a
client: sure.

But here the point is that for example the setup.py in the official
grok egg should *not* pin exact versions (and for the record: it does
not currently).  If we had for example said in the setup.py
"install_requires=['setuptools==0.6c9']" then someone who wanted to
use setuptools 0.6c11 would be *forced* to edit the setup.py of their
local copy of the grok egg and that is not good.

Is that clearer?

Cheers,

--
Maurits van Rees | http://maurits.vanrees.org/
            Work | http://zestsoftware.nl/
"This is your day, don't let them take it away." [Barlow Girl]

_______________________________________________
Grok-dev mailing list
Grok-dev@...
https://mail.zope.org/mailman/listinfo/grok-dev

[Roger Erens-2]

On Tue, Nov 10, 2009 at 12:32, Maurits van Rees <m.van.rees@...> wrote:

> Roger Erens, on 2009-11-10:
>> On Fri, Nov 6, 2009 at 19:17, Maurits van Rees
>><m.van.rees@...> wrote:
>>> Sebastien Douche, on 2009-11-06:
>>>> On Fri, Nov 6, 2009 at 16:30, Maurits van Rees
>>>><m.van.rees@...> wrote:
>>>>> Minimum versions in setup.py is fine, exact versions not.
>>>>
>>>> It could be great to use buildout to pin minimum versions and let
>>>> setup.py empty.
>>>
>>> -1.  That would just mean you allow a user to pick a version that you
>>> know is not going to work.
>>
>> I don't understand; can you elaborate? I mean, someone who can edit
>> buildout.cfg is also able to edit setup.py. Or isn't that the issue?
>
> If the setup.py is in your own package, say a package you wrote for a
> client: sure.
>
> But here the point is that for example the setup.py in the official
> grok egg should *not* pin exact versions (and for the record: it does
> not currently).  If we had for example said in the setup.py
> "install_requires=['setuptools==0.6c9']" then someone who wanted to
> use setuptools 0.6c11 would be *forced* to edit the setup.py of their
> local copy of the grok egg and that is not good.
>
> Is that clearer?

Yes, thanks. I had never given the setup.pys in eggs any thought. But this is just about the point w.r.t. the exact versions vs. the minimum versions in setup.py; how about the point w.r.t. empty vs. non-empty setup.py?

Cheers,

Roger


_______________________________________________
Grok-dev mailing list
Grok-dev@...
https://mail.zope.org/mailman/listinfo/grok-dev

[Roger Erens-2]

On Tue, Nov 10, 2009 at 12:32, Maurits van Rees <m.van.rees@...> wrote:

> Roger Erens, on 2009-11-10:
>> On Fri, Nov 6, 2009 at 19:17, Maurits van Rees
>><m.van.rees@...> wrote:
>>> Sebastien Douche, on 2009-11-06:
>>>> On Fri, Nov 6, 2009 at 16:30, Maurits van Rees
>>>><m.van.rees@...> wrote:
>>>>> Minimum versions in setup.py is fine, exact versions not.
>>>>
>>>> It could be great to use buildout to pin minimum versions and let
>>>> setup.py empty.
>>>
>>> -1.  That would just mean you allow a user to pick a version that you
>>> know is not going to work.
>>
>> I don't understand; can you elaborate? I mean, someone who can edit
>> buildout.cfg is also able to edit setup.py. Or isn't that the issue?
>
> If the setup.py is in your own package, say a package you wrote for a
> client: sure.
>
> But here the point is that for example the setup.py in the official
> grok egg should *not* pin exact versions (and for the record: it does
> not currently).  If we had for example said in the setup.py
> "install_requires=['setuptools==0.6c9']" then someone who wanted to
> use setuptools 0.6c11 would be *forced* to edit the setup.py of their
> local copy of the grok egg and that is not good.
>
> Is that clearer?

Yes, thanks. I had never given the setup.pys in eggs any thought. But this is just about the point w.r.t. the exact versions vs. the minimum versions in setup.py; how about the point w.r.t. empty vs. non-empty setup.py?

Cheers,

Roger


_______________________________________________
Grok-dev mailing list
Grok-dev@...
https://mail.zope.org/mailman/listinfo/grok-dev

[Maurits van Rees-3]

Roger Erens, on 2009-11-10:

>
> On Tue, Nov 10, 2009 at 12:32, Maurits van Rees wrote:
>> Roger Erens, on 2009-11-10:
>>> I don't understand; can you elaborate? I mean, someone who can edit
>>> buildout.cfg is also able to edit setup.py. Or isn't that the issue?
>>
>> If the setup.py is in your own package, say a package you wrote for a
>> client: sure.
>>
>> But here the point is that for example the setup.py in the official
>> grok egg should *not* pin exact versions (and for the record: it does
>> not currently).  If we had for example said in the setup.py
>> "install_requires=['setuptools==0.6c9']" then someone who wanted to
>> use setuptools 0.6c11 would be *forced* to edit the setup.py of their
>> local copy of the grok egg and that is not good.
>>
>> Is that clearer?
>
> Yes, thanks. I had never given the setup.pys in eggs any
> thought. But this is just about the point w.r.t. the exact versions
> vs. the minimum versions in setup.py; how about the point
> w.r.t. empty vs. non-empty setup.py?

If you have a dependency you should specify it, otherwise your package
will not work.  You cannot depend on users of your package to read the
pypi page of your package and add any dependencies themselved manually
to their buildout.cfg.

I encountered that in an own package recently.
Products.eXtremeManagement (project management tool in Plone) depended
on xm.tracker in its setup.py.  xm.tracker needed egenix-mx-base (for
the mx.DateTime module) but did not specify this in its dependencies.
The reason was that egenix-mx-base was not on pypi at the time so the
dependency could not be resolved by buildout anyway.  This was
described in the readme and on pypi, but someone did not read that and
the missing dependency caused errors on his plone site.

There is some middle ground of course.  For example at Zest Sofware if
we build a package for a client and it needs a third party product we
usually do not specify this in the setup.py but only in the
buildout.cfg.  The package is not going to be reused anyway.  This way
the information is in one place.

And if you are in Plone land and you have a package that needs Plone
and works on Plone 3.0 and higher, you cannot specify 'Plone>=3.0' as
Plone is only eggified since 3.2.  So if you put this in your setup.py
you make it hard for people on Plone 3.0 and 3.1 to use your package.
(Some fake-eggs might help but I have not tried.)

--
Maurits van Rees | http://maurits.vanrees.org/
            Work | http://zestsoftware.nl/
"This is your day, don't let them take it away." [Barlow Girl]

_______________________________________________
Grok-dev mailing list
Grok-dev@...
https://mail.zope.org/mailman/listinfo/grok-dev

[Reinout van Rees]

On 2009-11-10, Roger Erens <roger.erens@...> wrote:
> This is an OpenPGP/MIME signed message (RFC 2440 and 3156)
> --===============1886057361675339730==
> Content-Type: multipart/signed; micalg=pgp-sha1;

Off topic:

I could not read both of your messages.  They'll probably work fine in
thunderbird (as that's got great gpg support), but my "slrn" text-mode news
reader only displayed some base64.

I don't whether it works fine on nabble/gmane/googlegroups whatever.  If it is
just me just ignore me, but it warrants a quick check, I guess.



Reinout

--
Reinout van Rees - reinout@... - http://reinout.vanrees.org
Software developer at http://www.thehealthagency.com
"Military engineers build missiles. Civil engineers build targets"

_______________________________________________
Grok-dev mailing list
Grok-dev@...
https://mail.zope.org/mailman/listinfo/grok-dev

[Maurits van Rees-3]

Reinout van Rees, on 2009-11-11:

> On 2009-11-10, Roger Erens <roger.erens@...> wrote:
>> This is an OpenPGP/MIME signed message (RFC 2440 and 3156)
>> --===============1886057361675339730==
>> Content-Type: multipart/signed; micalg=pgp-sha1;
>
> Off topic:
>
> I could not read both of your messages.  They'll probably work fine in
> thunderbird (as that's got great gpg support), but my "slrn" text-mode news
> reader only displayed some base64.
>
> I don't whether it works fine on nabble/gmane/googlegroups whatever.  If it is
> just me just ignore me, but it warrants a quick check, I guess.

I had these problems too with several people a while ago.  Using a
newer version helped for me:

http://www.foory.de/thw/slrn/

--
Maurits van Rees | http://maurits.vanrees.org/
            Work | http://zestsoftware.nl/
"This is your day, don't let them take it away." [Barlow Girl]

_______________________________________________
Grok-dev mailing list
Grok-dev@...
https://mail.zope.org/mailman/listinfo/grok-dev

[Martijn Faassen-2]

Hanno Schlichting wrote:
> On Fri, Nov 6, 2009 at 11:45 AM, Jan-Wijbrand Kolman
> <janwijbrand@...> wrote:
>> I'm ambivalent about this topic: pinning the versions of zc.buildout and
>> setuptools and related packages in the version list of a project.
>
> I think the correct solution to this problem is to release a new
> bugfix release of Grok (grokproject?), which has an updated setuptools
> version in it ;-)

I think this is a good point. I prefer also to err on the side of
pinning too much, myself.

JW, would it be possible to release a new version of grokproject?

Regards,

Martijn

_______________________________________________
Grok-dev mailing list
Grok-dev@...
https://mail.zope.org/mailman/listinfo/grok-dev