People want CPAN :-)

[Guido van Rossum]

I just found this comment on my blog. People have told me this in
person too, so I believe it is real pain (even if the solution may be
elusive and the suggested solutions may not work). But I don't know
how to improve the world. Is the work on distutils-sig going to be
enough? Or do we need some other kind of work in addition? Do we need
more than PyPI?

--Guido

---------- Forwarded message ----------
From: dalloliogm <noreply-comment@...>
Date: Fri, Nov 6, 2009 at 8:01 AM
Subject: [Neopythonic] New comment on Python in the Scientific World.
To: gvanrossum@...


dalloliogm has left a new comment on your post "Python in the Scientific World":

Python is suffering a lot in the scientific word, because it has not a
CPAN-like repository.

PyPI is fine, but it is still far from the level of CPAN, CRAN,
Bioconductor, etc..

Scientists who use programming usually have a lot of different
interests and approaches, therefore it is really difficult to write a
package that can be useful to everyone.
Other programming language like Perl and R have repository-like
structure which enable people to download packages easily, and to
upload new ones and organize them withouth having to worry about
having to integrate them into existing packages.

This is what is happening to biopython now: it is a monolitic package
that it is supposed to work for any bioinformatic problem; but this is
so general that to accomplish that you would need to add a lot of
dependencies, to numpy, networkx, suds, any kind of library.
However, since easy_install is not as ready yet as the counterparts in
other languages, if the biopython developers add too many
dependencies, nobody will be able to install it properly, and nobody
will use it.



Posted by dalloliogm to Neopythonic at November 6, 2009 8:01 AM


--
--Guido van Rossum (python.org/~guido)
_______________________________________________
Distutils-SIG maillist  -  Distutils-SIG@...
http://mail.python.org/mailman/listinfo/distutils-sig

[Alex Grönholm-3]

Guido van Rossum kirjoitti:

> I just found this comment on my blog. People have told me this in
> person too, so I believe it is real pain (even if the solution may be
> elusive and the suggested solutions may not work). But I don't know
> how to improve the world. Is the work on distutils-sig going to be
> enough? Or do we need some other kind of work in addition? Do we need
> more than PyPI?
>
> --Guido
>
> ---------- Forwarded message ----------
> From: dalloliogm <noreply-comment@...>
> Date: Fri, Nov 6, 2009 at 8:01 AM
> Subject: [Neopythonic] New comment on Python in the Scientific World.
> To: gvanrossum@...
>
>
> dalloliogm has left a new comment on your post "Python in the Scientific World":
>
> Python is suffering a lot in the scientific word, because it has not a
> CPAN-like repository.
>
> PyPI is fine, but it is still far from the level of CPAN, CRAN,
> Bioconductor, etc..
>
> Scientists who use programming usually have a lot of different
> interests and approaches, therefore it is really difficult to write a
> package that can be useful to everyone.
> Other programming language like Perl and R have repository-like
> structure which enable people to download packages easily, and to
> upload new ones and organize them withouth having to worry about
> having to integrate them into existing packages.
>
> This is what is happening to biopython now: it is a monolitic package
> that it is supposed to work for any bioinformatic problem; but this is
> so general that to accomplish that you would need to add a lot of
> dependencies, to numpy, networkx, suds, any kind of library.
> However, since easy_install is not as ready yet as the counterparts in
> other languages, if the biopython developers add too many
> dependencies, nobody will be able to install it properly, and nobody
> will use it.
>  

I for one did not understand the problem. What does CPAN have that PyPI
doesn't?
It is natural for packages (distributions, in distutils terms) to have
dependencies on each other. Why is this a problem?
>
>
> Posted by dalloliogm to Neopythonic at November 6, 2009 8:01 AM
>
>
>  

_______________________________________________
Distutils-SIG maillist  -  Distutils-SIG@...
http://mail.python.org/mailman/listinfo/distutils-sig

[David Lyon]

On Fri, 6 Nov 2009 09:53:44 -0800, Guido van Rossum <guido@...>
wrote:
> I just found this comment on my blog. People have told me this in
> person too, so I believe it is real pain (even if the solution may be
> elusive and the suggested solutions may not work).

Hi Guido,

I'm a mere application developer - developing things for companies as
they require. Sometimes in perl more often in python.

I've worked in Perl with CPAN and yes I can attest to how good CPAN is
to the application developer. Anything you need is there. It's easy to
pull down stuff and load it in.

CPan have an 'over-the-top' philosophy towards developers. It works.

> Is the work on distutils-sig going to be enough?

It's not on the roadmap circulated thus far - but to answer the
question probably not.

At the moment, we don't have our target set towards working towards
that.

CPAN actually does so much, things that we could do, but are not.

For example, if Fred the developer makes a package 'microscope-
calibrate', and uploads, it will get tested and analysed on
CPAN with a buildbot. We don't do that yet.

So the packages on CPAN are typically of a higher quality, simply
because they've been machine checked. I like that.

Then, one the client machine, packages are so easy to install.

We don't have anything quite like that with Python yet but PJE
did get us one third of the way with setuptools. The skills to
get totally up with CPAN in this regard are there collectively
with us now. Not in any one individual, but as a group. No one
individual can do it - but one individual can say - 'make it so'

> Or do we need some other kind of work in addition? Do we need
> more than PyPI?

Yes we do.

The Perl/CPAN developer experience is so smooth.

May I say that the current pypi website is 100% as good as
cpan. As a developer, I can't find any significant fault with
pypi. Apart from not having a buildbot running in the background
for package smoke testing.

It's just when a developer goes to download and install a
package that things get complicated with python.

In an ideal world, a developer would just like to get eggs
or packages off pypi, install them and go back to work. It
would be fantastic for us to forget about what platform we
have and forget about what python version we have. CPAN
does that.

The totally best thing would be to see a package on pypi,
click on it and have it installed automatically there and
then.

The technical framework to do all this exists or is in
close to working order. Some things need doing but I would
regard the work as bridging rather than doing things from
scratch.

To me, the things Tarek is doing seem pretty sound and
sane.

We just need to have a 'developer experience' program
or something along those lines to connect the dots and
make things run more smoothly.

Then having something as good as CPAN is totally realistic.

David



_______________________________________________
Distutils-SIG maillist  -  Distutils-SIG@...
http://mail.python.org/mailman/listinfo/distutils-sig

[exarkun]

On 10:14 pm, alex.gronholm@... wrote:

>Guido van Rossum kirjoitti:
>>I just found this comment on my blog. People have told me this in
>>person too, so I believe it is real pain (even if the solution may be
>>elusive and the suggested solutions may not work). But I don't know
>>how to improve the world. Is the work on distutils-sig going to be
>>enough? Or do we need some other kind of work in addition? Do we need
>>more than PyPI?
>>
>>--Guido
>>
>>---------- Forwarded message ----------
>>From: dalloliogm <noreply-comment@...>
>>Date: Fri, Nov 6, 2009 at 8:01 AM
>>Subject: [Neopythonic] New comment on Python in the Scientific World.
>>To: gvanrossum@...
>>
>>
>>dalloliogm has left a new comment on your post "Python in the
>>Scientific World":
>>
>>Python is suffering a lot in the scientific word, because it has not a
>>CPAN-like repository.
>>
>>PyPI is fine, but it is still far from the level of CPAN, CRAN,
>>Bioconductor, etc..
>>
>>Scientists who use programming usually have a lot of different
>>interests and approaches, therefore it is really difficult to write a
>>package that can be useful to everyone.
>>Other programming language like Perl and R have repository-like
>>structure which enable people to download packages easily, and to
>>upload new ones and organize them withouth having to worry about
>>having to integrate them into existing packages.
>>
>>This is what is happening to biopython now: it is a monolitic package
>>that it is supposed to work for any bioinformatic problem; but this is
>>so general that to accomplish that you would need to add a lot of
>>dependencies, to numpy, networkx, suds, any kind of library.
>>However, since easy_install is not as ready yet as the counterparts in
>>other languages, if the biopython developers add too many
>>dependencies, nobody will be able to install it properly, and nobody
>>will use it.
>I for one did not understand the problem. What does CPAN have that PyPI
>doesn't?
>It is natural for packages (distributions, in distutils terms) to have
>dependencies on each other. Why is this a problem?

I'm also not sure I see what problem CPAN is solving that PyPI is
failing at.  At most, it sounds like the OP is complaining that the
software available to scientists in perl is better than the software
available to scientists in python (for some definition of better -
there's more of it, it solves their particular problems better,
whatever).

PyPI *does* let you download packages easily, it does let you upload new
ones and organize them without having to worry about integrating them
into existing packages.  So the features of CPAN that are described as
desirable are already available in PyPI.

Jean-Paul
_______________________________________________
Distutils-SIG maillist  -  Distutils-SIG@...
http://mail.python.org/mailman/listinfo/distutils-sig

[Bob Ippolito]

On Fri, Nov 6, 2009 at 2:53 PM,  <exarkun@...> wrote:

> On 10:14 pm, alex.gronholm@... wrote:
>>
>> Guido van Rossum kirjoitti:
>>>
>>> I just found this comment on my blog. People have told me this in
>>> person too, so I believe it is real pain (even if the solution may be
>>> elusive and the suggested solutions may not work). But I don't know
>>> how to improve the world. Is the work on distutils-sig going to be
>>> enough? Or do we need some other kind of work in addition? Do we need
>>> more than PyPI?
>>>
>>> --Guido
>>>
>>> ---------- Forwarded message ----------
>>> From: dalloliogm <noreply-comment@...>
>>> Date: Fri, Nov 6, 2009 at 8:01 AM
>>> Subject: [Neopythonic] New comment on Python in the Scientific World.
>>> To: gvanrossum@...
>>>
>>>
>>> dalloliogm has left a new comment on your post "Python in the Scientific
>>> World":
>>>
>>> Python is suffering a lot in the scientific word, because it has not a
>>> CPAN-like repository.
>>>
>>> PyPI is fine, but it is still far from the level of CPAN, CRAN,
>>> Bioconductor, etc..
>>>
>>> Scientists who use programming usually have a lot of different
>>> interests and approaches, therefore it is really difficult to write a
>>> package that can be useful to everyone.
>>> Other programming language like Perl and R have repository-like
>>> structure which enable people to download packages easily, and to
>>> upload new ones and organize them withouth having to worry about
>>> having to integrate them into existing packages.
>>>
>>> This is what is happening to biopython now: it is a monolitic package
>>> that it is supposed to work for any bioinformatic problem; but this is
>>> so general that to accomplish that you would need to add a lot of
>>> dependencies, to numpy, networkx, suds, any kind of library.
>>> However, since easy_install is not as ready yet as the counterparts in
>>> other languages, if the biopython developers add too many
>>> dependencies, nobody will be able to install it properly, and nobody
>>> will use it.
>>
>> I for one did not understand the problem. What does CPAN have that PyPI
>> doesn't?
>> It is natural for packages (distributions, in distutils terms) to have
>> dependencies on each other. Why is this a problem?
>
> I'm also not sure I see what problem CPAN is solving that PyPI is failing
> at.  At most, it sounds like the OP is complaining that the software
> available to scientists in perl is better than the software available to
> scientists in python (for some definition of better - there's more of it, it
> solves their particular problems better, whatever).
>
> PyPI *does* let you download packages easily, it does let you upload new
> ones and organize them without having to worry about integrating them into
> existing packages.  So the features of CPAN that are described as desirable
> are already available in PyPI.

It might be the CPAN user interface that makes the difference. Using
CPAN you can search for and install applications from an interactive
prompt. No web browser needed, just one command line invocation (perl
-MCPAN -e shell).

-bob
_______________________________________________
Distutils-SIG maillist  -  Distutils-SIG@...
http://mail.python.org/mailman/listinfo/distutils-sig

[David Lyon]

On Fri, 6 Nov 2009 14:56:51 -0800, Bob Ippolito <bob@...> wrote:

> It might be the CPAN user interface that makes the difference. Using
> CPAN you can search for and install applications from an interactive
> prompt. No web browser needed, just one command line invocation (perl
> -MCPAN -e shell).

That's definitely one thing. We have nothing like that yet.

David
_______________________________________________
Distutils-SIG maillist  -  Distutils-SIG@...
http://mail.python.org/mailman/listinfo/distutils-sig

[Kaelin Colclasure]

On Nov 6, 2009, at 2:14 PM, Alex Grönholm wrote:

> Guido van Rossum kirjoitti:
>> I just found this comment on my blog. People have told me this in
>> person too, so I believe it is real pain (even if the solution may be
>> elusive and the suggested solutions may not work). But I don't know
>> how to improve the world. Is the work on distutils-sig going to be
>> enough? Or do we need some other kind of work in addition? Do we need
>> more than PyPI?
>>
>> --Guido
>>
>> ---------- Forwarded message ----------
>> From: dalloliogm <noreply-comment@...>
>> Date: Fri, Nov 6, 2009 at 8:01 AM
>> Subject: [Neopythonic] New comment on Python in the Scientific World.
>> To: gvanrossum@...
>>
>>
>> dalloliogm has left a new comment on your post "Python in the  
>> Scientific World":
>>
>> Python is suffering a lot in the scientific word, because it has  
>> not a
>> CPAN-like repository.
>>
>> PyPI is fine, but it is still far from the level of CPAN, CRAN,
>> Bioconductor, etc..
>>
>> Scientists who use programming usually have a lot of different
>> interests and approaches, therefore it is really difficult to write a
>> package that can be useful to everyone.
>> Other programming language like Perl and R have repository-like
>> structure which enable people to download packages easily, and to
>> upload new ones and organize them withouth having to worry about
>> having to integrate them into existing packages.
>>
>> This is what is happening to biopython now: it is a monolitic package
>> that it is supposed to work for any bioinformatic problem; but this  
>> is
>> so general that to accomplish that you would need to add a lot of
>> dependencies, to numpy, networkx, suds, any kind of library.
>> However, since easy_install is not as ready yet as the counterparts  
>> in
>> other languages, if the biopython developers add too many
>> dependencies, nobody will be able to install it properly, and nobody
>> will use it.
>>
> I for one did not understand the problem. What does CPAN have that  
> PyPI doesn't?
> It is natural for packages (distributions, in distutils terms) to  
> have dependencies on each other. Why is this a problem?

As both a CPAN contributor and a recent PyPI contributor I think I can  
speak to this:

When I packaged my first Perl module for CPAN distribution, there was  
no question about how to go about it. There was a clear recipe and it  
worked without a hitch the first time I tried.

When I recently packaged my first Python module for PyPI I was a bit  
dismayed to learn that easy_install is just one of several ways to  
disseminate Python code. The documentation is fragmented... The stuff  
I read on easy_install basically assumed you already knew all about  
distutils, and I eventually had to reorganize my code from a simple  
module to a package just to get a few non-code files to be pulled in  
by the install. A less persistent would-be contributor might have  
given up in frustration.

I joined this mailing list instead, and got my package working with  
some kind assistance. :-) But CPAN was just easier to contribute to.

-- Kaelin

>>
>>
>> Posted by dalloliogm to Neopythonic at November 6, 2009 8:01 AM
>>
>>
>>
>
> _______________________________________________
> Distutils-SIG maillist  -  Distutils-SIG@...
> http://mail.python.org/mailman/listinfo/distutils-sig

_______________________________________________
Distutils-SIG maillist  -  Distutils-SIG@...
http://mail.python.org/mailman/listinfo/distutils-sig

[Guido van Rossum]

On Fri, Nov 6, 2009 at 2:52 PM, David Lyon <david.lyon@...> wrote:
> So the packages on CPAN are typically of a higher quality, simply
> because they've been machine checked. I like that.

Speaking purely on hearsay, I don't believe that. In fact, I've heard
plenty of laments about the complete lack of quality control on CPAN.

--
--Guido van Rossum (python.org/~guido)
_______________________________________________
Distutils-SIG maillist  -  Distutils-SIG@...
http://mail.python.org/mailman/listinfo/distutils-sig

[Tarek Ziadé]

On Fri, Nov 6, 2009 at 6:53 PM, Guido van Rossum <guido@...> wrote:
> I just found this comment on my blog. People have told me this in
> person too, so I believe it is real pain (even if the solution may be
> elusive and the suggested solutions may not work). But I don't know
> how to improve the world. Is the work on distutils-sig going to be
> enough? Or do we need some other kind of work in addition? Do we need
> more than PyPI?

I don't think PyPI has a structural problem. I do think the biggest
issue is related to the fact that it's not really possible to
describe a distribution and its dependencies today with a plain
Distutils (==stdlib). They can use Setuptools or Distribute, third
party tools. But then, there are using different installation
standards.

Changes are being done in PEP 345 + PEP 376 to address this. The goal
is to make sure Distutils provides what is required to describe
dependencies, a standard installation format, and a way to query
installed distributions.

Now, there's one thing CPAN has and PyPI don't: a network of package
repositories and an easy way to set your own
repository to serve distributions. IOW, one can sets its own CPAN
server, or add its server to an existing network.

That's useful for example if you want to have your own private PyPI
server and combine it with the public PyPI.

In PyPI, everything is centralized, and while this has some benefits,
it also makes it a single point of failure (SPF)
when you want to download distributions. We have suffered for that
problem in some sub communities
like Plone, that gets hundreds of packages at PyPI to get built. If
PyPI is down, there's nothing we can do unless we
have a local cache of packages. CPAN is way better in that field.

One work has been started to try to fix this SPF: a mirroring
standard, see PEP 381 - http://www.python.org/dev/peps/pep-0381/

In parallel, some server-side softwares are now allowing people to
register and upload packages using Distutils, like
what they would do with PyPI. For instance, you can use the "register"
and "upload" command to push
your distribution at plone.org That was made possible after some
changes to the .pypirc configuration file
(which is now allowing people to define several servers)

The next step is to add fail-over capabilities to tools like Pip or
Distribute, and ways to merge several sources of
distributions repositories: pypi and any other server that implements
the same protocol.

Tarek
_______________________________________________
Distutils-SIG maillist  -  Distutils-SIG@...
http://mail.python.org/mailman/listinfo/distutils-sig

[Tarek Ziadé]

On Fri, Nov 6, 2009 at 11:53 PM,  <exarkun@...> wrote:
[..]
>
> PyPI *does* let you download packages easily, it does let you upload new
> ones and organize them without having to worry about integrating them into
> existing packages.  So the features of CPAN that are described as desirable
> are already available in PyPI.

imho,

PyPI + PEP 381 + PEP 381 compliant clients == CPAN

Tarek
_______________________________________________
Distutils-SIG maillist  -  Distutils-SIG@...
http://mail.python.org/mailman/listinfo/distutils-sig

[Tarek Ziadé]

On Fri, Nov 6, 2009 at 11:56 PM, Bob Ippolito <bob@...> wrote:
[...]
>
> It might be the CPAN user interface that makes the difference. Using
> CPAN you can search for and install applications from an interactive
> prompt. No web browser needed, just one command line invocation (perl
> -MCPAN -e shell).

Although, we have an XML-RPC service for  PyPI, and Yolk for example
is a client that will let you query PyPI
through a prompt. I agree it could be enhanced, but it exists. Maybe
such a client could be added in Distutils ?

Plus, once the work on PEP 390 (or an equivalent one) is over, we will
be able to list a distribution's dependencies
on a given target system just by querying PYPI.

Tarek
_______________________________________________
Distutils-SIG maillist  -  Distutils-SIG@...
http://mail.python.org/mailman/listinfo/distutils-sig

[ianb]

On Fri, Nov 6, 2009 at 5:01 PM, David Lyon <david.lyon@...> wrote:
> On Fri, 6 Nov 2009 14:56:51 -0800, Bob Ippolito <bob@...> wrote:
>> It might be the CPAN user interface that makes the difference. Using
>> CPAN you can search for and install applications from an interactive
>> prompt. No web browser needed, just one command line invocation (perl
>> -MCPAN -e shell).
>
> That's definitely one thing. We have nothing like that yet.

My understanding is that they have all the same problems we do --
version management, dependencies, isolation, varying quality, etc.
When I've tried to install things from CPAN it's been surprisingly
obtuse.  This is not to say there aren't good things, but "we need
CPAN" doesn't really get to the heart of it.

>From (server-side) development perspective, CPAN seems to be a less
monolithic environment.  The core of it (at least from what Perl
people have told me) is really a very simple mirrored file repository,
with loose naming conventions.  Everything else is built up as added,
optional, decentralized services from that.  (I wonder if they are
using mirroring as a way to distribute these services?  E.g., register
your I-test-everything mirror, and it'll get packages uploaded to it
automatically.)

We have some of the same things.  Cheesecake is one example, and I
believe borrowed ideas and terminology from CPAN's equivalent service.
 Cheesecake, as an example, seems to be largely forgotten and unused.
It might be useful to unforget that product.

There are also places where (ironically?) Perl has One Way To Do It,
and benefits as a result.  Documentation of course, though maybe now
with Sphinx  we can start doing vertical integration using that tool.
Testing is a bit off; "setup.py test" hasn't caught on that well, and
I believe it is more prescriptive than Perl's system (which is really
just a script that outputs results in a specific format, while
setup.py test is based on unittest).  I think we'd do well to do
*less* vertical integration for testing (though maybe it's fine,
things like the nose entry point split the difference).

In terms of how a setup.py should be structured, and other developer
packaging issues, I think we're getting closer but we *really* need
someone to document the conventions and lessons that are usually just
spread through copy-and-paste and developer feedback.  (In some
communities paster create as had some positive influence, as a kind of
documentation-through-code, but it's adoption is somewhat coincidental
and goes along social lines.)

In terms specifically of the CPAN interactive shell, I don't think
there's any reason to duplicate it, as no one else makes interactive
shell installers; I think it's an accident of history.

--
Ian Bicking  |  http://blog.ianbicking.org  |  http://topplabs.org/civichacker
_______________________________________________
Distutils-SIG maillist  -  Distutils-SIG@...
http://mail.python.org/mailman/listinfo/distutils-sig

[Tarek Ziadé]

2009/11/7 Kaelin Colclasure <kaelin@...>:
[...]

> As both a CPAN contributor and a recent PyPI contributor I think I can speak
> to this:
>
> When I packaged my first Perl module for CPAN distribution, there was no
> question about how to go about it. There was a clear recipe and it worked
> without a hitch the first time I tried.
>
> When I recently packaged my first Python module for PyPI I was a bit
> dismayed to learn that easy_install is just one of several ways to
> disseminate Python code. The documentation is fragmented... The stuff I read
> on easy_install basically assumed you already knew all about distutils, and
> I eventually had to reorganize my code from a simple module to a package
> just to get a few non-code files to be pulled in by the install. A less
> persistent would-be contributor might have given up in frustration.
>
> I joined this mailing list instead, and got my package working with some
> kind assistance. :-) But CPAN was just easier to contribute to.

What was in particular, the pieces missing in Distutils (doc+code)
that made you use Setuptools (doc+code) ?

++
Tarek
_______________________________________________
Distutils-SIG maillist  -  Distutils-SIG@...
http://mail.python.org/mailman/listinfo/distutils-sig

[ianb]

On Fri, Nov 6, 2009 at 5:50 PM, Tarek Ziadé <ziade.tarek@...> wrote:

> On Fri, Nov 6, 2009 at 11:56 PM, Bob Ippolito <bob@...> wrote:
> [...]
>>
>> It might be the CPAN user interface that makes the difference. Using
>> CPAN you can search for and install applications from an interactive
>> prompt. No web browser needed, just one command line invocation (perl
>> -MCPAN -e shell).
>
> Although, we have an XML-RPC service for  PyPI, and Yolk for example
> is a client that will let you query PyPI
> through a prompt. I agree it could be enhanced, but it exists. Maybe
> such a client could be added in Distutils ?
>
> Plus, once the work on PEP 390 (or an equivalent one) is over, we will
> be able to list a distribution's dependencies
> on a given target system just by querying PYPI.

It's quite possible that pip will grow some of these features.  At
least search and querying the currently installed packages are planned
in some fashion (someone has indicated the intent to do both these, I
believe).

--
Ian Bicking  |  http://blog.ianbicking.org  |  http://topplabs.org/civichacker
_______________________________________________
Distutils-SIG maillist  -  Distutils-SIG@...
http://mail.python.org/mailman/listinfo/distutils-sig

[Kaelin Colclasure]

On Nov 6, 2009, at 3:54 PM, Tarek Ziadé wrote:

> 2009/11/7 Kaelin Colclasure <kaelin@...>:
> [...]
>> As both a CPAN contributor and a recent PyPI contributor I think I  
>> can speak
>> to this:
>>
>> When I packaged my first Perl module for CPAN distribution, there  
>> was no
>> question about how to go about it. There was a clear recipe and it  
>> worked
>> without a hitch the first time I tried.
>>
>> When I recently packaged my first Python module for PyPI I was a bit
>> dismayed to learn that easy_install is just one of several ways to
>> disseminate Python code. The documentation is fragmented... The  
>> stuff I read
>> on easy_install basically assumed you already knew all about  
>> distutils, and
>> I eventually had to reorganize my code from a simple module to a  
>> package
>> just to get a few non-code files to be pulled in by the install. A  
>> less
>> persistent would-be contributor might have given up in frustration.
>>
>> I joined this mailing list instead, and got my package working with  
>> some
>> kind assistance. :-) But CPAN was just easier to contribute to.
>
> What was in particular, the pieces missing in Distutils (doc+code)
> that made you use Setuptools (doc+code) ?

Since I bootstrapped the environment I used to learn Python with  
easy_install, I naturally went straight to the easy_install docs to  
learn how to give back. I wasn't even aware of this separate thing  
called distutils until I read about it in the easy_install (err  
setuptools) documentation (sic).

-- Kaelin

_______________________________________________
Distutils-SIG maillist  -  Distutils-SIG@...
http://mail.python.org/mailman/listinfo/distutils-sig

[David Lyon]

On Fri, 6 Nov 2009 19:15:52 -0500, "ssteinerX@..."
<ssteinerx@...> wrote:
>> That's definitely one thing. We have nothing like that yet.
>
http://bitbucket.org/tarek/distribute/issue/88/create-an-equivalent-of-perl-mcpan-e-shell-for

So what... :-)

I have my own shell to too....

But my shell is built into a standard setup.py. It has less command line
parameters to
remember than perl. So it's even better than perl.

http://bitbucket.org/djlyon/original-setup.py-with-metadata/src/tip/setup.py

C:\>python setup.py
setup - WARNING - Configuration file setup.conf not found
Usage: setup.py [options]

Options:
  -h, --help            show this help message and exit
  -i NEW_PACKAGE, --install=NEW_PACKAGE
                        Install a Package/Application
  -r PACKAGE_TO_REMOVE, --remove=PACKAGE_TO_REMOVE
                        Install a Package/Application
  -l, --list            List installed Packages
  -d, --debug           Show debug messages
  -u, --user            Use the User Package Directory
  -s, --site            Use the Site Package Directory
  -q, --quiet           don't print status messages to stdout
  -v, --comparepkgsetup
                        Compares package installation against Packages own
                        internal unverified setup.py
  -c, --cleanup         Cleanup workarea

David


_______________________________________________
Distutils-SIG maillist  -  Distutils-SIG@...
http://mail.python.org/mailman/listinfo/distutils-sig

[Milind Khadilkar]

QUOTE
imho,

PyPI + PEP 381 + PEP 381 compliant clients == CPAN
UNQUOTE
In addition to PyPI, PEP381, PEP381 compliant clients, we need a dash of changed outlook to get CPAN. The Perl community is --  by hearsay -- oriented sympathetically towards people who are not primarily programmers. In particular, I have heard scientists and finance persons developing web applications say this when introduced to python.

But the same people were more satisfied with the python stuff when somebody helped them through the technicalities.

When someone encounters PyPI for the first time, is it easy to learn about it from the PyPI page itself? Hopeful clicks on the "tutorial" link brings up a bewildering  CheeseShop Tutorial, without an explanation of what CheeseShop is. Reading through the options detailed in the page makes sense only if you already know about them. Definitely not the material for the beginner or for the technologically-just-adequate.

By the way, the PyPI tutorial is just an example. The python community, while excellent in the support it provides to the technically competent, needs to be more friendly towards mere end-users. The attitude should be more charitable than "GOD helps them, and only them, who help themselves".

Or, rather than between commuinities, is it actually the difference between Python and Perl as languages themselves?

Regards
Milind Khadilkar

On Sat, Nov 7, 2009 at 5:16 AM, Tarek Ziadé <ziade.tarek@...> wrote:

On Fri, Nov 6, 2009 at 11:53 PM,  <exarkun@...> wrote:
[..]

>
> PyPI *does* let you download packages easily, it does let you upload new
> ones and organize them without having to worry about integrating them into
> existing packages.  So the features of CPAN that are described as desirable
> are already available in PyPI.

imho,

PyPI + PEP 381 + PEP 381 compliant clients == CPAN

Tarek

_______________________________________________
Distutils-SIG maillist  -  Distutils-SIG@...
http://mail.python.org/mailman/listinfo/distutils-sig

_______________________________________________
Distutils-SIG maillist  -  Distutils-SIG@...
http://mail.python.org/mailman/listinfo/distutils-sig

[David Lyon]

On Fri, 6 Nov 2009 17:50:46 -0600, Ian Bicking <ianb@...> wrote:
> My understanding is that they have all the same problems we do --
> version management, dependencies, isolation, varying quality, etc.

Same problems. But CPAN does it a lot better.

> When I've tried to install things from CPAN it's been surprisingly
> obtuse.  This is not to say there aren't good things, but "we need
> CPAN" doesn't really get to the heart of it.

Works for me - it's like chalk and cheese - but python being the
chalk in this case. Corporate contractor developers know the
difference. Scientific dudes can obviously spot the difference.

More than anything, it is a culture difference. CPAN is taken
really seriously by those involved. There's so much passion to
get things right and make things work smoothly.

The "heart of it" is the CPAN culture.

I never experienced the 'we don't support your platform' issue
ever in perl. To CPAN, it is all just perl.

> Testing is a bit off; "setup.py test" hasn't caught on that
> well..

In perl there is more incentive to provide this.

In Python we don't have any place where this can be automatically
activated on a package. So there is no incentive.

The real world relevance would be some tests to ensure that
the package can work on a number of different platforms and
python versions before being accepted, and that the package
has some tests that can be automatically run.

> In terms of how a setup.py should be structured, and other developer
> packaging issues, I think we're getting closer but we *really* need
> someone to document the conventions and lessons that are usually just
> spread through copy-and-paste and developer feedback...

The individual components that we have are improving and we are
getting closer to having something. But CPAN has an energy level
behind it that we are yet to duplicate.

David

_______________________________________________
Distutils-SIG maillist  -  Distutils-SIG@...
http://mail.python.org/mailman/listinfo/distutils-sig

[Tarek Ziadé]

On Sat, Nov 7, 2009 at 1:01 AM, Kaelin Colclasure <kaelin@...> wrote:

>
> On Nov 6, 2009, at 3:54 PM, Tarek Ziadé wrote:
>
>> 2009/11/7 Kaelin Colclasure <kaelin@...>:
>> [...]
>>>
>>> As both a CPAN contributor and a recent PyPI contributor I think I can
>>> speak
>>> to this:
>>>
>>> When I packaged my first Perl module for CPAN distribution, there was no
>>> question about how to go about it. There was a clear recipe and it worked
>>> without a hitch the first time I tried.
>>>
>>> When I recently packaged my first Python module for PyPI I was a bit
>>> dismayed to learn that easy_install is just one of several ways to
>>> disseminate Python code. The documentation is fragmented... The stuff I
>>> read
>>> on easy_install basically assumed you already knew all about distutils,
>>> and
>>> I eventually had to reorganize my code from a simple module to a package
>>> just to get a few non-code files to be pulled in by the install. A less
>>> persistent would-be contributor might have given up in frustration.
>>>
>>> I joined this mailing list instead, and got my package working with some
>>> kind assistance. :-) But CPAN was just easier to contribute to.
>>
>> What was in particular, the pieces missing in Distutils (doc+code)
>> that made you use Setuptools (doc+code) ?
>
> Since I bootstrapped the environment I used to learn Python with
> easy_install, I naturally went straight to the easy_install docs to learn
> how to give back. I wasn't even aware of this separate thing called
> distutils until I read about it in the easy_install (err setuptools)
> documentation (sic).

Yes, that the great thing *and* the bad thing about Setuptools. It
provided missing features and boostraps
what people needed on the top of Distutils. But in the meantime, it
makes it fuzzy for the end-user that there are two projects, and for
him to understand what's from the Distutils project and what's from
the Setuptools project.

What is clear for us though, is that we need to change Distutils
w.r.t. the Setuptools experience feedback.

For the code part, PEP 376 is one important work.
(http://www.python.org/dev/peps/pep-0376/), it enhances Distutils
with a lot of ideas that were taken from Setuptools (with its author
help), and also adds missing features
we need (like an uninstall command).

For the documentation part I am afraid it will be messy for the end
users trying to package apps in Python *until* all PEPs
have made it into Python.

Although, as Ian Bicking says: we could write today some kind of
all-in-one tutorial so end-users can work out without having to run
after the documenation in several places.

Tarek
_______________________________________________
Distutils-SIG maillist  -  Distutils-SIG@...
http://mail.python.org/mailman/listinfo/distutils-sig

[Tarek Ziadé]

On Sat, Nov 7, 2009 at 1:35 AM, Milind Khadilkar <zedobject@...> wrote:

> QUOTE
> imho,
>
> PyPI + PEP 381 + PEP 381 compliant clients == CPAN
> UNQUOTE
> In addition to PyPI, PEP381, PEP381 compliant clients, we need a dash of
> changed outlook to get CPAN. The Perl community is --  by hearsay --
> oriented sympathetically towards people who are not primarily programmers.
> In particular, I have heard scientists and finance persons developing web
> applications say this when introduced to python.
>
> But the same people were more satisfied with the python stuff when somebody
> helped them through the technicalities.
>
> When someone encounters PyPI for the first time, is it easy to learn about
> it from the PyPI page itself? Hopeful clicks on the "tutorial" link brings
> up a bewildering  CheeseShop Tutorial, without an explanation of what
> CheeseShop is. Reading through the options detailed in the page makes sense
> only if you already know about them. Definitely not the material for the
> beginner or for the technologically-just-adequate.
>
> By the way, the PyPI tutorial is just an example. The python community,
> while excellent in the support it provides to the technically competent,
> needs to be more friendly towards mere end-users. The attitude should be
> more charitable than "GOD helps them, and only them, who help themselves".
>
> Or, rather than between commuinities, is it actually the difference between
> Python and Perl as languages themselves?
>

I don't know about the difference between Perl and Python, but I do
know this about me
(and this applies to other people in this room I believe)

As a "packaging tools" developer I am totally plunged into the
technical aspects of the project,
and I unfortunately don't have all the time I wish I had to work on this more.
Especially right now because there's a lot of work going on.

So, for the documentation part, I am hiding behind the agile
manifesto, which says,

"working software over comprehensive documentation"

but that's a shame :)

Now, if you do want to help in the documentation so it's better for
newcomers, (in PyPI, Distutils, etc),
I think we would all be happy on such contributions

Tarek
_______________________________________________
Distutils-SIG maillist  -  Distutils-SIG@...
http://mail.python.org/mailman/listinfo/distutils-sig