Recommendation for the procedure to add platform packages

[Duncan Coutts]

All,

The Haskell Platform steering committee have been very busy over the
past two weeks drafting, discussing and redrafting.

The result is our recommendation to the libraries list for a procedure
for adding new packages to the Haskell Platform.

Before you all go off and read it, we'd like to say what feedback we are
looking for and what the next steps will be. The document we've come up
with describes the procedure but only some very basic quality
requirements for packages. Our aim is to have the libraries list come to
an agreement on the procedure first and then we can discuss codifying a
more comprehensive set of package requirements and guidelines.

The main document contains:
      * the procedure itself, which is relatively short
      * a rationale, cross-linked to the procedure
      * a procedure to help us make decisions

http://trac.haskell.org/haskell-platform/wiki/AddingPackages

There is an accompanying "how to" guide to help the people who will
actually be making proposals:

http://trac.haskell.org/haskell-platform/wiki/AddingPackages/HowTo

There is also an example proposal:

http://trac.haskell.org/haskell-platform/wiki/Proposals/example

So please send in your comments and of course feel free to ask questions
and seek clarifications. Our hope is that we'll be able to agree this
procedure stuff relatively quickly and then move onto package quality
issues and indeed to actually start proposing packages.


Let me also say that I've been very pleased with how the steering
committee has worked. People have put in a lot of effort. It took 2
weeks, nearly 60 emails and over 100 edits to the draft. We didn't
always agree but we talked things through thoroughly and turned our
early draft into something much better that we're all satisfied with.

Duncan,
  on behalf of the rest of the steering committee:
  Iavor Diatchki, Isaac Dupree, Thomas Schilling,
  Johan Tibell and Adam Wick.

_______________________________________________
Libraries mailing list
Libraries@...
http://www.haskell.org/mailman/listinfo/libraries

[Sean Leather]

The Haskell Platform steering committee have been very busy over the
past two weeks drafting, discussing and redrafting.

Looks nice!
 

Our aim is to have the libraries list come to
an agreement on the procedure first and then we can discuss codifying a
more comprehensive set of package requirements and guidelines.

Sorry to diverge from this, but I just wanted to note that something to the effect of "following the license policy" should be mentioned in the package requirements. Yes, the "interim license policy" is not far down, but adding this would make it clearer, I think.

Regards,
Sean

_______________________________________________
Libraries mailing list
Libraries@...
http://www.haskell.org/mailman/listinfo/libraries

[Magnus Therning]

I found this

  "Compile on all operating systems and compilers that the platform targets."

to be a little confusing.  HP 2009.2.0.2 for Windows does include
Win32 (providing System.Win32).  Of course there's not much point in
including Win32 in HP for a Unix platform.  I assume the same is true
for unix (providing System.Posix), but in the opposite direction, so
to speak.  It would be a shame to not include these two
_platform-specific_ packages in HP.

OTOH, maybe I'm not using the word "compile" in the correct sense...
Would a platform-specific package qualify if building it on _other_
platforms resulted in a successful build, but with empty output?

/M

--
Magnus Therning                        (OpenPGP: 0xAB4DFBA4)
[e-mail]          Jabber: [e-mail]
http://therning.org/magnus         identi.ca|twitter: magthe
_______________________________________________
Libraries mailing list
Libraries@...
http://www.haskell.org/mailman/listinfo/libraries

[Duncan Coutts]

On Thu, 2009-08-20 at 08:51 +0200, Sean Leather wrote:
>
>         The Haskell Platform steering committee have been very busy
>         over the
>         past two weeks drafting, discussing and redrafting.
>
> Looks nice!

Thanks.

>         Our aim is to have the libraries list come to
>         an agreement on the procedure first and then we can discuss
>         codifying a
>         more comprehensive set of package requirements and guidelines.
>
> Sorry to diverge from this, but I just wanted to note that something
> to the effect of "following the license policy" should be mentioned in
> the package requirements. Yes, the "interim license policy" is not far
> down, but adding this would make it clearer, I think.

Technically the "interim license policy" is a sub-section of the
"package requirements" section, not that this is terribly clear from the
size of fonts used. So point taken.

Lets clarify this when we get a proper license policy, or if we need to
expand and reorganise the package requirements section in general.

Duncan


_______________________________________________
Libraries mailing list
Libraries@...
http://www.haskell.org/mailman/listinfo/libraries

[Duncan Coutts]

On Thu, 2009-08-20 at 08:56 +0100, Magnus Therning wrote:

> I found this
>
>   "Compile on all operating systems and compilers that the platform targets."
>
> to be a little confusing.  HP 2009.2.0.2 for Windows does include
> Win32 (providing System.Win32).  Of course there's not much point in
> including Win32 in HP for a Unix platform.  I assume the same is true
> for unix (providing System.Posix), but in the opposite direction, so
> to speak.  It would be a shame to not include these two
> _platform-specific_ packages in HP.

You're right that we somehow need to allow these exceptions but without
encouraging non-portable packages in general.

> OTOH, maybe I'm not using the word "compile" in the correct sense...
> Would a platform-specific package qualify if building it on _other_
> platforms resulted in a successful build, but with empty output?

I think "build and work" would be a better expression of the intention.

Duncan

_______________________________________________
Libraries mailing list
Libraries@...
http://www.haskell.org/mailman/listinfo/libraries

[Ian Lynagh]

On Thu, Aug 20, 2009 at 03:04:20AM +0100, Duncan Coutts wrote:
>
> There is also an example proposal:
>
> http://trac.haskell.org/haskell-platform/wiki/Proposals/example
>
> So please send in your comments and of course feel free to ask questions

I read the example first, so I'll give my comments on it first:


My first impression is that there is an awful lot of text, and that it's
mostly in paragraph form (i.e. I need to actually read it, rather than
the key points being highlighted). I think this will reduce the level of
review that proposals get.

    This is a proposal for the 'tar' package to be included in the next
    major release of the Haskell platform.

Should specify version number.

    Review comments should be sent to the libraries mailing list by
    ${deadline - 4 weeks}

This sounds like it is the wrong way round to me. The deadline for
review comments should be "${now + 2 weeks}" (for some value of 2, and
larger values for complicated/large proposals). If you make the proposal
early, then it can be agreed earlier. If you make it too late for the
discussion and consensus to happen by the deadline then it can wait
until the next release.

(that's pretty much how library submission proposals work).

    Abstract

I think this should be replaced with a link to the hackage page, which
shows the abstract from the Cabal file. You might argue that having all
the info in one place is convenient for reviewers, but
* I am unconvinced that it is much easier
* Packages that are proposed are probably widely used anyway, so this is
  just noise to a significant proportion of people
* by only linking to it you remove the temptation to alter the text in
  the proposal but not apply the improvments to the Cabal file.

    Rationale

I don't think the actual rationale part of this section is useful. It
mostly says "The foo library is useful because some people want to foo".
If during the discussion we find that the rationale is unclear then a
rationale could be added to the proposal, but mostly I expect it will be
self-evident (and if not then the abstract in the Cabal file probably
needs some work).

The list of users is useful (less convinced by the list of packages you
think /ought/ to use it), but a bullet-pointed list would be much easier
to read.

    Introduction to the API

What I said about the abstract applies here, only moreso. If an intro to
the API is useful, then one should already exist. People should not be
given the opportunity to /write/ an introduction to the API here; it
should already be in the haddock docs on hackage, or on the project page
on community, or in some similar location. Therefore this too should
just be a link. Again, this will remove the temptation to update the
proposal but not the real docs following the discussion. It will also
mean people don't waste time reformatting docs in wiki markup.

    Design decisions

Ditto.

    Open issues

This is the interesting bit of the proposal, IMO. What I want to see for
a proposal is just something like:

    Package:                                          mylib 1.2
    Is useful?               [green] [link to below]  Yes[/link]   [/green]
    Has intro to API?        [green] [link to hackage]Yes[/link]   [/green]
    Has design decision doc? [yellow]                 No           [/yellow]
    Builds on all platforms? [green]                  Yes          [/green]
    Builds with all impls?   [yellow]                 No           [/yellow]
    Good API?                [green]                  No objections[/green]

    Is useful ("Is useful?" above links to here)
    ---------
    Users include:
    * darcs
    * tar

    Has design decision
    -------------------
    The package is too simple for such a doc to be useful. [...]

    Builds with all impls
    ---------------------
    hugs is missing the WibbleTheThingy extension, so cannot build the
    package. But H2010 will include WibbleTheThingy, so hugs is expected
    to implement it soon.

and the goal of the discussion would then be to get a consensus on
whether the yellow sections are severe enough to not accept the package
(and also to give people a chance to disagree that some of the green
entries really are green, e.g. "Is useful?" is subjective).

I've mentioned this link on IRC before, but for those who haven't seen
it, this is partly inspired by:
    http://release.debian.org/etch/arch_qualify.html
(which shows at a glance which arches are acceptable to be in a Debian
release, and where the issues and potential issues are. The link in the
first row gives more detail on some of the points).

>
> http://trac.haskell.org/haskell-platform/wiki/AddingPackages
>

I've only skimmed this, due to the length, but my impression is that it
is too heavy-weight. e.g.
    http://www.haskell.org/haskellwiki/Library_submissions
has this to say about consensus:

    If someone has done all this, they are entitled to expect feedback;
    silence during the discussion period can be interpreted as consent.

    If consensus was achieved, [...]

and doesn't seem to have had problems with getting consensus, or
agreeing whether we have got it. The "Achieving consensus" section of
AddingPackages is 947 words according to wc (that's about double the
length of the entire Library_submissions page).

Also, the fact a credits section was deemed necessary suggests to me
that too much work is involved.



Thanks
Ian

_______________________________________________
Libraries mailing list
Libraries@...
http://www.haskell.org/mailman/listinfo/libraries

[Duncan Coutts]

On Fri, 2009-08-21 at 02:23 +0100, Ian Lynagh wrote:
> On Thu, Aug 20, 2009 at 03:04:20AM +0100, Duncan Coutts wrote:
> >
> > There is also an example proposal:
> >
> > http://trac.haskell.org/haskell-platform/wiki/Proposals/example
> >
> > So please send in your comments and of course feel free to ask questions
>
> I read the example first, so I'll give my comments on it first:

The policy is significantly less prescriptive than the example might
lead you to believe.

The essential requirement is given as:

        The proposal forms the nub of the argument for why the package
        should be included in the Haskell Platform. It should contain or
        link to enough information so that reviewers can make an
        informed decision on whether the package should be accepted.

With further specific details given in the "Proposal content" section:
http://trac.haskell.org/haskell-platform/wiki/AddingPackages#Proposalcontent


I should also note that I'm happy to improve my tar proposal to pick up
some of the points you make. In addition to being an example proposal I
intend to submit it as a real proposal, so improvements benefit both the
real proposal and the example.

> My first impression is that there is an awful lot of text, and that it's
> mostly in paragraph form (i.e. I need to actually read it, rather than
> the key points being highlighted). I think this will reduce the level of
> review that proposals get.

Below you suggest that instead the proposal should link to existing
documentation (and that such documentation be required to exist). I
presume the documentation linked to would have the same amount of text.
Assuming that's what you mean, I'm not sure what you're comparing to
that has less text that we might expect would give us an increased level
of review.

>     This is a proposal for the 'tar' package to be included in the next
>     major release of the Haskell platform.
>
> Should specify version number.

I expect that usually the exact number is not especially relevant. The
exact released version now is not necessarily the same as the final one
that gets included in the case that reviewers make suggestions.

If there's more than one active branch then yes it's relevant so
reviewers know which one to look at. Otherwise reviewers can reasonably
assume the version on the hackage page.

The guidance notes also mention that if reviewers are expected to have
to look at a development version, that the repo should be given in the
proposal.

>     Review comments should be sent to the libraries mailing list by
>     ${deadline - 4 weeks}
>
> This sounds like it is the wrong way round to me. The deadline for
> review comments should be "${now + 2 weeks}" (for some value of 2, and
> larger values for complicated/large proposals). If you make the proposal
> early, then it can be agreed earlier. If you make it too late for the
> discussion and consensus to happen by the deadline then it can wait
> until the next release.

Suggesting that comments be made by a time relative to the proposal time
rather than the final deadline would also be consistent with the policy.
The constraints are that people be given a reasonable amount of time to
review things and that discussions and issues are resolved before the
final deadline.

I'd be happy to change the guidance to suggest now + time rather than
final deadline - time.

> (that's pretty much how library submission proposals work).

Sure. Though I think we need to clarify how much time is reasonable for
review rather than letting the proposers pick whatever they like.

>     Abstract
>
> I think this should be replaced with a link to the hackage page, which
> shows the abstract from the Cabal file. You might argue that having all
> the info in one place is convenient for reviewers, but
> * I am unconvinced that it is much easier
> * Packages that are proposed are probably widely used anyway, so this is
>   just noise to a significant proportion of people
> * by only linking to it you remove the temptation to alter the text in
>   the proposal but not apply the improvments to the Cabal file.

The policy allows the info to be inline in the proposal or linked to.
It's up to the proposal author:

        Where appropriate, existing sources of information about the
        package may be copied or linked to.

>     Rationale
>
> I don't think the actual rationale part of this section is useful. It
> mostly says "The foo library is useful because some people want to foo".
> If during the discussion we find that the rationale is unclear then a
> rationale could be added to the proposal, but mostly I expect it will be
> self-evident (and if not then the abstract in the Cabal file probably
> needs some work).

It's intended to cover these proposal requirements:

        The proposal forms the nub of the argument for why the package
        should be included in the Haskell Platform.

        It should explain the gap in the existing set of platform
        packages that the new package addresses. Alternatively if the
        package would duplicate or replace any existing platform
        packages then the reasons why the new package is better should
        be clearly explained. In this case the proposal should be tied
        to another proposal to deprecate the existing package(s) (or
        parts thereof) and thought should be given to how the transition
        should be managed. [note-7.2]

The guidance notes say:

        The rationale is the core of your argument as to why this
        package should be added to the platform. Your task here is to
        convince people. What makes the package useful? What are the use
        cases? Being able to point to existing uses is always good.
        Similarly, existing comments or reviews from users can be
        helpful. Old release announcements can be a good source of
        inspiration here.
       
        If your package is a replacement for an existing package (or
        part of one) then you'll want to convince people why the new one
        is an improvement.

I agree that what I've written for the rationale for the tar proposal is
of somewhat dubious utility. In fact personally I don't like the term
"rationale" for the section.

> The list of users is useful (less convinced by the list of packages you
> think /ought/ to use it), but a bullet-pointed list would be much easier
> to read.

Well I'm convinced darcs ought to use it ;-). I'm citing it there as an
example of an alternative to using the tar package, to use an external
tar program (and the various disadvantages that has).

In general I don't think it's an unreasonable argument to point out
cases where existing packages could be improved by making use of the new
package rather than using some existing home-grown solution. Of course
this is all a judgement for the proposal author for what they think
makes a convincing argument (and for the reviewers to decide if they're
convinced). I'll keep it in mind that you don't find it convincing :-).

>     Introduction to the API
>
> What I said about the abstract applies here, only moreso. If an intro to
> the API is useful, then one should already exist.

It's explicitly ok to link to one where one already exists. The policy
and guidance notes say as much.

> People should not be given the opportunity to /write/ an introduction
> to the API here; it should already be in the haddock docs on hackage,
> or on the project page on community, or in some similar location.

If we add that as a package requirement then it would be reasonable to
change the advice to say simply to link to it. Currently the only doc
requirement we've put in is:

        Library packages should have Haddock API documentation for all
        exported functions and types.

I expect once we've agreed the procedure that we'll get onto more
discussing more detailed requirements on things like documentation.

In the tar example I had already made an introductory section to the API
in the haddock docs. However on trying to present it again in the
proposal (by doing a fair bit of copy and pasting) I think I made some
improvements to how and the order in which concepts are presented.
Writing with a particular audience in mind can help a lot.

> Therefore this too should just be a link. Again, this will remove the
> temptation to update the proposal but not the real docs following the
> discussion. It will also mean people don't waste time reformatting
> docs in wiki markup.

The guidance notes say:

        Of course, if you end up making something new for the proposal
        then it'd be great to keep it somewhere so that new users can
        benefit from it too. The obvious place is in the Haddock docs,
        or a project website.

>     Design decisions
>
> Ditto.

The same points apply. If we want to require that packages must have
this kind of documentation then it ought to be spelled out explicitly as
a package requirement. Currently it's just part of the proposal because
it's a less onerous requirement.

>     Open issues
>
> This is the interesting bit of the proposal, IMO. What I want to see for
> a proposal is just something like:

That's a quite different kind of thing from what the open issues section
is intended for. The policy says:

        Open issues and objections raised by reviewers should be tracked
        on the proposal wiki under a separate section. [note-3.2]

and

        An explicit checklist of the package requirements below is not
        required. The proposal should state however that all the
        requirements are met, or for any requirements that are not met,
        a reason why they are not met. [note-7.6]

What you're asking for is a checklist. That idea was particularly
controversial within the steering committee and it was not adopted.
Personally I was weakly in favour if a checklist but the concerns were
that it restricted the arguments and explanations that the proposal
author could make. The consensus was for a much more free-form proposal
rather than forcing things into a checklist.

> and the goal of the discussion would then be to get a consensus on
> whether the yellow sections are severe enough to not accept the package
> (and also to give people a chance to disagree that some of the green
> entries really are green, e.g. "Is useful?" is subjective).

I think that would come under the complaint that that is too narrow for
the argument for why the package should be included. Perhaps other
members of the steering committee would like to comment on this point.

On the other hand if it's not part of the proposal then I think you'd
get a better reception. I know that the release team will want to use
its own status board for the aspects of packages that particularly
affect the release team (like building on various arches).

> I've mentioned this link on IRC before, but for those who haven't seen
> it, this is partly inspired by:
>     http://release.debian.org/etch/arch_qualify.html
> (which shows at a glance which arches are acceptable to be in a Debian
> release, and where the issues and potential issues are. The link in the
> first row gives more detail on some of the points).

My concern if this is the main focus of discussions is that it covers
the "does it work" aspect reasonably well, it doesn't cover the "is it
any good? how could it be better?" aspects.

> >
> > http://trac.haskell.org/haskell-platform/wiki/AddingPackages
> >
>
> I've only skimmed this, due to the length, but my impression is that it
> is too heavy-weight. e.g.
>     http://www.haskell.org/haskellwiki/Library_submissions
> has this to say about consensus:
>
>     If someone has done all this, they are entitled to expect feedback;
>     silence during the discussion period can be interpreted as consent.
>
>     If consensus was achieved, [...]
>
> and doesn't seem to have had problems with getting consensus, or
> agreeing whether we have got it.

That's the main reason we agreed to go with a consensus method. The
policy states:

        The standard of consensus required is the same as that defined
        for the existing library submissions process. [note-5.4]

Our early draft simply said that and nothing else. Several members of
the steering committee and others who read drafts were unsure or
unconvinced that consensus would be a usable method. We agreed to try it
and see if it works. In addition we added the option to use this
consensus protocol.

> The "Achieving consensus" section of
> AddingPackages is 947 words according to wc (that's about double the
> length of the entire Library_submissions page).

You will notice that in the simple case there is essentially no
formality at all:

        The first stage is presenting the proposal followed by open
        group discussion. This stage lasts long enough to give everyone
        the opportunity to send in their review comments. There follows
        a call for consensus.

This is essentially the same as the libraries process (including the bit
about expecting feedback and silence is consent). We only need to go
further when there are remaining disagreements.

So I do not agree that it is too heavy-weight. It starts out very light
and gets more structured as the severity of the disagreement increases
and as the final deadline approaches.

> Also, the fact a credits section was deemed necessary suggests to me
> that too much work is involved.

It was agreed because it's a cheap and easy way to promote the work of
review which is often undervalued. We also think that serious review is
a great way to improve package quality and entry to the platform is an
appropriate moment to do it.

There is clearly a trade off between:
      * work for package proposers / maintainers
      * work for reviewers
      * quality of package review and quality of decisions

My impression so far is that opinions are reasonably evenly split on all
three. To me, that indicates we're roughly in the right place.

What are you suggesting specifically that would reduce the amount of
work for reviewers? Would those things increase work for the proposal
authors / package maintainers? Would those things increase or decrease
the level of depth/quality of review and decisions?

Of course if there are any suggestions for straightforward wins in this
trade-off then we should consider them seriously.

Duncan

_______________________________________________
Libraries mailing list
Libraries@...
http://www.haskell.org/mailman/listinfo/libraries

[Isaac Dupree-3]

>> Also, the fact a credits section was deemed necessary suggests to me
>> that too much work is involved.
>
> It was agreed because it's a cheap and easy way to promote the work of
> review which is often undervalued.

Exactly!  Probably the main motivation for package authors is not that
they get credited... but, nevertheless, they get credited.  It's helpful
(even, generally, as a matter of programmer/open-source culture) to try
and make equally important work be equally glamorous.

-Isaac
_______________________________________________
Libraries mailing list
Libraries@...
http://www.haskell.org/mailman/listinfo/libraries

[Ian Lynagh]

On Fri, Aug 21, 2009 at 08:58:56PM +0100, Duncan Coutts wrote:

> On Fri, 2009-08-21 at 02:23 +0100, Ian Lynagh wrote:
> > On Thu, Aug 20, 2009 at 03:04:20AM +0100, Duncan Coutts wrote:
> > >
> > > There is also an example proposal:
> > >
> > > http://trac.haskell.org/haskell-platform/wiki/Proposals/example
> > >
> > > So please send in your comments and of course feel free to ask questions
> >
> > I read the example first, so I'll give my comments on it first:
>
> The policy is significantly less prescriptive than the example might
> lead you to believe.

Looking again at AddingPackages, I think you are right. That said, if I
were making a proposal, I would probably base it on an example (or a
template, which is essentially the same thing).

The AddingPackages page confuses me. If I need to read all the notes,
then the page seems very hard to read, and if I don't then could the
rationale be put on a separate page? Also, the [note-1.3]s could all be
[rationale] or [rationale-1.3], right? That way I wouldn't think that I
have to read them unless I am actually interested in the rationale.
Currently the page looks daunting to me.

> I should also note that I'm happy to improve my tar proposal to pick up
> some of the points you make. In addition to being an example proposal I
> intend to submit it as a real proposal, so improvements benefit both the
> real proposal and the example.

I've made an alternative proposal for the tar package here:

http://trac.haskell.org/haskell-platform/wiki/Proposals/alternate_example

I've made up some of the answers, and also just taken most of the
suggested requirements to be the package requirement list.

(That the trac colour scheme means that some of the hyperlinks are hard
to identify, but we can sort that out somehow).

I would be interested to hear people's opinions on how they compare.

> >     This is a proposal for the 'tar' package to be included in the next
> >     major release of the Haskell platform.
> >
> > Should specify version number.
>
> I expect that usually the exact number is not especially relevant. The
> exact released version now is not necessarily the same as the final one
> that gets included in the case that reviewers make suggestions.
>
> If there's more than one active branch then yes it's relevant so
> reviewers know which one to look at. Otherwise reviewers can reasonably
> assume the version on the hackage page.

I don't see the harm in giving the intended version number in the
proposal.

It may be that 1.18 is originall proposed and 1.20 ends up being what
goes into the next release, if the changes are either following the
decisions made during the discussion, or small enough that the release
team would happily automatically upgrade 1.18 to 1.20 between two major
releases.

> The policy allows the info to be inline in the proposal or linked to.

Right, I saw that, but I think it should be /required/ to be a link.

> >     Introduction to the API
> >
> > People should not be given the opportunity to /write/ an introduction
> > to the API here; it should already be in the haddock docs on hackage,
> > or on the project page on community, or in some similar location.
>
> If we add that as a package requirement then it would be reasonable to
> change the advice to say simply to link to it. Currently the only doc
> requirement we've put in is:
>
>         Library packages should have Haddock API documentation for all
>         exported functions and types.

I didn't mean to give an opinion on whether or not an intro to the API
should be required or not, but:
* if it is required then I think it should be linked to
* if it isn't then I certainly don't think the proposal should have to
  include one (because then there would be no reason not to require that
  it exists).

Based on our earlier IRC discussion:

<Igloo> So is the API intro in the haddock docs?
<dcoutts> yes
<Igloo> OK, ta
<dcoutts> that's where I copied most of the stuff in the proposal from
<dcoutts> but re-edited, hopefully to be better
<dcoutts> I also left out detail to make it a shorter and easier intro

it sounds to me like this is an example of exactly what I wanted to
avoid. You've spent some time making a nice friendly introduction to the
tar API, but in 6 months time when someone comes to use the tar package
they won't find this documentation, as it only exists in a "package
addition" proposal buried on the haskell-platform wiki. If you're going
to go to all the effort of making an API intro, you ought to put it on
the package's homepage.

> I expect once we've agreed the procedure that we'll get onto more
> discussing more detailed requirements on things like documentation.

Drifting off-topic, but I think that the two discussions are orthogonal
(i.e. we could have them in either order), but the "detailed
requirements" discussion is the more important one. People can make
proposals without a process being nailed down (and that may even help us
work out what the process should be), but how can we decide if a package
should be included if we don't know what the acceptance criteria are? So
I think the other discussion is much closer on the critical path.

> > Therefore this too should just be a link. Again, this will remove the
> > temptation to update the proposal but not the real docs following the
> > discussion. It will also mean people don't waste time reformatting
> > docs in wiki markup.
>
> The guidance notes say:
>
>         Of course, if you end up making something new for the proposal
>         then it'd be great to keep it somewhere so that new users can
>         benefit from it too. The obvious place is in the Haddock docs,
>         or a project website.

Yes, and people will honestly intend to merge the changes and new docs
back while they are writing them. But 90% of the time they won't.

> >     Open issues
> >
> > This is the interesting bit of the proposal, IMO. What I want to see for
> > a proposal is just something like:
>
> That's a quite different kind of thing from what the open issues section
> is intended for. The policy says:
>
>         Open issues and objections raised by reviewers should be tracked
>         on the proposal wiki under a separate section. [note-3.2]
>
> and
>
>         An explicit checklist of the package requirements below is not
>         required.
>
> What you're asking for is a checklist.

Yes indeed. As in my example:

http://trac.haskell.org/haskell-platform/wiki/Proposals/alternate_example

> That idea was particularly
> controversial within the steering committee and it was not adopted.
> Personally I was weakly in favour if a checklist but the concerns were
> that it restricted the arguments and explanations that the proposal
> author could make. The consensus was for a much more free-form proposal
> rather than forcing things into a checklist.

For items that benefit from free-form text, you can link to a section
beyond the checklist, as in my example. Where there would no benefit, a
simple "Yes" in the table suffices, and doesn't waste any reviewer time.

> > and the goal of the discussion would then be to get a consensus on
> > whether the yellow sections are severe enough to not accept the package
> > (and also to give people a chance to disagree that some of the green
> > entries really are green, e.g. "Is useful?" is subjective).
>
> I think that would come under the complaint that that is too narrow for
> the argument for why the package should be included.

I don't understand what you mean by that.

> > I've mentioned this link on IRC before, but for those who haven't seen
> > it, this is partly inspired by:
> >     http://release.debian.org/etch/arch_qualify.html
> > (which shows at a glance which arches are acceptable to be in a Debian
> > release, and where the issues and potential issues are. The link in the
> > first row gives more detail on some of the points).
>
> My concern if this is the main focus of discussions is that it covers
> the "does it work" aspect reasonably well, it doesn't cover the "is it
> any good? how could it be better?" aspects.

You can have an "Any API issues?" question, which if the answer is not
"No" will link to one or more free-form text sections. But if the answer
is "No" then it's clear at a glance that it is no. Much like the "Other
issues" item in my example.

>         The standard of consensus required is the same as that defined
>         for the existing library submissions process. [note-5.4]
>
> Our early draft simply said that and nothing else. Several members of
> the steering committee and others who read drafts were unsure or
> unconvinced that consensus would be a usable method. We agreed to try it
> and see if it works. In addition we added the option to use this
> consensus protocol.
>
> You will notice that in the simple case there is essentially no
> formality at all:
>
>         The first stage is presenting the proposal followed by open
>         group discussion. This stage lasts long enough to give everyone
>         the opportunity to send in their review comments. There follows
>         a call for consensus.

In fact, I think that's the main problem I have with both the process
document and the example proposal: They try to anticipate all questions
that might be asked, and answer them. But the common case is that the
question wouldn't be asked, so this is just more stuff for someone to
write, and and more stuff for many people to read.

That's why here:

> I agree that what I've written for the rationale for the tar proposal is
> of somewhat dubious utility. In fact personally I don't like the term

you've had to write stuff "of somewhat dubious utility": because you're
answering a boilerplate question that didn't need asking in this case.

And in the case of consensus, I don't think we need to lay down rules
now for what happens if the current system doesn't work. For one thing,
it hasn't been an issue so far. For another, when we have the first case
of it not working, it might be clear that the solution would be to do
something else anyway.

> What are you suggesting specifically that would reduce the amount of
> work for reviewers?

I am suggesting that there be less text written.

> Would those things increase work for the proposal
> authors / package maintainers?

It should reduce work for the authors / package maintainers.

> Would those things increase or decrease
> the level of depth/quality of review and decisions?

I believe it would increase the number of reviewers, giving better
decisions.


Thanks
Ian

_______________________________________________
Libraries mailing list
Libraries@...
http://www.haskell.org/mailman/listinfo/libraries

[Duncan Coutts]

Ian,

You raise a number of points. I'll address the easy ones here and
summarise the others in a separate email.

What I've left out is:
      * the issue about whether things should be in the proposal or
        should be required to exist as separate documents that are
        linked from the proposal.
      * the issue of a checklist / status table in the proposal

Summary of the bits where I think we already agree:
      * use [rationale-1.3] links rather than [note-1.3] for better
        clarity
      * suggest the proposal mention the package's version number
      * the 'tar' example could be improved and shortened

On Sat, 2009-08-22 at 01:17 +0100, Ian Lynagh wrote:

> On Fri, Aug 21, 2009 at 08:58:56PM +0100, Duncan Coutts wrote:
> > On Fri, 2009-08-21 at 02:23 +0100, Ian Lynagh wrote:
> > > I read the example first, so I'll give my comments on it first:
> >
> > The policy is significantly less prescriptive than the example might
> > lead you to believe.
>
> Looking again at AddingPackages, I think you are right. That said, if I
> were making a proposal, I would probably base it on an example (or a
> template, which is essentially the same thing).
>
> The AddingPackages page confuses me. If I need to read all the notes,

It is not essential to read them to be able to understand the policy.
They are there to answer the question "why was this requirement
included?" and possibly also to clarify the meaning by stating the
intentions.

> then the page seems very hard to read, and if I don't then could the
> rationale be put on a separate page?

A separate page would make it much slower to jump back and forth between
the rationale and the policy.

The intro says:

      * The rationale section gives an explanation and justification for
        the policy.

Is there a wording that would make it clearer that you do not have to
read it unless you want an explanation for a point in the policy?

> Also, the [note-1.3]s could all be [rationale] or [rationale-1.3],
> right? That way I wouldn't think that I have to read them unless I am
> actually interested in the rationale.

That's a good idea, we should do that.

> Currently the page looks daunting to me.

We already split off the "how to" guide into a separate page. We could
possibly do the same with the consensus protocol. I think the rationale
should stay with the policy however.

> > > Should specify version number.
> >
> > I expect that usually the exact number is not especially relevant. The
> > exact released version now is not necessarily the same as the final one
> > that gets included in the case that reviewers make suggestions.
> >
> > If there's more than one active branch then yes it's relevant so
> > reviewers know which one to look at. Otherwise reviewers can reasonably
> > assume the version on the hackage page.
>
> I don't see the harm in giving the intended version number in the
> proposal.

Right, there's no harm and it's fairly easy for the author to add that
info. I'm happy to suggest it in the "how to" guide. I don't think it's
such crucial info that it needs to be explicitly required by the policy.


> Drifting off-topic, but I think that the two discussions are orthogonal
> (i.e. we could have them in either order), but the "detailed
> requirements" discussion is the more important one. People can make
> proposals without a process being nailed down (and that may even help us
> work out what the process should be), but how can we decide if a package
> should be included if we don't know what the acceptance criteria are?

The acceptance criteria is clearly specified. It is not that a set of
minimal requirements be met. It is that the reviewers, using their
judgement come to a consensus to accept the package.

> So I think the other discussion is much closer on the critical path.

I don't think it is on the critical path. I think it is quite possible
to start reviewing packages and try to codify what we agree additional
criteria should be. If there are significant disagreements between
reviewers while reviewing packages then these would indicate areas where
it may be helpful to codify a requirement or guideline.

There is certainly an advantage to having a more comprehensive set of
requirements and guidelines because it'll help people proposing packages
to know what to aim for and should help reviewers to agree with each
other about a package.


> In fact, I think that's the main problem I have with both the process
> document and the example proposal: They try to anticipate all questions
> that might be asked, and answer them. But the common case is that the
> question wouldn't be asked, so this is just more stuff for someone to
> write, and and more stuff for many people to read.

This is true, but is the balance of benefit such that we should leave
various questions open in favour of making it slightly quicker to read
the policy? I note that the policy is only 1500 words. We deliberately
tried to keep it short by separating it from the rationale and from the
practical advice.

Is there anything specific in the policy you would remove that would
make the thing appreciably shorter without making it too ambiguous? My
guess is that you'll say the consensus protocol.

> That's why here:
>
> > I agree that what I've written for the rationale for the tar proposal is
> > of somewhat dubious utility. In fact personally I don't like the term
>
> you've had to write stuff "of somewhat dubious utility": because you're
> answering a boilerplate question that didn't need asking in this case.

I think it's partly that I originally wrote the example for a draft of
the policy that was significantly more prescriptive about the structure
of the proposal and I did not update it significantly when I adapted it
for the final draft.

> And in the case of consensus, I don't think we need to lay down rules
> now for what happens if the current system doesn't work. For one thing,
> it hasn't been an issue so far.

That is what we had in an intermediate draft. We added the facility for
more formality in contentious cases because several members of the
steering committee and other people who read the draft expressed the
concern that despite consensus working ok for the libraries process,
that it might not work so well for deciding on package additions.

Is there any disadvantage to having this option available? Do you think
it is a bad option or merely unnecessary? Is it just the fact that it
makes the wiki page longer?

> For another, when we have the first case of it not working, it might
> be clear that the solution would be to do something else anyway.

That option is always open anyway. The rationale says:

        [note-5.4] The exact definition of what constitutes a consensus
        is possibly a contentious point. The obvious choice is to do
        what we already do with the existing library submissions
        process. It uses a relatively vague definition of consensus and
        yet has worked reasonably well in practice. If it does not work
        out in practice for the package addition process then the
        libraries list may want to revisit this point.

> > What are you suggesting specifically that would reduce the amount of
> > work for reviewers?
>
> I am suggesting that there be less text written.

Less text in the proposal or less text in total? You've suggested that
the proposal link to several external documents.

> > Would those things increase work for the proposal
> > authors / package maintainers?
>
> It should reduce work for the authors / package maintainers.

Even taking into account that you would expect them to write these
external documents? Where is the reduction coming from?

> > Would those things increase or decrease
> > the level of depth/quality of review and decisions?
>
> I believe it would increase the number of reviewers, giving better
> decisions.

I'm afraid I'm not clear about where the reduction is coming from in
what authors / package maintainers have to write and reviewers are
expected to read. Is it just the "rationale waffle" stuff from the 'tar'
example that you mean?

Duncan

_______________________________________________
Libraries mailing list
Libraries@...
http://www.haskell.org/mailman/listinfo/libraries

[Duncan Coutts]

All,

I'll try and summarise the concerns that Ian has raised:

      * That documentation written for the proposal will get lost.
       
        The "how to" says that if people write a new API intro, then it
        should be integrated into the haddock docs or saved on a project
        home page.
       
        Ian thinks this suggestion/advice is not sufficient to ensure
        that the docs do not get lost and thinks that some procedural
        device is necessary.
       
        Ian thinks that the best mechanism is to require the API intro
        be a separate doc that is linked from the proposal. Thus the
        proposal authors would not have the opportunity to "improve" the
        docs for the proposal without those improvements also being
        saved as part of the package.
       
        Ian accepts that there may be other mechanisms to make sure docs
        written for the proposal do not get lost but would have greatest
        confidence in the method he suggested.



      * That the package proposals would be too long and too wordy.
       
        There are a few aspects to this:
       
             1. That text included inline in the proposal is less
                readable than the same amount of text in separate
                documents linked from the proposal (like API intro);
             2. That the "rationale" suggested in the "how to" and
                example is mostly redundant/unnecessary;
             3. That things described in words could be summarised by a
                "yes" in a checklist.

        Specifically that a checklist augmented with text sections is a
        shorter and better proposal format than a free-form proposal.
       
        Related to this is a belief that the criteria for inclusion can
        and should be reduced to evaluating a list of requirements. Some
        of those requirements would be objective, some subjective, but
        all relatively specific. Passing all requirements would be the
        criteria for inclusion.



      * That the policy document itself is too long and too detailed.
       
        That it anticipates eventualities that may or may not arise in
        practise.
       
        That the overall length is a problem because it is off-putting,
        with the danger that people simply will not read it.


Ian, is this a fair summary? Is there anything I've missed that I didn't
address specifically in the previous email?


Duncan

_______________________________________________
Libraries mailing list
Libraries@...
http://www.haskell.org/mailman/listinfo/libraries

[Ian Lynagh]

On Sun, Aug 23, 2009 at 07:49:23PM +0100, Duncan Coutts wrote:
>
> The intro says:
>
>       * The rationale section gives an explanation and justification for
>         the policy.
>
> Is there a wording that would make it clearer that you do not have to
> read it unless you want an explanation for a point in the policy?

I didn't read that line. I'll expand on this (and the other questions
about process description length) in a reply to your other e-mail.

> > Drifting off-topic, but I think that the two discussions are orthogonal
> > (i.e. we could have them in either order), but the "detailed
> > requirements" discussion is the more important one. People can make
> > proposals without a process being nailed down (and that may even help us
> > work out what the process should be), but how can we decide if a package
> > should be included if we don't know what the acceptance criteria are?
>
> The acceptance criteria is clearly specified.

Well, it says (somewhat tautologically):

    A package is considered as accepted if, by the discussion deadline,
    the libraries mailing list reach consensus to accept it.

but I don't see how we can reach consensus about a package that (for
example) is not -Wall clean if we haven't decided whether or not all
packages in the platform must be -Wall clean.

> It is not that a set of
> minimal requirements be met. It is that the reviewers, using their
> judgement come to a consensus to accept the package.

I agree that some of the requirements will be fuzzy (such as "good
API"), and we may even overlook black-and-white requirements in
particular cases if there is a good reason.

> > > Would those things increase work for the proposal
> > > authors / package maintainers?
> >
> > It should reduce work for the authors / package maintainers.
>
> Even taking into account that you would expect them to write these
> external documents? Where is the reduction coming from?

Like you said at the start of your mail, "the 'tar' example could be
improved and shortened", so to some extent at least we are in agreement,
and the issue was just that the example was written for an earlier, more
rigid process.

I suspect that I would say that it could be shortened even more than you
are thinking, with more of your paragraphs being condensed into a line
in the checklist (in the common case where the package obviously
satisfies the criteria).

> Less text in the proposal or less text in total?

A mixture of both.


Thanks
Ian

_______________________________________________
Libraries mailing list
Libraries@...
http://www.haskell.org/mailman/listinfo/libraries

[Ian Lynagh]

On Sun, Aug 23, 2009 at 08:31:17PM +0100, Duncan Coutts wrote:
>
>         Passing all requirements would be the
>         criteria for inclusion.

I would say "Passing all requirements would be sufficient for
inclusion", but we may decide to include packages that don't meet all
the requirements.

>       * That the policy document itself is too long and too detailed.
>        
>         That it anticipates eventualities that may or may not arise in
>         practise.
>        
>         That the overall length is a problem because it is off-putting,
>         with the danger that people simply will not read it.

Right. I was admittedly reading it at ~2am, but IIRC the way I read it
was:

* Hmm, the widget on the scrollbar is very small. Reading this is going
  to take a while.
* Drag widget down a bit. See "[note-6.3]". Hmm, jumping up and down
  between the text and all the notes is really going to break the flow
  if I try and read it properly.
* Drag widget down more. See sections like "Acceptance", "Proposal
  content" and "Package requirements", not realising I was looking at
  rationale.
* Drag widget down more, and see the "Achieving consensus" major
  section. Boggle at the size as compared to the library submissions
  proposal.
* Read a little text, probably from the early "Procedure" section.

By contrast, I would probably have read this:
http://trac.haskell.org/haskell-platform/wiki/AddingPackagesCore


If you do want to keep the rationale in the same document then making it
expandable inline with JavaScript may be better. (It would still work in
non-JS browsers, but be very verbose in them).

> Ian, is this a fair summary? Is there anything I've missed that I didn't
> address specifically in the previous email?

Yes, I think it's a fair summary and covers everything.


Thanks
Ian

_______________________________________________
Libraries mailing list
Libraries@...
http://www.haskell.org/mailman/listinfo/libraries

[Simon Marlow-7]

On 20/08/2009 03:04, Duncan Coutts wrote:

> All,
>
> The Haskell Platform steering committee have been very busy over the
> past two weeks drafting, discussing and redrafting.
>
> The result is our recommendation to the libraries list for a procedure
> for adding new packages to the Haskell Platform.
>
> Before you all go off and read it, we'd like to say what feedback we are
> looking for and what the next steps will be. The document we've come up
> with describes the procedure but only some very basic quality
> requirements for packages. Our aim is to have the libraries list come to
> an agreement on the procedure first and then we can discuss codifying a
> more comprehensive set of package requirements and guidelines.
>
> The main document contains:
>        * the procedure itself, which is relatively short
>        * a rationale, cross-linked to the procedure
>        * a procedure to help us make decisions
>
> http://trac.haskell.org/haskell-platform/wiki/AddingPackages

On the whole, +1.

The process is quite detailed, so we shouldn't expect reviewers to be
familiar with all the rules; instead it's up to the steering committee
to guide the discussion and remind people of the administrative details
at each stage (e.g. "you have until X to respond, or we assume you
agree", or "this is the first phase, during which we are trying to reach
consensus on Z").

Specific comments:

"come to view on whether the package should be accepted"
-> "help achieve a consensus ... "

"the libraries mailing list reach consensus to accept it"
-> "the reviewers reach a consensus to accept it" ?

"Compile on all operating systems and compilers that the platform targets"
-> ".. except where the package is compiler- or platform-specific" ?

The "Interim Licesne Policy" should be one of the bullet points
under Package Requirements.  e.g.

   "The package must be distributed under an acceptable license.  The
    only license currently acceptable is BSD3 [rationale..]"

I'd like to see something about API style mentioned in the "Package
Requirements".  e.g.:

   Packages admitted to the platform should follow an API style
   similar to those packages already in the package.  While the
   style guidelines for package APIs have not yet been written down,
   we expect that to change in the future.

Cheers,
        Simon
_______________________________________________
Libraries mailing list
Libraries@...
http://www.haskell.org/mailman/listinfo/libraries

[Duncan Coutts]

On Tue, 2009-09-08 at 13:02 +0100, Simon Marlow wrote:

> > The main document contains:
> >        * the procedure itself, which is relatively short
> >        * a rationale, cross-linked to the procedure
> >        * a procedure to help us make decisions
> >
> > http://trac.haskell.org/haskell-platform/wiki/AddingPackages
>
> On the whole, +1.
>
> The process is quite detailed, so we shouldn't expect reviewers to be
> familiar with all the rules; instead it's up to the steering committee
> to guide the discussion and remind people of the administrative details
> at each stage (e.g. "you have until X to respond, or we assume you
> agree", or "this is the first phase, during which we are trying to reach
> consensus on Z").

Right. I think that's fine. We'll all get the hang of it after the first
round or so.

> Specific comments:
>
> "come to view on whether the package should be accepted"
> -> "help achieve a consensus ... "

Ok.

> "the libraries mailing list reach consensus to accept it"
> -> "the reviewers reach a consensus to accept it" ?

Ok.

> "Compile on all operating systems and compilers that the platform targets"
> -> ".. except where the package is compiler- or platform-specific" ?

Right, we want language that lets us have platform-specific stuff where
it's really essential but strongly encourage portable as the default.
For example we've got the win32 and unix packages because they're there
to expose the native platform apis. But for example if we added file
change notification I expect we'd want to provide a platform independent
api rather than three packages, one for each native api (or at least
we'd want to provide something portable in addition to low level native
apis).

> The "Interim Licesne Policy" should be one of the bullet points
> under Package Requirements.  e.g.
>
>    "The package must be distributed under an acceptable license.  The
>     only license currently acceptable is BSD3 [rationale..]"

Yes and no :-). I'll say that it must comply with the license policy.
I've been trying very hard to avoid writing the sentence "the only
license acceptable is BSD3" because it's highly misleading to say that.

> I'd like to see something about API style mentioned in the "Package
> Requirements".  e.g.:
>
>    Packages admitted to the platform should follow an API style
>    similar to those packages already in the package.  While the
>    style guidelines for package APIs have not yet been written down,
>    we expect that to change in the future.

That's for the discussion after we agree on the procedure. There are
plenty more package requirements that we'll want to discuss. We've
deliberately started with a minimal set because we wanted to separate
the two discussions.

Ok, so I'll check the above with someone else on the steering committee
and update the document.

Duncan

_______________________________________________
Libraries mailing list
Libraries@...
http://www.haskell.org/mailman/listinfo/libraries

[Duncan Coutts]

On Tue, 2009-09-08 at 13:02 +0100, Simon Marlow wrote:

> Specific comments:
>
> "come to view on whether the package should be accepted"
> -> "help achieve a consensus ... "

We decided to clarify it slightly differently to make clear the
distinction between forming a view and working with others to get a
consensus.

- * come to view on whether the package should be accepted, either
    as-is or with certain modifications

+ * come to a view on whether the package should be accepted, either
    as-is or with certain modifications
+ * work with other reviewers to try to achieve a consensus view


> "the libraries mailing list reach consensus to accept it"
> -> "the reviewers reach a consensus to accept it" ?

Changed as you suggested. It's simpler and means the same thing since
everyone one the libs mailing list is considered as a reviewer.

> "Compile on all operating systems and compilers that the platform targets"
> -> ".. except where the package is compiler- or platform-specific" ?

We decided not to list explicit exceptions on this point but to make
clear the "should" status of all the requirements:

        Every package should fulfil the following requirements. Any
        requirements that are not met must be clearly explained and
        justified in the proposal.

So for a platform-specific package you have to point it out in the
proposal, justify it and have the reviewers agree with you.

> The "Interim Licesne Policy" should be one of the bullet points
> under Package Requirements.  e.g.
>
>    "The package must be distributed under an acceptable license.  The
>     only license currently acceptable is BSD3 [rationale..]"

Added to the requirements list:

      * Be distributed under an acceptable copyright license. See the
        interim license policy.

So the changes as a whole are:
http://trac.haskell.org/haskell-platform/wiki/AddingPackages?action=diff&version=9&old_version=5

Duncan

_______________________________________________
Libraries mailing list
Libraries@...
http://www.haskell.org/mailman/listinfo/libraries