Sheep not obeying shepherds

[Brian E Carpenter]

(Allison copied, with her PROTO hat on.)

Hi,

It seems we've had one or two procedural glitches in which document authors
over-interpret Gen-ART reviews, leading for example to unexpected
new versions being posted after IESG approval, or to multiple edits
at AUTH48 to the surprise of the AD.

It shouldn't be so. Our FAQ says it shouldn't, by implication:

> Q: I have comments from a Gen-ART reviewer - do I revise the document immediately?
>
> A: Ask your AD/document shepherd. Practice varies between areas. A fair number
 > of comments area resolved with Notes to the RFC Editor. But the General Area
 > director will not ask for a re-spin before a telechat.

I suggest making this clearer:

A: No. Ask your AD/document shepherd, who may ask you to revise the document,
or the comments may be resolved with Notes to the RFC Editor. Alternatively,
the IESG may decide to approve the document without resolving the Gen-ART comments.
Gen-ART reviews are not authoritative, and in no case should you update the
document without consulting the AD/document shepherd.

The previous Q/A should also be tuned slightly:

> Q: Are Gen-ART comments binding?
>
> A: Not unless the General Area director agrees with the comments. Gen-ART
 > does not have a formal role in the standards approval process - we're
 > simply providing input for the General Area director who is deciding how
 > to ballot the document. If the General Area director agrees with a serious
 > concern expressed in a Gen-ART review, the General Area director may ask
>  the Gen-ART reviewer, "does the new version of the draft address your concerns?"

Suggested rewrite:

A: Only if IESG review makes them so, i.e. if the General Area director
agrees with the comments and incorporates them in the IESG ballot as
"DISCUSS" comments.  Gen-ART does not have a formal role in the standards
approval process - we're simply providing input for the General Area director
who is deciding how to ballot the document. If the General Area director
agrees with a serious concern expressed in a Gen-ART review, and uses it as
the basis for a "DISCUSS", the General Area director may later ask
the Gen-ART reviewer, "does the new version of the draft address your
concerns?" See the next question for when to produce a new version.

--------

A related point is that some Gen-ART reviewers choose to spend a lot of effort
on relatively minor editorial issues. Now this is always a matter of taste,
but in the light of the above, such editorial comments are *never* binding.
I quite often mention them in the tracker in case they can help the RFC Editor,
but unless they are grievous enough to kick the document back to the authors
(i.e. a DISCUSS), you might want to dial back your effort.

I realise this is in conflict with some words in the Gen-ART guidelines.
I'll send separately a proposed revision of those.

     Brian


_______________________________________________
Gen-art mailing list
Gen-art@...
https://www1.ietf.org/mailman/listinfo/gen-art

[Joel M. Halpern]

I think we want to be careful of tightening this too far.
If the author has to make a revision, and the authors thinks that a
suggestion from the gen-art team improves the text, then I would hate for
the author to feel that they could not make the change.

(This is for situations where a revision is appropriate anyway.  The issue
of extra later revisions or unanticipated auth48 changes is a different
matter.)

Yours,
Joel

At 09:02 AM 12/7/2005, Brian E Carpenter wrote:

>Q: Are Gen-ART comments binding?
>A: Not unless the General Area director agrees with the comments. Gen-ART
> > does not have a formal role in the standards approval process - we're
> > simply providing input for the General Area director who is deciding how
> > to ballot the document. If the General Area director agrees with a serious
> > concern expressed in a Gen-ART review, the General Area director may ask
>  the Gen-ART reviewer, "does the new version of the draft address your
> concerns?"
>
>Suggested rewrite:
>
>A: Only if IESG review makes them so, i.e. if the General Area director
>agrees with the comments and incorporates them in the IESG ballot as
>"DISCUSS" comments.  Gen-ART does not have a formal role in the standards
>approval process - we're simply providing input for the General Area director
>who is deciding how to ballot the document. If the General Area director
>agrees with a serious concern expressed in a Gen-ART review, and uses it as
>the basis for a "DISCUSS", the General Area director may later ask
>the Gen-ART reviewer, "does the new version of the draft address your
>concerns?" See the next question for when to produce a new version.

_______________________________________________
Gen-art mailing list
Gen-art@...
https://www1.ietf.org/mailman/listinfo/gen-art

[Spencer Dawkins]

Hi, Brian,

Responding as the author of most of the problematic text, and an
unintentional instigator in at least one over-interpretation (so please
filter appropriately)...

>> Q: I have comments from a Gen-ART reviewer - do I revise the document
>> immediately?
>>
>> A: Ask your AD/document shepherd. Practice varies between areas. A fair
>> number
> > of comments area resolved with Notes to the RFC Editor. But the General
> > Area
> > director will not ask for a re-spin before a telechat.
>
> I suggest making this clearer:
>
> A: No. Ask your AD/document shepherd, who may ask you to revise the
> document,
> or the comments may be resolved with Notes to the RFC Editor.
> Alternatively,
> the IESG may decide to approve the document without resolving the Gen-ART
> comments.
> Gen-ART reviews are not authoritative, and in no case should you update
> the
> document without consulting the AD/document shepherd.

I agree with your thought, of course. I think the thought is actually saying

A: Not unless your AD/document shepherd asks you to revise the document.
Some comments may be resolved with Notes to the RFC Editor, and the IESG may
decide to approve the document without resolving the Gen-ART comments.
Gen-ART reviews are not authoritative, and in no case should you update the
document without consulting the AD/document shepherd.

> The previous Q/A should also be tuned slightly:
>
>> Q: Are Gen-ART comments binding?
>>
>> A: Not unless the General Area director agrees with the comments. Gen-ART
> > does not have a formal role in the standards approval process - we're
> > simply providing input for the General Area director who is deciding how
> > to ballot the document. If the General Area director agrees with a
> > serious
> > concern expressed in a Gen-ART review, the General Area director may ask
>>  the Gen-ART reviewer, "does the new version of the draft address your
>> concerns?"
>
> Suggested rewrite:
>
> A: Only if IESG review makes them so, i.e. if the General Area director
> agrees with the comments and incorporates them in the IESG ballot as
> "DISCUSS" comments.  Gen-ART does not have a formal role in the standards
> approval process - we're simply providing input for the General Area
> director
> who is deciding how to ballot the document. If the General Area director
> agrees with a serious concern expressed in a Gen-ART review, and uses it
> as
> the basis for a "DISCUSS", the General Area director may later ask
> the Gen-ART reviewer, "does the new version of the draft address your
> concerns?" See the next question for when to produce a new version.

I've been asked "does the new version of the draft address your concerns?"
by other ADs (not just Brian), and suspect I am not unique. It might be more
accurate to say, "if an Area Director agrees with a serious concern
expressed in a Gen-ART review, and uses it as the basis for a "DISCUSS", the
Area Director may later ask ...", which covers this case as well.

Thanks,

Spencer


_______________________________________________
Gen-art mailing list
Gen-art@...
https://www1.ietf.org/mailman/listinfo/gen-art

[Spencer Dawkins]

>I think we want to be careful of tightening this too far.
> If the author has to make a revision, and the authors thinks that a
> suggestion from the gen-art team improves the text, then I would hate for
> the author to feel that they could not make the change.

(Sigh) I realized after looking at Joel's note that I had not actually said
what I think the point is - we would prefer not to generate several
different versions of the draft during the IESG review cycle, because it's
unproductive and confusing when thirteen different ADs are looking at four
different versions of a draft at the same time.

If this is, in fact, the point, it might be nice to add the explanation in
the FAQ - we are leaving it up to the AD/document shepherd to decide whether
a revision would be helpful or harmful during the review, and that's the
appropriate place for the responsibility.

Thanks,

SPencer


_______________________________________________
Gen-art mailing list
Gen-art@...
https://www1.ietf.org/mailman/listinfo/gen-art

[Brian E Carpenter]

Spencer Dawkins wrote:
>> I think we want to be careful of tightening this too far.
>> If the author has to make a revision, and the authors thinks that a
>> suggestion from the gen-art team improves the text, then I would hate
>> for the author to feel that they could not make the change.

But if they do it out of step with the AD, we get horrible
confusion about what the IESG has approved and what it hasn't.
The bug is when changes are made without the AD's advice and consent,
because that generates a race condition. I didn't want to give examples,
since that would look like criticism of particular reviewers - but we
have had examples. So using the AD as the semaphore (hopefully
not a spinlock) seems essential.

> (Sigh) I realized after looking at Joel's note that I had not actually
> said what I think the point is - we would prefer not to generate several
> different versions of the draft during the IESG review cycle, because
> it's unproductive and confusing when thirteen different ADs are looking
> at four different versions of a draft at the same time.

Exactly
>
> If this is, in fact, the point, it might be nice to add the explanation
> in the FAQ - we are leaving it up to the AD/document shepherd to decide
> whether a revision would be helpful or harmful during the review, and
> that's the appropriate place for the responsibility.

Yes. Send text?

     Brian


_______________________________________________
Gen-art mailing list
Gen-art@...
https://www1.ietf.org/mailman/listinfo/gen-art

[Spencer Dawkins]

Hi, Brian,

>> If this is, in fact, the point, it might be nice to add the explanation
>> in the FAQ - we are leaving it up to the AD/document shepherd to decide
>> whether a revision would be helpful or harmful during the review, and
>> that's the appropriate place for the responsibility.
>
> Yes. Send text?

Working from my proposal this morning,  I have changed my suggested text to
be...

A: Not unless your AD/document shepherd asks you to revise the document.
Some comments may be resolved with Notes to the RFC Editor, and the IESG may
choose to approve the document without requiring changes to address all
Gen-ART comments, since Gen-ART reviews are not a formal part of the
approval review cycle. The AD/document shepherd may also prefer to complete
the entire approval review cycle before requesting an updated version of the
document, in order to avoid confusion about exactly what version is being
last-called, reviewed and approved. In no case should you submit an updated
version of the document without consulting the AD/document shepherd.

Thanks,

Spencer


_______________________________________________
Gen-art mailing list
Gen-art@...
https://www1.ietf.org/mailman/listinfo/gen-art

[Brian E Carpenter]

Those are fine suggestions

    Brian


Spencer Dawkins wrote:

> Hi, Brian,
>
> Responding as the author of most of the problematic text, and an
> unintentional instigator in at least one over-interpretation (so please
> filter appropriately)...
>
>>> Q: I have comments from a Gen-ART reviewer - do I revise the document
>>> immediately?
>>>
>>> A: Ask your AD/document shepherd. Practice varies between areas. A
>>> fair number
>>
>> > of comments area resolved with Notes to the RFC Editor. But the
>> General > Area
>> > director will not ask for a re-spin before a telechat.
>>
>> I suggest making this clearer:
>>
>> A: No. Ask your AD/document shepherd, who may ask you to revise the
>> document,
>> or the comments may be resolved with Notes to the RFC Editor.
>> Alternatively,
>> the IESG may decide to approve the document without resolving the
>> Gen-ART comments.
>> Gen-ART reviews are not authoritative, and in no case should you
>> update the
>> document without consulting the AD/document shepherd.
>
>
> I agree with your thought, of course. I think the thought is actually
> saying
>
> A: Not unless your AD/document shepherd asks you to revise the document.
> Some comments may be resolved with Notes to the RFC Editor, and the IESG
> may decide to approve the document without resolving the Gen-ART
> comments. Gen-ART reviews are not authoritative, and in no case should
> you update the document without consulting the AD/document shepherd.
>
>> The previous Q/A should also be tuned slightly:
>>
>>> Q: Are Gen-ART comments binding?
>>>
>>> A: Not unless the General Area director agrees with the comments.
>>> Gen-ART
>>
>> > does not have a formal role in the standards approval process - we're
>> > simply providing input for the General Area director who is deciding
>> how
>> > to ballot the document. If the General Area director agrees with a >
>> serious
>> > concern expressed in a Gen-ART review, the General Area director may
>> ask
>>
>>>  the Gen-ART reviewer, "does the new version of the draft address
>>> your concerns?"
>>
>>
>> Suggested rewrite:
>>
>> A: Only if IESG review makes them so, i.e. if the General Area director
>> agrees with the comments and incorporates them in the IESG ballot as
>> "DISCUSS" comments.  Gen-ART does not have a formal role in the standards
>> approval process - we're simply providing input for the General Area
>> director
>> who is deciding how to ballot the document. If the General Area director
>> agrees with a serious concern expressed in a Gen-ART review, and uses
>> it as
>> the basis for a "DISCUSS", the General Area director may later ask
>> the Gen-ART reviewer, "does the new version of the draft address your
>> concerns?" See the next question for when to produce a new version.
>
>
> I've been asked "does the new version of the draft address your
> concerns?" by other ADs (not just Brian), and suspect I am not unique.
> It might be more accurate to say, "if an Area Director agrees with a
> serious concern expressed in a Gen-ART review, and uses it as the basis
> for a "DISCUSS", the Area Director may later ask ...", which covers this
> case as well.
>
> Thanks,
>
> Spencer
>
> _______________________________________________
> Gen-art mailing list
> Gen-art@...
> https://www1.ietf.org/mailman/listinfo/gen-art
>

_______________________________________________
Gen-art mailing list
Gen-art@...
https://www1.ietf.org/mailman/listinfo/gen-art

[Scott Brim]

On 12/07/2005 10:04 AM, Spencer Dawkins allegedly wrote:
>> A: Only if IESG review makes them so, i.e. if the General Area
director
>> agrees with the comments and incorporates them in the IESG ballot as
>> "DISCUSS" comments.  Gen-ART does not have a formal role in the
standards
>> approval process - we're simply providing input for the General Area
>> director
>> who is deciding how to ballot the document. If the General Area
director

>> agrees with a serious concern expressed in a Gen-ART review, and uses
>> it as
>> the basis for a "DISCUSS", the General Area director may later ask
>> the Gen-ART reviewer, "does the new version of the draft address your
>> concerns?" See the next question for when to produce a new version.
>
>
> I've been asked "does the new version of the draft address your
> concerns?" by other ADs (not just Brian), and suspect I am not unique.
> It might be more accurate to say, "if an Area Director agrees with a
> serious concern expressed in a Gen-ART review, and uses it as the basis
> for a "DISCUSS", the Area Director may later ask ...", which covers

this
> case as well.

Right.  Other ADs may pick up the comments.

The only other comment I have is that I don't think we want to say
editorial fix-ups are nonproductive, because they can be used assuming
the draft goes around one more time.  Rather, purely editorial changes
are not productive if that is all the review consists of.

  However, a review which consists only of copy-editing is not
  productive.  If the reviewer feels that a draft is too badly written
  to advance, it will be sufficient to say so with one or two
  examples.

 swb

_______________________________________________
Gen-art mailing list
Gen-art@...
https://www1.ietf.org/mailman/listinfo/gen-art

[Brian E Carpenter]

Scott W Brim wrote:

> On 12/07/2005 10:04 AM, Spencer Dawkins allegedly wrote:
>
>>>A: Only if IESG review makes them so, i.e. if the General Area
>
> director
>
>>>agrees with the comments and incorporates them in the IESG ballot as
>>>"DISCUSS" comments.  Gen-ART does not have a formal role in the
>
> standards
>
>>>approval process - we're simply providing input for the General Area
>>>director
>>>who is deciding how to ballot the document. If the General Area
>
> director
>
>>>agrees with a serious concern expressed in a Gen-ART review, and uses
>>>it as
>>>the basis for a "DISCUSS", the General Area director may later ask
>>>the Gen-ART reviewer, "does the new version of the draft address your
>>>concerns?" See the next question for when to produce a new version.
>>
>>
>>I've been asked "does the new version of the draft address your
>>concerns?" by other ADs (not just Brian), and suspect I am not unique.
>>It might be more accurate to say, "if an Area Director agrees with a
>>serious concern expressed in a Gen-ART review, and uses it as the basis
>>for a "DISCUSS", the Area Director may later ask ...", which covers
>
> this
>
>>case as well.
>
>
> Right.  Other ADs may pick up the comments.
>
> The only other comment I have is that I don't think we want to say
> editorial fix-ups are nonproductive, because they can be used assuming
> the draft goes around one more time.  Rather, purely editorial changes
> are not productive if that is all the review consists of.
>
>   However, a review which consists only of copy-editing is not
>   productive.  If the reviewer feels that a draft is too badly written
>   to advance, it will be sufficient to say so with one or two
>   examples.

OK
    Brian


_______________________________________________
Gen-art mailing list
Gen-art@...
https://www1.ietf.org/mailman/listinfo/gen-art

[Brian E Carpenter]

Spencer Dawkins wrote:

> Hi, Brian,
>
>>> If this is, in fact, the point, it might be nice to add the
>>> explanation in the FAQ - we are leaving it up to the AD/document
>>> shepherd to decide whether a revision would be helpful or harmful
>>> during the review, and that's the appropriate place for the
>>> responsibility.
>>
>>
>> Yes. Send text?
>
>
> Working from my proposal this morning,  I have changed my suggested text
> to be...
>
> A: Not unless your AD/document shepherd asks you to revise the document.
> Some comments may be resolved with Notes to the RFC Editor, and the IESG
> may choose to approve the document without requiring changes to address
> all Gen-ART comments, since Gen-ART reviews are not a formal part of the
> approval review cycle. The AD/document shepherd may also prefer to
> complete the entire approval review cycle before requesting an updated
> version of the document, in order to avoid confusion about exactly what
> version is being last-called, reviewed and approved. In no case should
> you submit an updated version of the document without consulting the
> AD/document shepherd.

OK.

     Brian


_______________________________________________
Gen-art mailing list
Gen-art@...
https://www1.ietf.org/mailman/listinfo/gen-art