Append behavior of __setitem__

[Barry Warsaw]

So I'm just starting to read RFC 5322 and I'm starting by skimming  
over Appendix A (differences between RFC 5322 and 2822).  I see this:

26.  No multiple occurrences of fields (except resent and received).*

Which i find very interesting, and possibly relevant to the discussion  
about changing the semantics of Message.__setitem__() to not append to  
the list of headers, as well as some of the other semantics of message  
headers (e.g. get_all()).

thinking-out-loud-ly y'rs,
-Barry



_______________________________________________
Email-SIG mailing list
Email-SIG@...
Your options: http://mail.python.org/mailman/options/email-sig/lists%40nabble.com

[Tony Nelson]

At 17:57 -0400 04/10/2009, Barry Warsaw wrote:
 ...
>So I'm just starting to read RFC 5322 and I'm starting by skimming
>over Appendix A (differences between RFC 5322 and 2822).

Oh, bother!

>I see this:
>
>26.  No multiple occurrences of fields (except resent and received).*
>
>Which i find very interesting, and possibly relevant to the discussion
>about changing the semantics of Message.__setitem__() to not append to
>the list of headers, as well as some of the other semantics of message
>headers (e.g. get_all()).

Thank you for mentioning this.  Darn it.
--
____________________________________________________________________
TonyN.:'                       <mailto:tonynelson@...>
      '                              <http://www.georgeanelson.com/>
_______________________________________________
Email-SIG mailing list
Email-SIG@...
Your options: http://mail.python.org/mailman/options/email-sig/lists%40nabble.com

[Tony Nelson]

(Fired too fast.)

At 18:39 -0400 04/10/2009, Tony Nelson wrote:
>At 17:57 -0400 04/10/2009, Barry Warsaw wrote:
> ...
>>So I'm just starting to read RFC 5322 and I'm starting by skimming
>>over Appendix A (differences between RFC 5322 and 2822).
>
>Oh, bother!

Appendix B?

 ...
>Thank you for mentioning this.  Darn it.

I note that there is also RFC 5321, "Simple Mail Transfer Protocol", which
obsoletes RFC 2821 and updates RFC 1123, "Registration of Mail and MIME
Header Fields".
--
____________________________________________________________________
TonyN.:'                       <mailto:tonynelson@...>
      '                              <http://www.georgeanelson.com/>
_______________________________________________
Email-SIG mailing list
Email-SIG@...
Your options: http://mail.python.org/mailman/options/email-sig/lists%40nabble.com

[Barry Warsaw]

On Apr 10, 2009, at 6:46 PM, Tony Nelson wrote:

> (Fired too fast.)
>
> At 18:39 -0400 04/10/2009, Tony Nelson wrote:
>> At 17:57 -0400 04/10/2009, Barry Warsaw wrote:
>> ...
>>> So I'm just starting to read RFC 5322 and I'm starting by skimming
>>> over Appendix A (differences between RFC 5322 and 2822).
>>
>> Oh, bother!
>
> Appendix B?

Oops, yep!

> ...
>> Thank you for mentioning this.  Darn it.
>
> I note that there is also RFC 5321, "Simple Mail Transfer Protocol",  
> which
> obsoletes RFC 2821 and updates RFC 1123, "Registration of Mail and  
> MIME

Yeah.  We'll let the smtplib.py people worry about that one <wink>.
-Barry



_______________________________________________
Email-SIG mailing list
Email-SIG@...
Your options: http://mail.python.org/mailman/options/email-sig/lists%40nabble.com

[Stephen J. Turnbull]

Barry Warsaw writes:

 > So I'm just starting to read RFC 5322 and I'm starting by skimming  
 > over Appendix A (differences between RFC 5322 and 2822).

I know Barry's a big supporter of the Postel Principle.  As a
guideline[1], how far back should we be lenient?  RFC 822 (no leading "2"
;-)?

Footnotes:
[1]  Presumably over time we'll accrete definitely non-conforming
practices that we need to accept and do something sane with (eg, we
can't just raise ArmageddonException because we get a header with
8-bit characters in it).  But I think we also should have a plan for
formerly acceptable syntax that has been restricted in more recent
RFCs, etc.


_______________________________________________
Email-SIG mailing list
Email-SIG@...
Your options: http://mail.python.org/mailman/options/email-sig/lists%40nabble.com

[Tony Nelson]

At 16:43 +0900 04/11/2009, Stephen J. Turnbull wrote:
>Barry Warsaw writes:
>
> > So I'm just starting to read RFC 5322 and I'm starting by skimming
> > over Appendix A (differences between RFC 5322 and 2822).
>
>I know Barry's a big supporter of the Postel Principle.  As a
>guideline[1], how far back should we be lenient?  RFC 822 (no leading "2"
>;-)?

Sure.  The header field should be parsed, if possible, and possibly add a
defect to the message.  For some header fields, the data should be added to
the previous Header instance; for others, an extra Header instance might
need to be created.

Message /generation/ should comply with what was in RFC 2822, where this
requirement was added, and also the new RFC 5322.


>Footnotes:
>[1]  Presumably over time we'll accrete definitely non-conforming
>practices that we need to accept and do something sane with (eg, we
>can't just raise ArmageddonException because we get a header with
>8-bit characters in it).  But I think we also should have a plan for
>formerly acceptable syntax that has been restricted in more recent
>RFCs, etc.

Any email parser must cope with both obsolete-* syntax and common bad
practices.  Python's already does in various places.
--
____________________________________________________________________
TonyN.:'                       <mailto:tonynelson@...>
      '                              <http://www.georgeanelson.com/>
_______________________________________________
Email-SIG mailing list
Email-SIG@...
Your options: http://mail.python.org/mailman/options/email-sig/lists%40nabble.com

[Barry Warsaw]

On Apr 11, 2009, at 3:43 AM, Stephen J. Turnbull wrote:

> Barry Warsaw writes:
>
>> So I'm just starting to read RFC 5322 and I'm starting by skimming
>> over Appendix A (differences between RFC 5322 and 2822).
>
> I know Barry's a big supporter of the Postel Principle.  As a
> guideline[1], how far back should we be lenient?  RFC 822 (no  
> leading "2"
> ;-)?
>
> Footnotes:
> [1]  Presumably over time we'll accrete definitely non-conforming
> practices that we need to accept and do something sane with (eg, we
> can't just raise ArmageddonException because we get a header with
> 8-bit characters in it).  But I think we also should have a plan for
> formerly acceptable syntax that has been restricted in more recent
> RFCs, etc.

We could potentially have strict and lenient modes, or possible RFC  
822, 2822, 5322 modes.  OTOH, I feel very strongly that the parser  
should accept just about any stream of bytes without throwing an  
exception.  Thinking about an application like Mailman, it's rather  
inconvenient for the parsing phase to throw any exception.  Much  
better is to register defects and then decide the disposition of  
messages based on the defect list.

OTOH, when creating messages from whole cloth, I think it's okay to  
raise exception.  You just have to be careful because often the same  
APIs are used by the parser.

Barry



_______________________________________________
Email-SIG mailing list
Email-SIG@...
Your options: http://mail.python.org/mailman/options/email-sig/lists%40nabble.com

[Barry Warsaw]

On Apr 11, 2009, at 5:17 PM, Tony Nelson wrote:

> Sure.  The header field should be parsed, if possible, and possibly  
> add a
> defect to the message.  For some header fields, the data should be  
> added to
> the previous Header instance; for others, an extra Header instance  
> might
> need to be created.

I don't follow this part.

> Message /generation/ should comply with what was in RFC 2822, where  
> this
> requirement was added, and also the new RFC 5322.

+1

-Barry



_______________________________________________
Email-SIG mailing list
Email-SIG@...
Your options: http://mail.python.org/mailman/options/email-sig/lists%40nabble.com

[Tony Nelson]

At 10:04 -0400 04/13/2009, Barry Warsaw wrote:
 ...
>We could potentially have strict and lenient modes, or possible RFC
>822, 2822, 5322 modes.

Is there any need to produce emails that don't conform to the latest spec?
Those specs are crafted to produce backward-compatible messages.

>OTOH, I feel very strongly that the parser
>should accept just about any stream of bytes without throwing an
>exception.  Thinking about an application like Mailman, it's rather
>inconvenient for the parsing phase to throw any exception.  Much
>better is to register defects and then decide the disposition of
>messages based on the defect list.
>
>OTOH,

The second other hand should be the Gripping hand, as it should be the
overriding point.

>when creating messages from whole cloth, I think it's okay to
>raise exception.  You just have to be careful because often the same
>APIs are used by the parser.

APIs raise exceptions, parser catches them, makes into defects?
--
____________________________________________________________________
TonyN.:'                       <mailto:tonynelson@...>
      '                              <http://www.georgeanelson.com/>
_______________________________________________
Email-SIG mailing list
Email-SIG@...
Your options: http://mail.python.org/mailman/options/email-sig/lists%40nabble.com

[Tony Nelson]

At 10:05 -0400 04/13/2009, Barry Warsaw wrote:

>On Apr 11, 2009, at 5:17 PM, Tony Nelson wrote:
>
>>Sure. The header field should be parsed, if possible, and possibly add a
>>defect to the message. For some header fields, the data should be added
>>to the previous Header instance; for others, an extra Header instance
>>might need to be created.
>
>I don't follow this part.

When a duplicate header field is parsed, errors are made into defects.  If
duplicates are not allowed for that header field, the contents should be
added to the previous header if that is possible (Subject:, just append
with whitespace; address headers, just append addresses), or a new
(improper) Header should be created if it is not possible to add to the
previous Header (Message-ID:, Content-Type:).

OK, those examples for extra Headers aren't good; it may be that they only
produce message defects.
--
____________________________________________________________________
TonyN.:'                       <mailto:tonynelson@...>
      '                              <http://www.georgeanelson.com/>
_______________________________________________
Email-SIG mailing list
Email-SIG@...
Your options: http://mail.python.org/mailman/options/email-sig/lists%40nabble.com