[Fwd: Re: Bug in BODYSTRUCTURE]

[tanstaafl_bh]

Fyi...

Sam Varshavik (courier-imap developer) claims (on the courier-imap list,
see attached) dovecot is incorrectly parsing messages with
missing/invalid "MIME-Version:" header...

I have no clue if he is correct... if so, maybe this has already been fixed?

Chris St Denis writes:

> Sam Varshavchik wrote:
>> Chris St Denis writes:
>>
>>> Courier-imap is incorrectly parsing the message structure of some
>>> emails.
>>
>> Courier-IMAP parsed your sample message correctly, according to RFC 2822.
>>
>> This appears to be a bug in your "Barracuda spam firewall" product,
>> which corrupted the original headers, when it processed it.
>
> What header(s) in particular are wrong and what should they be for it to
> be correct? I will pass the data as a bug report to Barracuda and get
> this resolved.

An RFC 2822 message consists of one or more header lines, then an empty
line, then followed by the body of the message. Then, in order for a message
to be a valid MIME message, it must include the MIME-Version: header. See
section 4 of RFC 2045. It's clear, and unambiguous. A MIME-Version: header
must be present. Your sample message does not contain a "MIME-Version:"
header, in the header portion of the message; as such it is not a MIME
message. Without a valid MIME-Version: header present, none of the MIME
headers, including Content-Type: carry any meaning.

There is a line in your message that reads "MIME-Version:", however it is
not a part of this message's header portion. The message's headers precede
the first empty line of the message, see above. In the example message
"MIME-Version:" occurs after the first empty line.

If you actually examine the message closely, Barracuda inserted its junk *in
the middle* of an existing References: header! After all of that garbage,
you can see what's obviously the last line of the original References:
header, containing the last message ID, followed by a "Mime-Version: 1.0".
However, since the junk inserted by Barracuda included a bunch of empty
lines, everything below that junk is considered a part of the message's
contents, and not its headers.

Ready!… Fire!… Aim???

> However this situation does appear to be specific to courier-imap.
> Dovecot is able to parse it

If so, it violates RFC 2045, section 4. Its wording is clear:

   Messages composed in accordance with this document MUST include such
   a header field, with the following verbatim text:

     MIME-Version: 1.0

   The presence of this header field is an assertion that the message
   has been composed in compliance with this document.

If so, it fails to check for the presence of the MIME-Version: header, so it
processes the Content-Type: header even if MIME-Version: is missing.

>                              and Thunderbird (with courier-imap as the
> server) displays it correctly (therefore it must not use BODYSTRUCTURE).

Correct. Thunderbird does not use BODYSTRUCTURE. And, it has the same sloppy
logic as Dovecot.

This is somewhat sad. Internet standards are supposed to have some meaning.
I could see ignoring something that's may be burdensome or onerous, but this
is basic, elementary stuff.




------------------------------------------------------------------------------
Come build with us! The BlackBerry® Developer Conference in SF, CA
is the only developer event you need to attend this year. Jumpstart your
developing skills, take BlackBerry mobile applications to market and stay
ahead of the curve. Join us from November 9-12, 2009. Register now!
http://p.sf.net/sfu/devconf
_______________________________________________
Courier-imap mailing list
Courier-imap@...
Unsubscribe: https://lists.sourceforge.net/lists/listinfo/courier-imap

[Jakob Hirsch]

Charles Marcus, 2009-11-10 12:52:

> Sam Varshavik (courier-imap developer) claims (on the courier-imap list,
> see attached) dovecot is incorrectly parsing messages with
> missing/invalid "MIME-Version:" header...

Since when is being robust incorrect?

>> However this situation does appear to be specific to courier-imap.
>> Dovecot is able to parse it
>
> If so, it violates RFC 2045, section 4. Its wording is clear:
>
>   Messages composed in accordance with this document MUST include such
>   a header field, with the following verbatim text:
>
>     MIME-Version: 1.0
>
>   The presence of this header field is an assertion that the message
>   has been composed in compliance with this document.
>
> If so, it fails to check for the presence of the MIME-Version: header,
> so it processes the Content-Type: header even if MIME-Version: is missing.

Well, that is a MUST for the sender part (that creates the message), but
the RFC (or at least the part that Sam Varshavchik quoted) does not say
"you are not allowed to handle a MIME message if some part of it is
broken". You can does, if you like to be a pedantic bureaucrat, but I
doubt that this improves the user experience.

> This is somewhat sad. Internet standards are supposed to have some
> meaning. I could see ignoring something that's may be burdensome or
> onerous, but this is basic, elementary stuff.

It's actually the other way round: It's harder to write software that
follows the robustness principle, "be liberal in what you accept from
others". Sam's ranting would apply to violations of the other part, "be
conservative in what you do".

[Timo Sirainen]

I fully agree with Jakob. Actually this is a somewhat recent change  
(v1.1.alpha2). Mime-Version: was previously required, but there were  
enough broken mails that I decided to make this optional in code (but  
not by admin):

         /* Buggy software creates Content-Type: headers without Mime-
Version:
            header. By default we allow this and assume message is  
MIME if
            Content-Type: is found. This flag disables this. */
         MESSAGE_PARSER_FLAG_MIME_VERSION_STRICT = 0x02

On Nov 10, 2009, at 8:02 AM, Jakob Hirsch wrote:

> Charles Marcus, 2009-11-10 12:52:
>
>> Sam Varshavik (courier-imap developer) claims (on the courier-imap  
>> list,
>> see attached) dovecot is incorrectly parsing messages with
>> missing/invalid "MIME-Version:" header...
>
> Since when is being robust incorrect?
>
>>> However this situation does appear to be specific to courier-imap.
>>> Dovecot is able to parse it
>>
>> If so, it violates RFC 2045, section 4. Its wording is clear:
>>
>>  Messages composed in accordance with this document MUST include such
>>  a header field, with the following verbatim text:
>>
>>    MIME-Version: 1.0
>>
>>  The presence of this header field is an assertion that the message
>>  has been composed in compliance with this document.
>>
>> If so, it fails to check for the presence of the MIME-Version:  
>> header,
>> so it processes the Content-Type: header even if MIME-Version: is  
>> missing.
>
> Well, that is a MUST for the sender part (that creates the message),  
> but
> the RFC (or at least the part that Sam Varshavchik quoted) does not  
> say
> "you are not allowed to handle a MIME message if some part of it is
> broken". You can does, if you like to be a pedantic bureaucrat, but I
> doubt that this improves the user experience.
>
>> This is somewhat sad. Internet standards are supposed to have some
>> meaning. I could see ignoring something that's may be burdensome or
>> onerous, but this is basic, elementary stuff.
>
> It's actually the other way round: It's harder to write software that
> follows the robustness principle, "be liberal in what you accept from
> others". Sam's ranting would apply to violations of the other part,  
> "be
> conservative in what you do".
>

[tanstaafl_bh]

On 11/10/2009, Timo Sirainen (tss@...) wrote:
> I fully agree with Jakob. Actually this is a somewhat recent change
> (v1.1.alpha2). Mime-Version: was previously required, but there were
> enough broken mails that I decided to make this optional in code (but
> not by admin):

Good enough for me... I just wanted confirmation... thx!