double angle brackets

[Ernie Dainow]

One of the key goals in simplifying Downgrade is to drop the double
angle bracket notation, due to compatibility and security concerns.
There have been a few email threads on this. I think the following
summarizes the current thinking.

Dropping double angle brackets on Recipient addresses is part of the
downgrade simplification to not support downgrade of forward pointing
addresses (considering these cases as configuration errors).

To replace double angle brackets for the Sender, John Klensin proposed
using multiple From fields. SM amended this with Reply-To to specify the
preferred address, as in
        From: EAI-addr, ASCII-addr
        Reply-To: EAI-addr
where the EAI-addr gets dropped in Downgrade.

I did some (non-EAI) tests of multiple From fields with Reply-To on two
different email systems and both worked as expected. This looks like an
elegant solution.

SM raised some possible problems with multiple From addresses on mailing
lists. The suggestion was to address these issues in the mailinglist
document.

There still needs to be a mechanism to carry the alternate address of
the sender in the envelope through SMTP relays up until the point that a
non-EAI transport is encountered. No change is being proposed to the EAI
SMTP extension for the ALT-ADDRESS parameter on MAIL FROM. If a non-EAI
server is encountered, Downgrade is done on the message and the EAI
address in the Envelope is replaced with the ALT-ADDRESS. However, we
would drop the ALT-ADDRESS parameter on RCPT TO.

    -Ernie Dainow





_______________________________________________
IMA mailing list
IMA@...
https://www.ietf.org/mailman/listinfo/ima

[Charles Lindsey]

On Fri, 04 Sep 2009 22:46:26 +0100, Ernie Dainow <edainow@...>  
wrote:

> One of the key goals in simplifying Downgrade is to drop the double  
> angle bracket notation, due to compatibility and security concerns.  
> There have been a few email threads on this. I think the following  
> summarizes the current thinking.
>
> Dropping double angle brackets on Recipient addresses is part of the  
> downgrade simplification to not support downgrade of forward pointing  
> addresses (considering these cases as configuration errors).
>
> To replace double angle brackets for the Sender, John Klensin proposed  
> using multiple From fields. SM amended this with Reply-To to specify the  
> preferred address, as in
>         From: EAI-addr, ASCII-addr
>         Reply-To: EAI-addr
> where the EAI-addr gets dropped in Downgrade.
>
> I did some (non-EAI) tests of multiple From fields with Reply-To on two  
> different email systems and both worked as expected. This looks like an  
> elegant solution.

Yes, but the interesting case is multiple fields in Reply-To. Normally,  
these will cause the reply to go to _both_ (but if one is utf-8 and one is  
ascii, then the utf-8 might fail).

If we retain the double angle brackets, then one could still say, in the  
Reply-To, "please reply to my utf-8 address if possible, but if your  
system (or some intermediate server that knows how to downgrade) can't do  
that, then please reply to my ascii address".

I would expect that to be the commonest (and most useful) situation in  
which those double angle brackets will appear in practice.

So my preference would be to retain them, at least so long as these remain  
Experimental documents. If the eventual Standard still retains them, then  
it could still say "this feature may well be withdrawn in a future version  
of this standard" (with the intent to do so when utf-8 has become so  
widely implemented that it is no longer needed).

--
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: [phone]                       
   Web: http://www.cs.man.ac.uk/~chl
Email: chl@...      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5
_______________________________________________
IMA mailing list
IMA@...
https://www.ietf.org/mailman/listinfo/ima

[John C Klensin-3]

--On Sunday, September 06, 2009 12:58 PM +0100 Charles Lindsey
<chl@...> wrote:

> On Fri, 04 Sep 2009 22:46:26 +0100, Ernie Dainow
> <edainow@...> wrote:
>
>> One of the key goals in simplifying Downgrade is to drop the
>> double   angle bracket notation, due to compatibility and
>> security concerns.   There have been a few email threads on
>> this. I think the following   summarizes the current thinking.
>>
>> Dropping double angle brackets on Recipient addresses is part
>> of the   downgrade simplification to not support downgrade of
>> forward pointing   addresses (considering these cases as
>> configuration errors).
>>
>> To replace double angle brackets for the Sender, John Klensin
>> proposed   using multiple From fields. SM amended this with
>> Reply-To to specify the   preferred address, as in
>>         From: EAI-addr, ASCII-addr
>>         Reply-To: EAI-addr
>> where the EAI-addr gets dropped in Downgrade.
>>
>> I did some (non-EAI) tests of multiple From fields with
>> Reply-To on two   different email systems and both worked as
>> expected. This looks like an   elegant solution.
>
> Yes, but the interesting case is multiple fields in Reply-To.
> Normally, these will cause the reply to go to _both_ (but if
> one is utf-8 and one is ascii, then the utf-8 might fail).
>
> If we retain the double angle brackets, then one could still
> say, in the Reply-To, "please reply to my utf-8 address if
> possible, but if your system (or some intermediate server that
> knows how to downgrade) can't do that, then please reply to my
> ascii address".

And this is different from adding "if this address doesn't work,
try that other one" to the email model -- something we have
never had, nor missed very much?

The cost and risks of adding new syntax to addresses is quite
high, especially given the risk of leakage and of causing
serious interoperability problems if things do leak.   It may be
worth it (I'm still trying to keep an open mind although I'm
increasingly doubtful that it is), but, if so, we should have a
much stronger justification than a marginal reply-to case, at
least IMO.

> I would expect that to be the commonest (and most useful)
> situation in which those double angle brackets will appear in
> practice.
>
> So my preference would be to retain them, at least so long as
> these remain Experimental documents. If the eventual Standard
> still retains them, then it could still say "this feature may
> well be withdrawn in a future version of this standard" (with
> the intent to do so when utf-8 has become so widely
> implemented that it is no longer needed).

Our experience has been that, if one does that, it will only
affect new implementations created after the feature is
withdrawn and maybe, for competitive reasons, not even them.
For everyone else, if a feature like that is ever supported as
part of a standard, it is forever.

    john\
_______________________________________________
IMA mailing list
IMA@...
https://www.ietf.org/mailman/listinfo/ima

[sm-7]

At 14:46 04-09-2009, Ernie Dainow wrote:
>One of the key goals in simplifying Downgrade is to drop the double
>angle bracket notation, due to compatibility and security concerns.
>There have been a few email threads on this. I think the following
>summarizes the current thinking.

Yes.

At 04:58 06-09-2009, Charles Lindsey wrote:
>If we retain the double angle brackets, then one could still say, in the
>Reply-To, "please reply to my utf-8 address if possible, but if your
>system (or some intermediate server that knows how to downgrade) can't do
>that, then please reply to my ascii address".
>
>I would expect that to be the commonest (and most useful) situation in
>which those double angle brackets will appear in practice.

Ernie summarizes what may be considered in my opinion as easier, i.e.
lower amount of changes to existing syntax.

>So my preference would be to retain them, at least so long as these remain
>Experimental documents. If the eventual Standard still retains them, then
>it could still say "this feature may well be withdrawn in a future version
>of this standard" (with the intent to do so when utf-8 has become so
>widely implemented that it is no longer needed).

If the decision is taken to retain them in an existing Proposed
Standard, it will be more difficult to withdraw that feature even
though there is a statement about it.

Regards,
-sm

_______________________________________________
IMA mailing list
IMA@...
https://www.ietf.org/mailman/listinfo/ima

[Shawn Steele]

> To replace double angle brackets for the Sender, John Klensin proposed
> using multiple From fields. SM amended this with Reply-To to specify the
> preferred address, as in
>         From: EAI-addr, ASCII-addr
>         Reply-To: EAI-addr
> where the EAI-addr gets dropped in Downgrade.

I'd imagine that "something" isn't going to be happy with 2 addresses.

It seems to me that 99% of the time email can be viewed as primarily "sender side" or "recipient side".  Eg: Mail goes from a sender system to a recipient system.  It is very rare (now) that an in-between system would relay mail from an untrusted server to another 3rd party server.  If there is a relay, it'd have to trust the sender, so it's pretty much "sender side," or recognize the recipient "recipient side."

Assuming that a "sender" has an EAI aware system that is completely conformant and "works" with UTF-8 addresses, then the entire "sender side" would have to be updated to support UTF-8 addresses.  If the sender's relays aren't upgraded, then all of their mail ends up being downgraded anyway.

Similarly a recipient either has an entirely EAI aware system, or they don't.  

So I'm wondering how interesting downgrade really is.  If my sending system trusts the sender, presumably it could request an ASCII address from the trusted sender-side server if it needed to downgrade.  (Through some other mechanism, like asking the trusted server what other aliases were valid for that account using some new, or even proprietary, protocol).

So the proposed solution seems like overkill.

VERY worst case a client (like Outlook) can try to send an EAI email.  If it bounces, the client could resend using my ASCII address (assuming it is properly configured).  It could even do that silently, or with maybe a little warning saying that had been downgraded.

-Shawn

_______________________________________________
IMA mailing list
IMA@...
https://www.ietf.org/mailman/listinfo/ima

[John C Klensin-3]

--On Tuesday, September 08, 2009 16:51 +0000 Shawn Steele
<Shawn.Steele@...> wrote:

>> To replace double angle brackets for the Sender, John Klensin
>> proposed  using multiple From fields. SM amended this with
>> Reply-To to specify the  preferred address, as in
>>         From: EAI-addr, ASCII-addr
>>         Reply-To: EAI-addr
>> where the EAI-addr gets dropped in Downgrade.
>
> I'd imagine that "something" isn't going to be happy with 2
> addresses.

Right.  In particular, there are a lot of systems that do not
support multiple-address From:, multiple-address Reply-to:, or
both.

> It seems to me that 99% of the time email can be viewed as
> primarily "sender side" or "recipient side".  Eg: Mail goes
> from a sender system to a recipient system.  It is very rare
> (now) that an in-between system would relay mail from an
> untrusted server to another 3rd party server.  If there is a
> relay, it'd have to trust the sender, so it's pretty much
> "sender side," or recognize the recipient "recipient side."
>
> Assuming that a "sender" has an EAI aware system that is
> completely conformant and "works" with UTF-8 addresses, then
> the entire "sender side" would have to be updated to support
> UTF-8 addresses.  If the sender's relays aren't upgraded, then
> all of their mail ends up being downgraded anyway.
>
> Similarly a recipient either has an entirely EAI aware system,
> or they don't.  
>
> So I'm wondering how interesting downgrade really is.  If my
> sending system trusts the sender, presumably it could request
> an ASCII address from the trusted sender-side server if it
> needed to downgrade.  (Through some other mechanism, like
> asking the trusted server what other aliases were valid for
> that account using some new, or even proprietary, protocol).

This is the "do we really need downgrade" argument that some of
us have been raising on and off for years.  The answer has never
been "yes" or "no" but "are there enough edge cases (that do not
conform to the analysis) out there to make downgrading useful
anyway, at some cost level".   If anything has changed recently
it has been either

        * a perception that there are fewer relevant cases where
        downgrade is actually needed than was originally
        believed.
       
        * a perception that the costs and risks of having a
        downgrading arrangement (e.g., with double angle
        brackets) make the number of relevant cases needed to
        justify the mechanism much larger than we had originally
        believed.

Of course, both may be true, but would make the case for "no
downgrade" even more open and shut.

> So the proposed solution seems like overkill.
>
> VERY worst case a client (like Outlook) can try to send an EAI
> email.  If it bounces, the client could resend using my ASCII
> address (assuming it is properly configured).  It could even
> do that silently, or with maybe a little warning saying that
> had been downgraded.

That, of course, means that it knows your ASCII address.  Absent
some configuration changes, it might not.

But I think we are in violent agreement, or at least getting
close to it.

    john

_______________________________________________
IMA mailing list
IMA@...
https://www.ietf.org/mailman/listinfo/ima

[Shawn Steele]

> * a perception that there are fewer relevant cases where
> downgrade is actually needed than was originally
> believed.

For me, this is true
       
> * a perception that the costs and risks of having a
> downgrading arrangement (e.g., with double angle
> brackets) make the number of relevant cases needed to
> justify the mechanism much larger than we had originally
> believed.

And so I think the risk is more interesting now that I think the downgrade is less helpful :)

> That, of course, means that it knows your ASCII address.  Absent
> some configuration changes, it might not.

If I don't know my ASCII address, I'm not sure how I can send From: EAI, Latn or other syntax :)  So I don't think that's an interesting point.

- Shawn
_______________________________________________
IMA mailing list
IMA@...
https://www.ietf.org/mailman/listinfo/ima

[Ernie Dainow]

John C Klensin wrote:

> --On Tuesday, September 08, 2009 16:51 +0000 Shawn Steele
> <Shawn.Steele@...> wrote:
>
>  
>>> To replace double angle brackets for the Sender, John Klensin
>>> proposed  using multiple From fields. SM amended this with
>>> Reply-To to specify the  preferred address, as in
>>>         From: EAI-addr, ASCII-addr
>>>         Reply-To: EAI-addr
>>> where the EAI-addr gets dropped in Downgrade.
>>>      
>> I'd imagine that "something" isn't going to be happy with 2
>> addresses.
>>    
>
> Right.  In particular, there are a lot of systems that do not
> support multiple-address From:, multiple-address Reply-to:, or
> both.
>
>  

I don't think it's important if a lot of existing systems do not support
multiple from addresses.  As non-EAI systems they would only receive
downgraded messages, which have only a single From address and no Reply-To.

EAI systems, which will receive multiple From addresses, would have to
support it since it would be part of the EAI spec.

    -Ernie Dainow



_______________________________________________
IMA mailing list
IMA@...
https://www.ietf.org/mailman/listinfo/ima

[Shawn Steele]

How do you tell multiple from meaning EAI apart from non-EAI multiple from?

-----Original Message-----
From: Ernie Dainow [mailto:edainow@...]
Sent: Tuesday, September 08,  2009 14:30
To: John C Klensin
Cc: Shawn Steele; ima@...
Subject: Re: [EAI] double angle brackets



John C Klensin wrote:

> --On Tuesday, September 08, 2009 16:51 +0000 Shawn Steele
> <Shawn.Steele@...> wrote:
>
>  
>>> To replace double angle brackets for the Sender, John Klensin
>>> proposed  using multiple From fields. SM amended this with
>>> Reply-To to specify the  preferred address, as in
>>>         From: EAI-addr, ASCII-addr
>>>         Reply-To: EAI-addr
>>> where the EAI-addr gets dropped in Downgrade.
>>>      
>> I'd imagine that "something" isn't going to be happy with 2
>> addresses.
>>    
>
> Right.  In particular, there are a lot of systems that do not
> support multiple-address From:, multiple-address Reply-to:, or
> both.
>
>  

I don't think it's important if a lot of existing systems do not support
multiple from addresses.  As non-EAI systems they would only receive
downgraded messages, which have only a single From address and no Reply-To.

EAI systems, which will receive multiple From addresses, would have to
support it since it would be part of the EAI spec.

    -Ernie Dainow




_______________________________________________
IMA mailing list
IMA@...
https://www.ietf.org/mailman/listinfo/ima

[John C Klensin-3]

--On Tuesday, September 08, 2009 22:04 +0000 Shawn Steele
<Shawn.Steele@...> wrote:

> How do you tell multiple from meaning EAI apart from non-EAI
> multiple from?

I don't think you need to.

If I correctly understand the point that Ernie was trying to
make, there are the following cases:

        * non-EAI, support for multiple "From:": will accept
        both and probably discard the i18n email address.
       
        * non-EAI, no support for multiple "From:": Will reject
        either both addresses or the second one.  If one is
        accepted, probably the first one (which should probably
        be the ASCII address) will probably be used.  If both
        are rejected, then nothing much is lost compared to the
        failed downgrade case -- especially if you remember that
        an EAI envelope is unlikely to reach a non-EAI system.
       
        * EAI compliant system: will support multiple "From:"
        because the spec will require it.  

Then the EAI-compliant system will either respond to both
addresses on "reply" or to just the EAI one, but that is
consistent with how much we've historically underspecified what
to do with multiple "From:" situations.

The analysis for Reply-to won't be significantly different, IMO.

   john

_______________________________________________
IMA mailing list
IMA@...
https://www.ietf.org/mailman/listinfo/ima

[Shawn Steele]

> * non-EAI, support for multiple "From:": will accept
> both and probably discard the i18n email address.

That's an interesting assumption.  Presumably I could also reject the entire message since I have no clue how to parse the address.
       
> * non-EAI, no support for multiple "From:": Will reject
> either both addresses or the second one.  If one is
> accepted, probably the first one (which should probably
> be the ASCII address) will probably be used.  If both
> are rejected, then nothing much is lost compared to the
> failed downgrade case -- especially if you remember that
> an EAI envelope is unlikely to reach a non-EAI system.

Ditto.
       
> * EAI compliant system: will support multiple "From:"
> because the spec will require it.  

I still can't tell that they're a pair.  It could be intentionally from 2 different people, one of which has an EAI address, but the other has an ASCII address, and we don't know (or didn't bother to mention) what the EAI address user's downgrade address is.

So an EAI compliant system that "assumed" that "From: Unicode, ASCII" was EAI would not bother replying to the ASCII address, &/or it would

Besides, if this was reasonable for From, what about To: or CC: or Reply-To:?  Presumably I could mail it to other EAI users as well, so how does a recipient respond?  How would one reliably match up the pairs?

Also, presumably I know my own downgrade address, but am I going to know all of the downgrade addresses for everyone on To: (or hopefully Bcc:) that I'm forwarding a joke to?

I remain unconvinced that this solves enough downgrade problems completely enough to be interesting.

-Shawn

_______________________________________________
IMA mailing list
IMA@...
https://www.ietf.org/mailman/listinfo/ima

[Ernie Dainow]

Shawn Steele wrote:

>> * non-EAI, support for multiple "From:": will accept
>> both and probably discard the i18n email address.
>>    
>
> That's an interesting assumption.  Presumably I could also reject the entire message since I have no clue how to parse the address.
>
>  
>> * non-EAI, no support for multiple "From:": Will reject
>> either both addresses or the second one.  If one is
>> accepted, probably the first one (which should probably
>> be the ASCII address) will probably be used.  If both
>> are rejected, then nothing much is lost compared to the
>> failed downgrade case -- especially if you remember that
>> an EAI envelope is unlikely to reach a non-EAI system.
>>    
>
> Ditto.
>  

For non-EAI, downgrade would remove the i18n email address, so From does
not have multiple addresses.

>
>  
>> * EAI compliant system: will support multiple "From:"
>> because the spec will require it.  
>>    
>
> I still can't tell that they're a pair.  It could be intentionally from 2 different people, one of which has an EAI address, but the other has an ASCII address, and we don't know (or didn't bother to mention) what the EAI address user's downgrade address is.
>
> So an EAI compliant system that "assumed" that "From: Unicode, ASCII" was EAI would not bother replying to the ASCII address, &/or it would
>
>  

This case would have a Reply-To with the EAI address, so the multiple
 From fields are not used. The reply is composed to the address(es)
contained in the Reply-To and none of the From headers (see RFC 5322). I
have tested this with a web mail client and a widely used windows/linux
email client. From: had two addresses and Reply-To: had a third. When
replying to such an email, both clients composed mail with To: that
contained the Reply-To: address only.

Based on this, I expect that most email clients that are upgraded to
support EAI will actually support the proposal (repeated below) without
requiring any further work.
    From: EAI-addr, ASCII-addr
    Reply-To: EAI-addr
(where the EAI-addr gets dropped in Downgrade)

> Besides, if this was reasonable for From, what about To: or CC: or Reply-To:?  Presumably I could mail it to other EAI users as well, so how does a recipient respond?  How would one reliably match up the pairs?
>
> Also, presumably I know my own downgrade address, but am I going to know all of the downgrade addresses for everyone on To: (or hopefully Bcc:) that I'm forwarding a joke to?
>  
As part of the downgrade simplification, we are proposing to drop
support for downgrade of forward pointing addresses such as To:, Cc:.If
someone gives you an EAI address, the assumption is that their email
system can receive EAI email.If their system cannot receive EAI email,
it is a 'configuration' error on their end and they shouldn't be giving
out an EAI address until they are ready to receive such mail.

> I remain unconvinced that this solves enough downgrade problems completely enough to be interesting.
>
> -Shawn
>
>  
_______________________________________________
IMA mailing list
IMA@...
https://www.ietf.org/mailman/listinfo/ima

[Shawn Steele]

> (From: EAI for user 1, ASCII for user 2)
>
> This case would have a Reply-To with the EAI address, so the multiple
>  From fields are not used.

I don't think you understood me correctly.  I'm saying if I wanted the mail to be "From: billg@..., steveb@..." when Bill had an EAI address and Steve did not, then it'd be confused with an EAI downgrade format.

I also don't see how Reply-To: helps.  If Reply-To: always has the EAI fallback path, then it can't be used normally.

> As part of the downgrade simplification, we are proposing to drop
> support for downgrade of forward pointing addresses such as To:, Cc:.If
> someone gives you an EAI address, the assumption is that their email
> system can receive EAI email.If their system cannot receive EAI email,
> it is a 'configuration' error on their end and they shouldn't be giving
> out an EAI address until they are ready to receive such mail.

If I send email from an EAI aware account to 10 other EAI aware users, then that's great.  (Like on a list like this).  If someone replies-all, everything works fine.

Now if I add one non-EAI aware user, unless To: & Cc: are fixed, then that person can't reply-all.  In fact I guess all the other addresses have to be dropped for downgrade (or the mail fails)  If they replied-all, they'd get me presumably because of From, but all the other users would fail.  I use Reply-All more than reply.  That's how I replied to this mail in fact.

I'm not sure how much good it does to "fix" from if the others can't be fixed as well.

-Shawn
_______________________________________________
IMA mailing list
IMA@...
https://www.ietf.org/mailman/listinfo/ima

[Ernie Dainow]

Shawn Steele wrote:
>> (From: EAI for user 1, ASCII for user 2)
>>
>> This case would have a Reply-To with the EAI address, so the multiple
>>  From fields are not used.
>>    
>
> I don't think you understood me correctly.  I'm saying if I wanted the mail to be "From: billg@..., steveb@..." when Bill had an EAI address and Steve did not, then it'd be confused with an EAI downgrade format.
>  
Can you actually compose mail like this with your email system? The
assumption was that multiple From addresses are not widely used and are
not generally a feature that users can utilize.The proposal was not to
make the use of multiple From fields available for general use. The From
headers would be generated automatically specifically to carry the
downgrade information.
> I also don't see how Reply-To: helps.  If Reply-To: always has the EAI fallback path, then it can't be used normally.
>  
If you don't want the Reply-To to be your EAI address, I don't see any
reason you couldn't change it, or add additional addresses. This header
would need to be made available to the user composing the email, with
its default being the EAI address.

>  
>> As part of the downgrade simplification, we are proposing to drop
>> support for downgrade of forward pointing addresses such as To:, Cc:.If
>> someone gives you an EAI address, the assumption is that their email
>> system can receive EAI email.If their system cannot receive EAI email,
>> it is a 'configuration' error on their end and they shouldn't be giving
>> out an EAI address until they are ready to receive such mail.
>>    
>
> If I send email from an EAI aware account to 10 other EAI aware users, then that's great.  (Like on a list like this).  If someone replies-all, everything works fine.
>
> Now if I add one non-EAI aware user, unless To: & Cc: are fixed, then that person can't reply-all.  In fact I guess all the other addresses have to be dropped for downgrade (or the mail fails)  If they replied-all, they'd get me presumably because of From, but all the other users would fail.  I use Reply-All more than reply.  That's how I replied to this mail in fact.
>
> I'm not sure how much good it does to "fix" from if the others can't be fixed as well.
>  

This is the "triangle" case raised by Harald. As noted, there may have
to be trade offs in achieving simplification over complexity. This may
be one of those cases that we have to consider giving up.
> -Shawn
_______________________________________________
IMA mailing list
IMA@...
https://www.ietf.org/mailman/listinfo/ima

[Shawn Steele]

I'm going to take a couple steps back.  Mostly I'm focusing on the local part of the address, and I think there's a solution to get us unstuck.
 
A lot of background.  At a high level there are really only a few requirements:

* We need Unicode addresses (That's the point of EAI after all :)
* Many people still need a user friendly ASCII address, for the English side of a Japanese business card if nothing else.
* Some sort of "downgrade" must exist for backwards compatibility.  I'm being liberal with the term.  "Downgrade" could be a user trying Unicode and then retrying with ASCII if necessary, or just giving out ASCII because they know EAI won't work in some scenario.

1) We need Unicode addresses.
The Unicode address is solved reasonably well by UTF-8 and the existing RFCs.  UTF-8 also solves a myriad of problems regarding code pages in the rest of the message, but that's not specifically an EAI issue.
 
2) Friendly ASCII addresses.
Friendly ASCII addresses are pretty much a non-issue since mailbox aliases are a common mail feature.  (I get mail to shawnste & Shawn.Steele)  I make the small assumption that aliasing would be extended to Unicode and that aliases will be more common in an EAI world.  This probably primarily impacts mailbox administration tools.

3) Downgrade
So that leaves downgrade.  Downgrade is necessary for a "transition" period, which is probably many years.  Note that there's no technical requirement that downgrade use the friendly ASCII address mechanism.  It is also worth mentioning that downgrade necessarily requires an EAI aware system (even if it's "just" human downgrade, the human has to be aware of the issue) to do the downgrade.

There are also various degrees of downgrade:

* Human-only downgrade with no automated mechanism.  If my mail bounces I try a different address.  Some big problems are that I may not know the other address, I may not know how to send From: an ASCII address, and some systems may accept mail they can't reply to.

* Partially automated downgrade.  This could be something like From:, or the older headers or something else that doesn't provide the same experience in a mixed environment as in a legacy or EAI only environment.  Eg: simple mail may work, but DL's, newsgroups, reply-to, or other cases could get tricky or fail.

Some big problems with partial downgrade are that it is partial, so user's may think mail works, but get unexpected failures in edge cases.  Also (depending on the degree of "partial"), it may fail if the fallback address(es) aren't know or configured correctly.  Also some mail may appear to succeed but may not be able to be replied to.

* Fully automated downgrade.  This would be some magic system that would downgrade all headers so that "everything" worked, including mailing lists, etc.  In addition to the problem of discovering the downgrade address, we've run into numerous technical concerns and edge cases with the solutions investigated so far.

I've been flip-flopping on how much downgrade is necessary.

Human-only downgrade:
I think it's possible that a human-only mechanism may "work", although that may slow EAI adoption significantly.  (If I don't think my EAI address'll work, then I'll only give out my ASCII address, then why bother with EAI?)

Partially automated downgrade:
Personally I don't see much point in partially automated downgrade (folks from Exchange & Outlook agree.)  If my mail to you works, and your replies work, but everything breaks when I send to a DL or newsgroup, then it's almost worse.

Fully automated downgrade:
Fully automated downgrade would be cool, but we've pretty much proved that it's impossible.  Mostly because pairing the addresses breaks down in several cases, or because the syntax to make the pairing explicit breaks downlevel syntax.

A possible solution:

A while back Mark suggested an solution that's been discussed before and I rejected out of hand.  Either I've had a paradigm shift in my thinking or I misunderstood Mark's earlier suggestion, however there is a way that allows fully automated downgrade without breaking any of the other ASCII address alias or UTF8SMTP behavior.  It also only requires updating the EAI aware servers/clients.  Unaware systems wouldn't see anything different, probably even mailing lists.  Upgrade is even supported.

Mark's suggestion (or my modification of it) is something like this:

* Everyone that wants one gets a Unicode address, and a human friendly ASCII alias (if desired).
* When people exchange addresses in written form, they can use Unicode if they think the recipient is EAI aware, or they can use the ASCII address.
* Mail with EAI to EAI and the Unicode address works as spec'd for UTF8SMTP, etc.  No extra fields are sent.
* When a human or form isn't expecting Unicode, the user shares the human-friendly ASCII alias. Then the ASCII to EAI server behaves as normal.  No extra fields are sent.
* When an EAI server (or client) hits an EAI unaware system, the message is downgraded.  Addresses are downgraded by using unmapped punycode, which is algorithmic, so it avoids all the address pairing problems.  An EAI aware client app (like Outlook) could then upgrade the punycode if desired.  Before downgrade a message is identical to the existing practice, just with UTF-8, and is non-breaking.  After downgrade, a message is still identical to existing legacy behavior, so there's no breaking.

By using UNMAPPED punycode (raw ACE encoding), servers can control their own mappings (Turkish I or whatever's interesting to them).  Presumably they'd decode back to Unicode, then do their mapping and routing.  The problems that downgrade had with mailing lists is avoided because all addresses could be downgraded at any point.  Additionally a ASCII-only mailbox user doesn't need an EAI aware server to use EAI because their EAI aware client could do the mapping.

To me, the differences between this and previous punycode proposals (maybe I misunderstood them) are:
a) UTF-8 is preferred, and is the long term form, not punycode.
b) Human-readable ASCII aliases are a preferred method of exchange on the sticky note or whatnot.  (Nobody's going to exchange a punycode address).
c) Punycode is only used for fallback, when UTF8STMP or other EAI protocols aren't recognized.
d) It isn't IDN punycode, but just unmapped Unicode that the server has to decode and map.

The advantages are:
* Users get a Unicode address.
* Users still get a human friendly ASCII address as needed.
* There's a full downgrade capability (with upgrade if desired)
* Interoperability with legacy systems should be very high.
* No problems with pairing of the addresses.
* Works whenever an EAI-aware client/mailbox behave.  Intermediate systems don't impact it, and an EAI aware recipient client can have an EAI experience even if they don’t have an EAI mailbox themselves.
* Only servers/clients upgraded to EAI need to be touched.  Legacy systems in the middle will still behave.  (I'm told by Exchange that it would take decades to upgrade everything).
* Mapping still happens by the mailbox server's rules.
* Algorithmic rules for downgrade mean that downgrade is always possible instead of requiring knowledge of the ASCII alias.
* Senders can downgrade (if the client is EAI aware) even with only a Unicode recipient address, even if their mail server is a legacy server.

The disadvantages are:
* Non-EAI aware recipients would receive punycode addresses instead of human friendly ASCII.  This is mitigated by the fact that most clients display the display name.
* Punycode downgrade requires that EAI aware systems know the punycode mapping.  This is mitigated by reducing the complexity of the other pairing schemes.
* EAI mailbox servers have to unmap the Punycode to Unicode before doing any mapping to find the mailbox.  (Aliases might work in some cases, but you'd still want case mapping probably).
* For systems that don't need EAI (eg: many US mail servers) there's reduced incentive to upgrade to be EAI aware since an EAI aware client is sufficient.  This could delay complete adoption.  (It's also an advantage that could speed up early adoption.)
* Three addresses (Unicode, ASCII, Punycode) instead of 2 (though the Punycode is just an encoded form of the Unicode).

Couple minor comments:
* As I said, the punycode should be unmapped ACE, not IDN form.
* To make that clear, I'd pick a different prefix, or some other signature mechanism.  (My fear is that developers would accidentally call the IDN APIs and get inappropriate mappings).

I know we've discussed Punycode before, but I think the difference is that I'm suggesting it as a last-resort case, not as a replacement for UTF8SMTP, nor a replacement for the human friendly ASCII aliases.  

Please note that I am very much against Punycode as a solution for EAI (eg: replacing UTF-8).  By itself as a long-term solution Punycode has huge problems.  I think it has a place a "hack" to solve downgrade though.

If necessary to further discussion I could put this in draft form.

-Shawn



_______________________________________________
IMA mailing list
IMA@...
https://www.ietf.org/mailman/listinfo/ima

[Harald Alvestrand]

As I've said several times before, the situation where <<>> has value is
the situation where an EAI user sends a message to both an EAI user and
a non-EAI user, and the non-EAI user replies to the message.

Shawn, if you think that in this scenario, it's OK that the recipient
EAI user does not get the reply from the non-EAI user, please say so
explicitly.

The one sender / one recipient case works without it.

                       Harald


Shawn Steele wrote:

>> To replace double angle brackets for the Sender, John Klensin proposed
>> using multiple From fields. SM amended this with Reply-To to specify the
>> preferred address, as in
>>         From: EAI-addr, ASCII-addr
>>         Reply-To: EAI-addr
>> where the EAI-addr gets dropped in Downgrade.
>>    
>
> I'd imagine that "something" isn't going to be happy with 2 addresses.
>
> It seems to me that 99% of the time email can be viewed as primarily "sender side" or "recipient side".  Eg: Mail goes from a sender system to a recipient system.  It is very rare (now) that an in-between system would relay mail from an untrusted server to another 3rd party server.  If there is a relay, it'd have to trust the sender, so it's pretty much "sender side," or recognize the recipient "recipient side."
>
> Assuming that a "sender" has an EAI aware system that is completely conformant and "works" with UTF-8 addresses, then the entire "sender side" would have to be updated to support UTF-8 addresses.  If the sender's relays aren't upgraded, then all of their mail ends up being downgraded anyway.
>
> Similarly a recipient either has an entirely EAI aware system, or they don't.  
>
> So I'm wondering how interesting downgrade really is.  If my sending system trusts the sender, presumably it could request an ASCII address from the trusted sender-side server if it needed to downgrade.  (Through some other mechanism, like asking the trusted server what other aliases were valid for that account using some new, or even proprietary, protocol).
>
> So the proposed solution seems like overkill.
>
> VERY worst case a client (like Outlook) can try to send an EAI email.  If it bounces, the client could resend using my ASCII address (assuming it is properly configured).  It could even do that silently, or with maybe a little warning saying that had been downgraded.
>
> -Shawn
>
> _______________________________________________
> IMA mailing list
> IMA@...
> https://www.ietf.org/mailman/listinfo/ima
>
>  

_______________________________________________
IMA mailing list
IMA@...
https://www.ietf.org/mailman/listinfo/ima

[Andrew Sullivan-8]

On Sun, Sep 13, 2009 at 07:02:09PM +0200, Harald Alvestrand wrote:
> As I've said several times before, the situation where <<>> has value is  
> the situation where an EAI user sends a message to both an EAI user and  
> a non-EAI user, and the non-EAI user replies to the message.
>
> Shawn, if you think that in this scenario, it's OK that the recipient  
> EAI user does not get the reply from the non-EAI user, please say so  
> explicitly.

As someone who has more or less stopped participating in this WG, I
just want to note that the above amounts to a serious and important
restriction on "backward compatibility".  If the recipient EAI user is
the loser from the non-EAI user, this really means that non-EAI users
(i.e. the non-adopters) lose something, _but also_ that the EAI users
(the adopters) lose something.

I am completely uninformed about the values in play in that trade-off,
and I have no opinion at all on how it ought to go.  But it's a pretty
serious limitation of interoperability between strictly-822-derived
systems and EAI, and it seems to me the WG has to have a strong
statement of the ways that interoperability is expected to succeed and
the ways it is expected to fail.  (Note that I regard "often" as an
acceptable answer.  But clarity is really important.)

A

--
Andrew Sullivan
ajs@...
_______________________________________________
IMA mailing list
IMA@...
https://www.ietf.org/mailman/listinfo/ima

[Shawn Steele]

Harald asked:

> As I've said several times before, the situation where <<>> has value is
> the situation where an EAI user sends a message to both an EAI user and
> a non-EAI user, and the non-EAI user replies to the message.

> Shawn, if you think that in this scenario, it's OK that the recipient
> EAI user does not get the reply from the non-EAI user, please say so
> explicitly.

No, I don't think that partial support in cases like this is helpful.

My current thinking is that:

1) Partial downgrade is worse than no downgrade.  Edge cases like this make the behavior difficult to predict, whereas with no downgrade users know to exchange their friendly-ASCII address.

2) I think that a downgrade solution cannot be sufficiently complete for cases like this unless it is algorithmic (eg: unmapped punycode/ACE).

- Shawn
_______________________________________________
IMA mailing list
IMA@...
https://www.ietf.org/mailman/listinfo/ima

[Harald Alvestrand]

The overarching issue found with variants of this proposal previously
rejected was:

Can we guarantee (for a sufficiently strong version of "guarantee") that
there are no valid mailboxes that "just happen to look like" the
Punycoded strings?

If not, we have to stick with the "don't mess with the left-hand side"
principle and not let the specs define any upgrading, ever. (But then,
if all home systems for the email recognized the Punycoded form, there
isn't much reason to upgrade. But (3rd level) then that's a burden
they'll have to carry around roughly forever..... more levels?)

IDNA made the leap of faith that no existing application would be
inconvenienced if we assumed that no domain name label starting "xn--"
existed with any other use than IDNA.

What assumption can we make?

                          Harald (who was around when X.400 LHS encoding
met Sendmail).

Shawn Steele wrote:

> I'm going to take a couple steps back.  Mostly I'm focusing on the local part of the address, and I think there's a solution to get us unstuck.
>  
> A lot of background.  At a high level there are really only a few requirements:
>
> * We need Unicode addresses (That's the point of EAI after all :)
> * Many people still need a user friendly ASCII address, for the English side of a Japanese business card if nothing else.
> * Some sort of "downgrade" must exist for backwards compatibility.  I'm being liberal with the term.  "Downgrade" could be a user trying Unicode and then retrying with ASCII if necessary, or just giving out ASCII because they know EAI won't work in some scenario.
>
> 1) We need Unicode addresses.
> The Unicode address is solved reasonably well by UTF-8 and the existing RFCs.  UTF-8 also solves a myriad of problems regarding code pages in the rest of the message, but that's not specifically an EAI issue.
>  
> 2) Friendly ASCII addresses.
> Friendly ASCII addresses are pretty much a non-issue since mailbox aliases are a common mail feature.  (I get mail to shawnste & Shawn.Steele)  I make the small assumption that aliasing would be extended to Unicode and that aliases will be more common in an EAI world.  This probably primarily impacts mailbox administration tools.
>
> 3) Downgrade
> So that leaves downgrade.  Downgrade is necessary for a "transition" period, which is probably many years.  Note that there's no technical requirement that downgrade use the friendly ASCII address mechanism.  It is also worth mentioning that downgrade necessarily requires an EAI aware system (even if it's "just" human downgrade, the human has to be aware of the issue) to do the downgrade.
>
> There are also various degrees of downgrade:
>
> * Human-only downgrade with no automated mechanism.  If my mail bounces I try a different address.  Some big problems are that I may not know the other address, I may not know how to send From: an ASCII address, and some systems may accept mail they can't reply to.
>
> * Partially automated downgrade.  This could be something like From:, or the older headers or something else that doesn't provide the same experience in a mixed environment as in a legacy or EAI only environment.  Eg: simple mail may work, but DL's, newsgroups, reply-to, or other cases could get tricky or fail.
>
> Some big problems with partial downgrade are that it is partial, so user's may think mail works, but get unexpected failures in edge cases.  Also (depending on the degree of "partial"), it may fail if the fallback address(es) aren't know or configured correctly.  Also some mail may appear to succeed but may not be able to be replied to.
>
> * Fully automated downgrade.  This would be some magic system that would downgrade all headers so that "everything" worked, including mailing lists, etc.  In addition to the problem of discovering the downgrade address, we've run into numerous technical concerns and edge cases with the solutions investigated so far.
>
> I've been flip-flopping on how much downgrade is necessary.
>
> Human-only downgrade:
> I think it's possible that a human-only mechanism may "work", although that may slow EAI adoption significantly.  (If I don't think my EAI address'll work, then I'll only give out my ASCII address, then why bother with EAI?)
>
> Partially automated downgrade:
> Personally I don't see much point in partially automated downgrade (folks from Exchange & Outlook agree.)  If my mail to you works, and your replies work, but everything breaks when I send to a DL or newsgroup, then it's almost worse.
>
> Fully automated downgrade:
> Fully automated downgrade would be cool, but we've pretty much proved that it's impossible.  Mostly because pairing the addresses breaks down in several cases, or because the syntax to make the pairing explicit breaks downlevel syntax.
>
> A possible solution:
>
> A while back Mark suggested an solution that's been discussed before and I rejected out of hand.  Either I've had a paradigm shift in my thinking or I misunderstood Mark's earlier suggestion, however there is a way that allows fully automated downgrade without breaking any of the other ASCII address alias or UTF8SMTP behavior.  It also only requires updating the EAI aware servers/clients.  Unaware systems wouldn't see anything different, probably even mailing lists.  Upgrade is even supported.
>
> Mark's suggestion (or my modification of it) is something like this:
>
> * Everyone that wants one gets a Unicode address, and a human friendly ASCII alias (if desired).
> * When people exchange addresses in written form, they can use Unicode if they think the recipient is EAI aware, or they can use the ASCII address.
> * Mail with EAI to EAI and the Unicode address works as spec'd for UTF8SMTP, etc.  No extra fields are sent.
> * When a human or form isn't expecting Unicode, the user shares the human-friendly ASCII alias. Then the ASCII to EAI server behaves as normal.  No extra fields are sent.
> * When an EAI server (or client) hits an EAI unaware system, the message is downgraded.  Addresses are downgraded by using unmapped punycode, which is algorithmic, so it avoids all the address pairing problems.  An EAI aware client app (like Outlook) could then upgrade the punycode if desired.  Before downgrade a message is identical to the existing practice, just with UTF-8, and is non-breaking.  After downgrade, a message is still identical to existing legacy behavior, so there's no breaking.
>
> By using UNMAPPED punycode (raw ACE encoding), servers can control their own mappings (Turkish I or whatever's interesting to them).  Presumably they'd decode back to Unicode, then do their mapping and routing.  The problems that downgrade had with mailing lists is avoided because all addresses could be downgraded at any point.  Additionally a ASCII-only mailbox user doesn't need an EAI aware server to use EAI because their EAI aware client could do the mapping.
>
> To me, the differences between this and previous punycode proposals (maybe I misunderstood them) are:
> a) UTF-8 is preferred, and is the long term form, not punycode.
> b) Human-readable ASCII aliases are a preferred method of exchange on the sticky note or whatnot.  (Nobody's going to exchange a punycode address).
> c) Punycode is only used for fallback, when UTF8STMP or other EAI protocols aren't recognized.
> d) It isn't IDN punycode, but just unmapped Unicode that the server has to decode and map.
>
> The advantages are:
> * Users get a Unicode address.
> * Users still get a human friendly ASCII address as needed.
> * There's a full downgrade capability (with upgrade if desired)
> * Interoperability with legacy systems should be very high.
> * No problems with pairing of the addresses.
> * Works whenever an EAI-aware client/mailbox behave.  Intermediate systems don't impact it, and an EAI aware recipient client can have an EAI experience even if they don’t have an EAI mailbox themselves.
> * Only servers/clients upgraded to EAI need to be touched.  Legacy systems in the middle will still behave.  (I'm told by Exchange that it would take decades to upgrade everything).
> * Mapping still happens by the mailbox server's rules.
> * Algorithmic rules for downgrade mean that downgrade is always possible instead of requiring knowledge of the ASCII alias.
> * Senders can downgrade (if the client is EAI aware) even with only a Unicode recipient address, even if their mail server is a legacy server.
>
> The disadvantages are:
> * Non-EAI aware recipients would receive punycode addresses instead of human friendly ASCII.  This is mitigated by the fact that most clients display the display name.
> * Punycode downgrade requires that EAI aware systems know the punycode mapping.  This is mitigated by reducing the complexity of the other pairing schemes.
> * EAI mailbox servers have to unmap the Punycode to Unicode before doing any mapping to find the mailbox.  (Aliases might work in some cases, but you'd still want case mapping probably).
> * For systems that don't need EAI (eg: many US mail servers) there's reduced incentive to upgrade to be EAI aware since an EAI aware client is sufficient.  This could delay complete adoption.  (It's also an advantage that could speed up early adoption.)
> * Three addresses (Unicode, ASCII, Punycode) instead of 2 (though the Punycode is just an encoded form of the Unicode).
>
> Couple minor comments:
> * As I said, the punycode should be unmapped ACE, not IDN form.
> * To make that clear, I'd pick a different prefix, or some other signature mechanism.  (My fear is that developers would accidentally call the IDN APIs and get inappropriate mappings).
>
> I know we've discussed Punycode before, but I think the difference is that I'm suggesting it as a last-resort case, not as a replacement for UTF8SMTP, nor a replacement for the human friendly ASCII aliases.  
>
> Please note that I am very much against Punycode as a solution for EAI (eg: replacing UTF-8).  By itself as a long-term solution Punycode has huge problems.  I think it has a place a "hack" to solve downgrade though.
>
> If necessary to further discussion I could put this in draft form.
>
> -Shawn
>
>
>
> _______________________________________________
> IMA mailing list
> IMA@...
> https://www.ietf.org/mailman/listinfo/ima
>  

_______________________________________________
IMA mailing list
IMA@...
https://www.ietf.org/mailman/listinfo/ima

[Shawn Steele]

> IDNA made the leap of faith that no existing application would be
> inconvenienced if we assumed that no domain name label starting "xn--"
> existed with any other use than IDNA.

> What assumption can we make?

It seems unlikely that mailboxes start with xn--, though other values could maybe be used.  Should such an address already be used, I suggest that it is esoteric enough that such existing usage shouldn't penalize the rest of the Internet moving forward.  Additionally, it'll probably still "work", though some clients might render it funny.

My earlier objections had nothing to do with "does a mailbox looking like that already happen to exist", but rather:
* Punycode is NOT human-friendly.
* UTF-8 on the wire is much cleaner.

My shift in thinking is that UTF-8 on the wire is still strongly preferred, and that a human-friendly address is also required.

-Shawn
_______________________________________________
IMA mailing list
IMA@...
https://www.ietf.org/mailman/listinfo/ima