SMTP pull anyone?

[Ravi shankar-15]

Hi,

 

Me and my buddy had a interesting discussion, which i thought could put across the geeks here.

 

It goes something like this:

 

SMTP is currently a push protocol and is initiated by the the sender, no controlling that fact.

 

But it is possible to overcome the relay problems, IP spoofing and domain impersonation etc,

 

by making the servers pull the mails.

 

1. Sending server contacts the destination and proovides the Message ID and sender details(and other details) and disconnects the session.

 

2. The receiving server queues it up and looks up the messages one by one using DNS to determine their legitimacy.

 

3. If the IP that contacted is legitimate(can be verified by say SPF?), it contacts the sender and provides the message ID with other details.

 

4. The sending server then hands over the message.

 

5. To overcome DDoS attacks, the receiving server can be made to request the next 10 or so Message IDs that it will assign to messages,

 

so that if a attacker tries to give those details, it will know from the next list of message IDs that it's fake connection.

 

6. May be by this collection of data, the IP addresses can be reported to a RBL and blacklisted.

 

Please point the holes in this model, so that we might get a entirely new insight.

 

Note: I have gone trough IM2000 and other similar discussions in the archive. Just thought this version of C/R is worth getting discussed.

 

Regards,

Ravi

 

 

_______________________________________________
Asrg mailing list
Asrg@...
http://www.irtf.org/mailman/listinfo/asrg

[Bill Cole-2]

Ravi shankar wrote, On 8/16/09 7:20 AM:

> Hi,
> Me and my buddy had a interesting discussion, which i thought could put
> across the geeks here.
> It goes something like this:
> SMTP is currently a push protocol and is initiated by the the sender, no
> controlling that fact.
> But it is possible to overcome the relay problems, IP spoofing and
> domain impersonation etc,
> by making the servers pull the mails.
> 1. Sending server contacts the destination and proovides the Message ID
> and sender details(and other details) and disconnects the session.
> 2. The receiving server queues it up and looks up the messages one by
> one using DNS to determine their legitimacy.

DNS itself provides no mechanism for determining the legitimacy of email.

DNS is used *as a medium* for various applications that are used to identify
mail as legitimate or illegitimate by various standards of legitimacy, and a
major reason for its use in those applications is to make it feasible for
mail systems to do the validation synchronously during the SMTP session. By
using a lightweight, distributed, cached database, mail systems are spared
from deferring a message, queuing its validation, remembering the results,
and waiting for the sender to offer it in an identical way again. You are
suggesting that receivers should take on all the heavyweight management but
retain using DNS for something unspecified. It makes no sense.

> 3. If the IP that contacted is legitimate(can be verified by say SPF?),
> it contacts the sender and provides the message ID with other details.

The *most* that SPF can provide towards showing "legitimacy" is to confirm
that the envelope sender address of a message is not forged. It is very rare
for large senders of any sort to deploy records that can do that strongly.
There is nothing about SPF that directly attacks spamming. It could in
theory be used to attack sender forgery, but the collateral damage has
proven to be too great for either sending or receiving systems to actually
apply it strongly to that end. Meanwhile, a lot of spammers are sending a
lot of spam with senders that are validated to the degree that SPF can
validate anything.


> 4. The sending server then hands over the message.
> 5. To overcome DDoS attacks, the receiving server can be made to request
> the next 10 or so Message IDs that it will assign to messages,
> so that if a attacker tries to give those details, it will know from the
> next list of message IDs that it's fake connection.

That sentence makes no sense. What did you mean to say?

> 6. May be by this collection of data, the IP addresses can be reported
> to a RBL and blacklisted.
> Please point the holes in this model, so that we might get a entirely
> new insight.

Nothing you have described would add to spam control as it is currently
being done, as far as I can see. The 'model' is too vague to critique inn
detail because you aren't really providing any meaningful details.

In order to bring anything truly new and useful to controlling email spam, a
new idea has to either attack spam in a way that existing tactics don't, do
a demonstrably better job than existing tactics, or overcome the negative
aspects of existing tactics. You have identified none of those in your new
idea.

> Note: I have gone trough IM2000 and other similar discussions in the
> archive. Just thought this version of C/R is worth getting discussed.

As described, there's not much to discuss. It looks like it might be a
little like IM2000 (which has been a complete failure) if fleshed out, but
you haven't explained how this would differ from it, how it would integrate
with existing SMTP, or how it would offer anything significantly better than
existing mechanisms.

Right now, receivers can choose to validate the sender against SPF records
after receiving the MAIL command and reject non-validating messages at that
point, or they can wait until RCPT if they want their rejections to be more
broadly respected, or they can wait until the second phase of DATA if they
want to weight SPF results along with other filters that rely on message
data. Many sites already explicitly reject the vast majority of offered
messages in response to RCPT or earlier without requiring legitimate senders
to sit on a pile of pending offered messages that may or may not be asked
for later. You have not identified any benefits for receiving systems or
legitimate senders, so it is hard to agree that it is "worth getting
discussed" or indeed worth anything at all. Where is the added value?
_______________________________________________
Asrg mailing list
Asrg@...
http://www.irtf.org/mailman/listinfo/asrg

On Sun, Aug 16, 2009 at 06:20, Ravi shankar<ravisha22@...> wrote:
> Me and my buddy had a interesting discussion, which i thought could
> put across the geeks here.
[... pull e-mail description...]

I wrote up something very similar, and as noted, there was IM2000.

At this point, the consensus on this list as far as I can determine is
"Nothing will work, and if you think otherwise go implement it and
prove us wrong". So I'd suggest digging out an ANSI C compiler as your
only productive next step.


mathew
--
<URL:http://www.pobox.com/~meta/>
_______________________________________________
Asrg mailing list
Asrg@...
http://www.irtf.org/mailman/listinfo/asrg

[Dave Crocker-2]

Ravi shankar wrote:
> SMTP is currently a push protocol and is initiated by the the sender, no
> controlling that fact.
>  
> But it is possible to overcome the relay problems, IP spoofing and
> domain impersonation etc,
>  
> by making the servers pull the mails.


<http://v3.espacenet.com/textdoc?DB=EPODOC&IDX=US6192407>

--

   Dave Crocker
   Brandenburg InternetWorking
   bbiw.net
_______________________________________________
Asrg mailing list
Asrg@...
http://www.irtf.org/mailman/listinfo/asrg

[John Levine-3]

>> SMTP is currently a push protocol and is initiated by the the
>> sender, no controlling that fact.
 
>> But it is possible to overcome the relay problems, IP spoofing and
>> domain impersonation etc,
 
>> by making the servers pull the mails.

> US6192407

Ah, right, the famous Tumbleweed patent.  Here's a better link
with a readable version.

http://www.google.com/patents?vid=USPAT6192407

This patent covers the idea of putting a message on a web server and
sending the recipient a URL to retrieve the message.  Tumbleweed
enforces it very aggressively.  See the list of suits in their
Wikipedia page.

Were it not for the patent, I expect we'd have a lot of institutions
setting up per-user password protected RSS or Atom feeds with
statements and other messages, and sending you mail telling you to
check your feed when there's something new.

R's,
John
_______________________________________________
Asrg mailing list
Asrg@...
http://www.irtf.org/mailman/listinfo/asrg

[Ian Eiloart]

--On 16 August 2009 14:14:00 -0400 Bill Cole <asrg3@...>
wrote:

>
>> 3. If the IP that contacted is legitimate(can be verified by say SPF?),
>> it contacts the sender and provides the message ID with other details.
>
> The *most* that SPF can provide towards showing "legitimacy" is to
> confirm that the envelope sender address of a message is not forged. It
> is very rare for large senders of any sort to deploy records that can do
> that strongly. There is nothing about SPF that directly attacks spamming.
> It could in theory be used to attack sender forgery, but the collateral
> damage has proven to be too great for either sending or receiving systems
> to actually apply it strongly to that end. Meanwhile, a lot of spammers
> are sending a lot of spam with senders that are validated to the degree
> that SPF can validate anything.

SPF deployment has grown rapidly from 5% of 2,000,000 sampled domains to
17% over the past three years, apparently including most USA banks. About
half of spf publishing domains, including some large senders like facebook,
use "-all" records. Apart from anything else "-all" on its own is a good
way of saying "this isn't an email domain", and it's probably a good idea
to publish it for every A record that doesn't point to a mail server.

Furthermore, some large recipients like gmail use these records to help
assign reputation to senders. Forwarded email is likely already suffering
from deliverability problems when the sender address is not rewritten (at
least in cases like forwarding facebook mail to gmail, for example), and
these problems will continue to get worse, not better as SPF deployment
increases, and as records are increasingly respected.

People who want to offer a reliable forwarding service to their users
already need to be thinking about rewriting sender addresses.

It might take a few years, but I'm convinced that we'll look back on SPF
deployment in the same was that we look back on the campaign against open
relays some years ago.

--
Ian Eiloart
IT Services, University of Sussex
01273-873148 x3148
For new support requests, see http://www.sussex.ac.uk/its/help/
_______________________________________________
Asrg mailing list
Asrg@...
http://www.irtf.org/mailman/listinfo/asrg

[Michael Thomas-3]

I don't see how push or pull fundamentally changes the spam
equation in any case. The problem wrt spam is the any-any nature
of who you receive communication from, not who initiates the
connection.

I can create a walled garden trivially with smtp just be rejecting
connections from people not on my whitelist. Likewise, anything that
is promiscuous with who it receives from better be prepared for
mail-transmitted diseases (MTD's). That's just the nature of opening
up. So SMTP itself isn't the problem, or more specifically fretting
about SMTP is nipping around the edges of 20 with the 80-20 rule.

Mike

On 08/16/2009 03:29 PM, John Levine wrote:

>>> SMTP is currently a push protocol and is initiated by the the
>>> sender, no controlling that fact.
>
>>> But it is possible to overcome the relay problems, IP spoofing and
>>> domain impersonation etc,
>
>>> by making the servers pull the mails.
>
>> US6192407
>
> Ah, right, the famous Tumbleweed patent.  Here's a better link
> with a readable version.
>
> http://www.google.com/patents?vid=USPAT6192407
>
> This patent covers the idea of putting a message on a web server and
> sending the recipient a URL to retrieve the message.  Tumbleweed
> enforces it very aggressively.  See the list of suits in their
> Wikipedia page.
>
> Were it not for the patent, I expect we'd have a lot of institutions
> setting up per-user password protected RSS or Atom feeds with
> statements and other messages, and sending you mail telling you to
> check your feed when there's something new.
>
> R's,
> John
> _______________________________________________
> Asrg mailing list
> Asrg@...
> http://www.irtf.org/mailman/listinfo/asrg

_______________________________________________
Asrg mailing list
Asrg@...
http://www.irtf.org/mailman/listinfo/asrg

[Douglas Otis]

On 8/17/09 9:34 AM, Michael Thomas wrote:
> I don't see how push or pull fundamentally changes the spam
> equation in any case. The problem wrt spam is the any-any nature
> of who you receive communication from, not who initiates the
> connection.

The pull concept is fairly simple.  When creating a white-list, it
becomes important to identify the source of the message.  Since email
uses store and forwarding, originating sources of messages can be
difficult to determine, based upon the message alone.  Of course DKIM
helps with that, but it would be more ideal to shift a greater portion
of the burden toward the sender for email to continue to scale.

By exchanging only references to messages that are held on-line, the
message itself does not need to be initially exchanged.  When contacted
by a specific source that has been white-listed, your MUA could fetch
the desired message from the on-line server at the same time references
have been retrieved.

When a reference has been falsified, no message can be fetched.  This
would not demand cryptographic efforts by the sender, or expect receipt
of an exponentially increasing volume of junk, or tens or hundreds of
DNS transactions per domain to resolve server authorizations.

One wonders whether IMAP might be tweaked to provide an online function
rather than using traditional URIs.  The problem should not be viewed as
the 80-20 rule where 80% of the problems are caused by 20% of the
sources.  This should be viewed as the 99-99 rule, in today's email
environment.

-Doug
_______________________________________________
Asrg mailing list
Asrg@...
http://www.irtf.org/mailman/listinfo/asrg

[John Levine-3]

>I don't see how push or pull fundamentally changes the spam equation
>in any case. The problem wrt spam is the any-any nature of who you
>receive communication from, not who initiates the connection.

I agree.  The main advantage I see to pull is bandwidth management,
since recipients can decide when and whether to retrieve mail for
users, rather than having to accept it all just in case someone wants
to read it.

The disadvantage is that it requires that senders and receivers all be
connected to the same network all the time, which is a lot closer to
true now than it was in the 1980s, but still a long way from
universally true.

R's,
John

_______________________________________________
Asrg mailing list
Asrg@...
http://www.irtf.org/mailman/listinfo/asrg

[Bart Schaefer-2]

On Aug 18,  3:47am, John Levine wrote:
>
> The disadvantage is that it requires that senders and receivers all
> be connected to the same network all the time, which is a lot closer
> to true now than it was in the 1980s, but still a long way from
> universally true.

I'm not a proponent of the pull model, but it seems to me this could
be overcome if each receiver in a store-and-forward sequence were
able to inform its immediate upstream when it is not able to perform
the pull.

E.g., transmit the "pull handle" as far as possible, then have the
last relay that is capable of doing so, issue the pull request.

This could probably even be done with one new SMTP verb plus ETRN.
E.g. DATA FROM:<message-id> where the domain part of the message-id
must resolve to the host that will cough up the message.  If the
receiver responds 50z or 455, the sender must do the pull.  If the
receiver responds 354, sender sends dot, and eventually some receiver
connects to <message-id-domain> and issues ETRN #<message-id-local>.

All sorts of security implications to be worked out, Tumbleweed
problems aside.
_______________________________________________
Asrg mailing list
Asrg@...
http://www.irtf.org/mailman/listinfo/asrg

[Ale2008]

Michael Thomas wrote:
> I can create a walled garden trivially with smtp just be rejecting
> connections from people not on my whitelist. Likewise, anything that
> is promiscuous with who it receives from better be prepared for
> mail-transmitted diseases (MTD's). That's just the nature of opening
> up. So SMTP itself isn't the problem, or more specifically fretting
> about SMTP is nipping around the edges of 20 with the 80-20 rule.

I don't think "opening up" is bound to have a unique nature. A
protocol might provide for ways to open up _and_ allow
traceability, responsibility, and accountability. Such features
are implicit in walled gardens. Explicating them requires a more
thorough analysis of the relevant conventions and rules, at an
abstraction level such that its outcome can be globally
accepted. Much of the required "political" work is already
available as privacy recommendations --later than SMTP. It is
possible to tweak SMTP so as to comply with that. While that
would not "eliminate spam", it would permit to manage it
cleanly, properly, and without sacrificing reliability.
_______________________________________________
Asrg mailing list
Asrg@...
http://www.irtf.org/mailman/listinfo/asrg

[Dave Crocker-2]

John Levine wrote:
> I agree.  The main advantage I see to pull is bandwidth management,
...
> The disadvantage is that it requires that senders and receivers all be
> connected to the same network all the time,


The disadvantages are more substantial.  This is a very high overhead model, in
terms of hassling the recipient and in terms of transactions costs.

d/

--

   Dave Crocker
   Brandenburg InternetWorking
   bbiw.net
_______________________________________________
Asrg mailing list
Asrg@...
http://www.irtf.org/mailman/listinfo/asrg