Off Topic - SPF - What a Disaster

[Jeff Koch]

In an effort to reduce spam further we tried implementing SPF enforcement.
Within three days we turned it off. What we found was that:

- domain owners are allowing SPF records to be added to their zone files
without understanding the implications or that are just not correct
- domain owners and their employees regularly send email from mailservers
that violate their SPF.
- our customers were unable to receive email from important business contacts
- our customers were unable to understand why we would be enforcing a
system that prevented
   them from getting important email.
- our customers couldn't understand what SPF does.
- our customers could not explain SPF to their business contacts who would
have had to contact their IT people to correct the SPF records.

Our assessment is that SPF is a good idea but pretty much unworkable for an
ISP/host without a major education program which we neither have the time
or money to do. Since we like our customers and they pay the bills it is
now a dead issue.

Any other experiences? I love to hear.



Best Regards,

Jeff Koch, Intersessions

[Michael Hutchinson-4]

Hello,

My company attempted to adopt SPF before I started working here. I recall it
was a recent event when I joined, and I looked into what went wrong (as I
became the mail administrator not long after). Basically the exact same
experience was encountered. Customers could not understand the system, which
is basically what killed it. Some Admin's of remote systems sending our
customers important E-Mail did not understand the system, or even want to
deal with it - leaving us without the resources to fix all SPF related
problems.

Adoption of SPF was dropped after 3 days, and we're never going back.

Same result, SPF is a good idea, but we certainly cannot afford to train
other site's administrators, nor all of our customers, on SPF.

Cheers,
Mike,


-----Original Message-----
From: Jeff Koch [mailto:jeffkoch@...]
Sent: Wednesday, 24 February 2010 9:38 a.m.
To: users@...
Subject: Off Topic - SPF - What a Disaster


In an effort to reduce spam further we tried implementing SPF enforcement.
Within three days we turned it off. What we found was that:

- domain owners are allowing SPF records to be added to their zone files
without understanding the implications or that are just not correct
- domain owners and their employees regularly send email from mailservers
that violate their SPF.
- our customers were unable to receive email from important business
contacts
- our customers were unable to understand why we would be enforcing a
system that prevented
   them from getting important email.
- our customers couldn't understand what SPF does.
- our customers could not explain SPF to their business contacts who would
have had to contact their IT people to correct the SPF records.

Our assessment is that SPF is a good idea but pretty much unworkable for an
ISP/host without a major education program which we neither have the time
or money to do. Since we like our customers and they pay the bills it is
now a dead issue.

Any other experiences? I love to hear.



Best Regards,

Jeff Koch, Intersessions

[Aaron Wolfe]

On Tue, Feb 23, 2010 at 4:11 PM, Mike Hutchinson <packetloss@...> wrote:

> Hello,
>
> My company attempted to adopt SPF before I started working here. I recall it
> was a recent event when I joined, and I looked into what went wrong (as I
> became the mail administrator not long after). Basically the exact same
> experience was encountered. Customers could not understand the system, which
> is basically what killed it. Some Admin's of remote systems sending our
> customers important E-Mail did not understand the system, or even want to
> deal with it - leaving us without the resources to fix all SPF related
> problems.
>
> Adoption of SPF was dropped after 3 days, and we're never going back.
>
> Same result, SPF is a good idea, but we certainly cannot afford to train
> other site's administrators, nor all of our customers, on SPF.

ditto here.  the only folks that seem capable of implementing SPF
properly are the spammers

>
> Cheers,
> Mike,
>
>
> -----Original Message-----
> From: Jeff Koch [mailto:jeffkoch@...]
> Sent: Wednesday, 24 February 2010 9:38 a.m.
> To: users@...
> Subject: Off Topic - SPF - What a Disaster
>
>
> In an effort to reduce spam further we tried implementing SPF enforcement.
> Within three days we turned it off. What we found was that:
>
> - domain owners are allowing SPF records to be added to their zone files
> without understanding the implications or that are just not correct
> - domain owners and their employees regularly send email from mailservers
> that violate their SPF.
> - our customers were unable to receive email from important business
> contacts
> - our customers were unable to understand why we would be enforcing a
> system that prevented
>   them from getting important email.
> - our customers couldn't understand what SPF does.
> - our customers could not explain SPF to their business contacts who would
> have had to contact their IT people to correct the SPF records.
>
> Our assessment is that SPF is a good idea but pretty much unworkable for an
> ISP/host without a major education program which we neither have the time
> or money to do. Since we like our customers and they pay the bills it is
> now a dead issue.
>
> Any other experiences? I love to hear.
>
>
>
> Best Regards,
>
> Jeff Koch, Intersessions
>
>

[Bowie Bailey]

Jeff Koch wrote:

>
> In an effort to reduce spam further we tried implementing SPF
> enforcement. Within three days we turned it off. What we found was that:
>
> - domain owners are allowing SPF records to be added to their zone
> files without understanding the implications or that are just not correct
> - domain owners and their employees regularly send email from
> mailservers that violate their SPF.
> - our customers were unable to receive email from important business
> contacts
> - our customers were unable to understand why we would be enforcing a
> system that prevented
>   them from getting important email.
> - our customers couldn't understand what SPF does.
> - our customers could not explain SPF to their business contacts who
> would have had to contact their IT people to correct the SPF records.
>
> Our assessment is that SPF is a good idea but pretty much unworkable
> for an ISP/host without a major education program which we neither
> have the time or money to do. Since we like our customers and they pay
> the bills it is now a dead issue.
>
> Any other experiences? I love to hear.

SPF enforcement at the MTA is useless for the reasons you specified.
The only exception is if you have a strict SPF policy for your own
domain, you can use it to reject spam pretending to be from your users.

--
Bowie

[Michael Scheidell]

On 2/23/10 3:38 PM, Jeff Koch wrote:

since SpamAssassin doesn't block email (and actually, the scoring for
spf failures is pretty low), you must have munged something else up.

if you tried to do pre-queue SPF blocking, yep, go to wsj, yahoo, 'send
link to a friend' and you don't get email, its because your pre-queue
filter messed things up.

Can't get email from important business contacts? what has that go to do
with your clients SPF records? nothing.  maybe the SENDERS had it messed up.

you are right, if you don't know what SPF is, don't use it.

If I send email to someone and they FWD it (.forward) without proper
forwarding, then maybe I didn't want that important email forwarded to
hell and back.

Its all about the RFC's.  and (80%?) of the mail servers out there
violated the RFC's (and SPF is just one of the misused RFC's).  How many
don't even have valid FQDN's in EHLO?  try to explain to a client that
we don't allow inbound email from 'domain.com'.  When the sender decided
that a good internal microsoft 'domain' was domain? and the default FQDN
on their MessServer is mail.domain.com?

or (simi) static dsl or business cable, where the provider is too stupid
or too lazy to set up a proper RDNS (PTR record)?  or someone who's
lawyer insists on using the freebie aol account for their business email
address and wonders why it takes 6 hours to send a simple email to 100
of their clients?

No, there are a lot stupider things you can do than set up SPF records.  
The best thing to do is publish them, but don't block if you have
mismatches.

(yes, the FAQ on our web site still says don't use SPF records)

--
Michael Scheidell, CTO
Phone: [phone], x 1259
 > *| *SECNAP Network Security Corporation

    * Certified SNORT Integrator
    * 2008-9 Hot Company Award Winner, World Executive Alliance
    * Five-Star Partner Program 2009, VARBusiness
    * Best Anti-Spam Product 2008, Network Products Guide
    * King of Spam Filters, SC Magazine 2008

______________________________________________________________________
This email has been scanned and certified safe by SpammerTrap(r).
For Information please see http://www.secnap.com/products/spammertrap/
______________________________________________________________________  

[Martin Gregorie-2]

On Tue, 2010-02-23 at 16:17 -0500, Bowie Bailey wrote:

> The only exception is if you have a strict SPF policy for your own
> domain, you can use it to reject spam pretending to be from your users.
>
Agreed. That's all I use it for. I installed SPF during a backscatter
storm, which immediately decreased in volume. Since then the periodic
backscatter showers have got steadily smaller, so it looks as though
mailservers configured check SPF before bouncing undeliverable mail have
been getting steadily more common.


Martin

[Kelson Vibber]

On 2/23/2010 12:38 PM, Jeff Koch wrote:
> In an effort to reduce spam further we tried implementing SPF
> enforcement. Within three days we turned it off. What we found was that:
<snip>
> Our assessment is that SPF is a good idea but pretty much unworkable for
> an ISP/host without a major education program which we neither have the
> time or money to do. Since we like our customers and they pay the bills
> it is now a dead issue.
>
> Any other experiences? I love to hear.

SPF works great as a selective whitelist in SpamAssassin. (And I don't
mean whitelisting all SPF passes. That would be stupid. I mean
whitelisting mail coming from domain X, but only when it passes SPF and
demonstrates that yes, it really came from domain X.)

I'd say that what you found is *not* that SPF itself is a disaster, but
that enforcing SPF by rejecting failures is a disaster.

It's a data point. It all depends on how you use it.

--
Kelson Vibber
SpeedGate Communications <www.speed.net>

[Jeff Mincy-2]

   From: Martin Gregorie <martin@...>
   Date: Tue, 23 Feb 2010 22:04:07 +0000
   
   On Tue, 2010-02-23 at 16:17 -0500, Bowie Bailey wrote:
   
   > The only exception is if you have a strict SPF policy for your own
   > domain, you can use it to reject spam pretending to be from your users.
   Agreed. That's all I use it for.

The SPF checks in SpamAssassin will score SPF_FAIL without adding
enough points to block the email by itself.   I'm not ready to
outright block email that fail SPF.

   I installed SPF during a backscatter
   storm, which immediately decreased in volume. Since then the periodic
   backscatter showers have got steadily smaller, so it looks as though
   mailservers configured check SPF before bouncing undeliverable mail have
   been getting steadily more common.
   
Either that or spammers tend to avoid forging domains that have SPF.

-jeff

[Dave Pooser]

> Any other experiences? I love to hear.

1) Publishing SPF records at $DAYJOB coincided with a significant drop in
backscatter seen. I don't know whether it's a matter of spammers forging
fewer spam runs from SPFed domains, or other hosts being smart bout bounces,
or....

2) whitelist_auth is worth its weight in platinum
--
Dave Pooser
Cat-Herder-in-Chief, Pooserville.com
"...Life is not a journey to the grave with the intention of arriving
safely in one pretty and well-preserved piece, but to slide across the
finish line broadside, thoroughly used up, worn out, leaking oil, and
shouting GERONIMO!!!" -- Bill McKenna

[Daryl C. W. O'Shea]

On 23/02/2010 7:51 PM, Dave Pooser wrote:
> 2) whitelist_auth is worth its weight in platinum

Damn!  I knew that should have been a subscription only feature! ;)

[LuKreme]

On 23-Feb-10 14:17, Bowie Bailey wrote:
> SPF enforcement at the MTA is useless for the reasons you specified.
> The only exception is if you have a strict SPF policy for your own
> domain, you can use it to reject spam pretending to be from your users.

And that makes it worthwhile all by itself.

--
Is this planter made of lead?

[Marc Perkel]

Jeff Koch wrote:

>
> In an effort to reduce spam further we tried implementing SPF
> enforcement. Within three days we turned it off. What we found was that:
>
> - domain owners are allowing SPF records to be added to their zone
> files without understanding the implications or that are just not correct
> - domain owners and their employees regularly send email from
> mailservers that violate their SPF.
> - our customers were unable to receive email from important business
> contacts
> - our customers were unable to understand why we would be enforcing a
> system that prevented
>   them from getting important email.
> - our customers couldn't understand what SPF does.
> - our customers could not explain SPF to their business contacts who
> would have had to contact their IT people to correct the SPF records.
>
> Our assessment is that SPF is a good idea but pretty much unworkable
> for an ISP/host without a major education program which we neither
> have the time or money to do. Since we like our customers and they pay
> the bills it is now a dead issue.
>
> Any other experiences? I love to hear.
>
>
>
> Best Regards,
>
> Jeff Koch, Intersessions
>

I agree. I've been in the spam filtering business for many years and
have yetto find any use for SPF at all. It's disturbing this useless
technology is getting the false positive support we are seeing.

[Ramprasad-5]

Marc,
> >
> > Which fails when you have someone that has multiple domains that may be sending mail "from" the same organization. Mail to me from Citi may comes from any one of at least 6 different domains, and the mailserver is not necessarily in the same domain.
> >  
> Whitelist all 6 domains.
> >  

What if Citi starts using mail services from another provider with a
different ptr. Do you expect them to announce that on this mailing
list ?
Conversely what if City stops services from one and then a
phisher/spammer buys of the server space. Thanks to the stupid whitelist
I will be sending all these spams whitelisted until we have angry  calls
on the customer support helpdesk.

This is useless for me to keep tracking what servers Citi ,Bank of
America, or ICICIBank  uses. I put just 1 line in my ".cf" file and
forget about it. Because their SPF record already keeps track.

Even the largest banks today are outsourcing their email. FcRDNS works
only if the organization runs their own mailing and dont keep changing
their mailhost names.


Thanks
Ram

[Matus UHLAR - fantomas]

On 25.02.10 15:22, Marc Perkel wrote:
> I'd like to find a way to get people to get their FCrDNS correct. The  
> way I see it if they can't get RDNS correct they aren't going to get SPF  
> correct. Unfortunately I get a lot of ham from IPs with no RDNS.

fcrdns can't be used to filter spam because many spammers use ir properly.

--
Matus UHLAR - fantomas, uhlar@... ; http://www.fantomas.sk/
Warning: I wish NOT to receive e-mail advertising to this address.
Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
The early bird may get the worm, but the second mouse gets the cheese.

[Matus UHLAR - fantomas]

On 25.02.10 17:08, Marc Perkel wrote:
> The forward issue is definitely an annoyance. But SPF has a problem in  
> that as the supporters admit, it doesn't block spam, and it can't be  
> used as a white rule because spammers often use SPF correctly.

Marc, why are YOU trolling?

Are you attempting to say that whitelisting on IPs can't be used because
spammers use IPs correctly?

--
Matus UHLAR - fantomas, uhlar@... ; http://www.fantomas.sk/
Warning: I wish NOT to receive e-mail advertising to this address.
Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
I drive way too fast to worry about cholesterol.

[Matus UHLAR - fantomas]

> LuKreme wrote:
>> Here's where spf is useful.

On 25.02.10 15:31, Marc Perkel wrote:
> Except that it breaks forwarded email.

I have never seen any occurence of SPF breaking forwarding.

But if you forward e-mail from someone and you are pretending to be him,
we may reject it because you are forging.
--
Matus UHLAR - fantomas, uhlar@... ; http://www.fantomas.sk/
Warning: I wish NOT to receive e-mail advertising to this address.
Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
I intend to live forever - so far so good.

[Per Jessen]

Matus UHLAR - fantomas wrote:

>> LuKreme wrote:
>>> Here's where spf is useful.
>
> On 25.02.10 15:31, Marc Perkel wrote:
>> Except that it breaks forwarded email.
>
> I have never seen any occurence of SPF breaking forwarding.

Really?  Do you know which problem SRS was meant to address then?  If
SPF doesn't break forwarding, surely we have no need for SRS.


/Per Jessen, Zürich

[Matus UHLAR - fantomas]

> >> LuKreme wrote:
> >>> Here's where spf is useful.
> >
> > On 25.02.10 15:31, Marc Perkel wrote:
> >> Except that it breaks forwarded email.

> Matus UHLAR - fantomas wrote:
> > I have never seen any occurence of SPF breaking forwarding.

On 26.02.10 09:46, Per Jessen wrote:
> Really?  Do you know which problem SRS was meant to address then?  If
> SPF doesn't break forwarding, surely we have no need for SRS.

I have explained it many times, even in the mail you quote.
I don't see reason to repeat that to people who can't / don't want to
understand.

The funniest anti-SPF anti-SRS argument at
http://david.woodhou.se/why-not-spf.html
was the "alternative" SES which was inspired by SRS.
--
Matus UHLAR - fantomas, uhlar@... ; http://www.fantomas.sk/
Warning: I wish NOT to receive e-mail advertising to this address.
Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
If Barbie is so popular, why do you have to buy her friends?

[Mike Cardwell-16]

On 25/02/2010 23:31, Marc Perkel wrote:

> As someone who forwards email what I see is this.
>
> Sender has restrictive SPF.
> Recipient server enforces SPF.
> Mail coming through me bounces.
>
> Then they call me to complain and I say, I didn't bounce it. Get rid of
> your SPF nd your email will be received.

In your scenario, there are two broken systems. Neither of which are SPF.

The first broken system is your user. They've applied SPF to their
domain. They've set up mail forwarding from your service. Yet they still
apply SPF checking against your servers? That is stupid. They've
misconfigured their mail service. They should either remove SPF, get rid
of the forwarding, or change the forwarding provider to one which
rewrites the envelope sender.

The second broken system is your forwarding system. It's is forging the
envelope sender instead of correctly rewriting it. Fix that, or continue
to offer a sub-standard forwarding service. Your choice.

It's not SPF's fault that your clueless user can't receive some email.
It's a combination of your broken forwarding configuration, and your
clueless users misconfiguration of their email.

It's too late anyway. Your opinion is no longer relevant. SPF is
absolutely here to stay. It is supported by *many* large providers, and
a large proporition of ham is already using SPF.

Ideally, 100% of Ham and 100% of Spam would use SPF. You don't seem to
get this though. You think SPF is only useful if 100% of Ham uses it and
0% of Spam uses it. That's a flaw in your understanding of what it's
there for.

If a Spam comes from "example.com" and it's SPF protected, then you know
the domain hasn't been forged, and it's safe to blacklist it. If it
*isn't* SPF protected, then for all you know it has been forged and
blacklisting it might cause collateral damage.

The positive aspects of *any* mail being "signed" with SPF, ham *or*
spam, are so damn obvious, I don't know how you manage to mis-represent
them so blatantly and so poorly.

--
Mike Cardwell    : UK based IT Consultant, Perl developer, Linux admin
Cardwell IT Ltd. : UK Company - http://cardwellit.com/       #06920226
Technical Blog   : Tech Blog  - https://secure.grepular.com/
Spamalyser       : Spam Tool  - http://spamalyser.com/

[Rick Cooper]

----Original Message----
From: Marc Perkel [mailto:marc@...]
Sent: Thursday, February 25, 2010 6:11 PM
To: Rick Cooper
Cc: 'ram'; users@...
Subject: Re: Off Topic - SPF - What a Disaster

> Rick Cooper wrote:
>>
>>  >>> The anti-SPF bandwagon is not ego driven but results driven. Than
>>  you >>> for admitting that SPF in not a spam filtering solution.
>>  However it >>> is also not a white listing solution because as many
>>  people have said >>> here - spammers are the ones who are using SPF
>>  correctly. I can see >>> some theoretical benefits that if you have a
>>  list of banks with SPF >>> and you receive an email from an address
>>  that the bank lists then you >>> can safely pass it. But I find that an
>>  easier way to do that is to >>> use FCrDNS to do the same thing.
>>
>>  >>> On the down site SPF breaks email forwarding and it creates a false
>>  >>> sense that people are doing something to fight spam or protect ham
>>  >>> that is not supported by reality. SPF has received intellectual
>>  >>> welfare because stuff that doesn't work tends to be culled out of
>>  >>> spam assassin and other than backscatter most people here are
>>  telling >>> the SPF supporters that it doesn't work. If SPF is becoming
>>  more >>> popular it just means that more people are misled.
>>
>> So then SRS Doesn't work for forwarding systems? I ask because I am
>> not a forwarding service and, as I only handle corporate mail
>> systems, do not give access to arbitrary forwarding to the mail
>> users so we do not have tons of (external) forwarding going on. Since
>> SPF and SRS are like legs on the same body I will assume trying to walk
>> with
>> one leg produces results similar to a forwarding service using SPF
>> without SRS. I personally would love comcast would list all of their
>> Valid outbound mail hosts and hard fail all others, same with aol,
>> yahoo, gmail, etc. Seems to me if you are going to push email all over
>> hell's half acre it behooves you To use any and all tools available to
>> take responsibility for those mails and SPF is One of several tools that
>> can do that, at least to some extent. If there would have been Some kind
>> of total commitment to spam 10 years ago we would not be where we are
>> today and Spamassassin (as it is) would not be quite so necessary.
>>
>> (My apologies for the pathetic attempt at manually reformatting
>> the original html post)
>>
>>
>
> SRS is even more broken than SPF. I allow users to white list or black
> list based on the sender. If you rewrite the sender then you lose sender
> based conditionals. SRS has no use other than to try to fix SPF which
> has no use in the first place.

I suppose you would have to add logic to your whitlisting to accommodate an
SRS message, it's not like you cannot tell and the return path remains
intact so the original sending address is still available for the white
list. Pobox.com uses it (of course) and the are a forwarding service. I
don't personally see SPF as a spam tool so much as someone taking
responsibility for the mail they send. I suppose since all forwarding
services are legitimate the world should just take messages originating from
them as legitimate as well.... My bad

Rick


--
This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.