How would you provide a 554 rejection notice for spam?

[John Rudd]

John Rudd wrote:

> dalchri wrote:
>> Well, I setup MIMEDefang.  Everything is working as I want except that
>> the
>> (fake) rejected mail does not make it through the milter to Exchange.  I
>> used action_bounce to reject the message in mimedefang-filter.
>>
>> Is there a way to send the rejection code but still get the message
>> through
>> the milter?
>>
>> I think I might be leaning towards Exim for it's fake reject feature if I
>> can't get this to work.
>>
>> Thanks for all the feedback!
>
> You should probably ask that question on the mimedefang mailing list.

To elaborate: it IS possible, but there's probably multiple ways to do
it.  People on the mimedefang list will probably have already done it
various ways and it'd be better to discuss it there than on the
spamassassin list :-}

(I've recently moved from sendmail+mimedefang to communigatepro+plugins,
so my mindset isn't really deep into mimedefang anymore, or I'd try to
give a more specific answer)

[John Rudd]

John Rudd wrote:

> dalchri wrote:
>> Well, I setup MIMEDefang.  Everything is working as I want except that
>> the
>> (fake) rejected mail does not make it through the milter to Exchange.  I
>> used action_bounce to reject the message in mimedefang-filter.
>>
>> Is there a way to send the rejection code but still get the message
>> through
>> the milter?
>>
>> I think I might be leaning towards Exim for it's fake reject feature if I
>> can't get this to work.
>>
>> Thanks for all the feedback!
>
> You should probably ask that question on the mimedefang mailing list.

Ok, you tickled my curiosity.  My thought is:

where you're doing "action_bounce($error_message, $code, $dsn)" do this:


resend_message("your_holding_area@...");
action_bounce($error_message, $code, $dsn);


resend_message() sends the full original message (including bad content
that you may have rejected in other parts of mimedefang) to the new
recipient(s).  As the mimedefang-filter manpage warns, you should only
call it in filter_end or filter_begin, otherwise it might get called
multiple times.  If you're doing your spam rejection in filter_end, then
that's a good place to call it.


You could, alternately, do:

action_quarantine_entire_message($some_message_to_the_sysadmin);
action_bounce($error_message, $code, $dsn);


That will put the message into the mimedefang quarantine, instead of
sending it to some address on the exchange server.  It's just a matter
of where you want to hold the message.

[Matus UHLAR - fantomas]

> Matus UHLAR - fantomas escribió:
> >when we ran qmail, we had false positives, and we did not like the fact we
> >could not tell sender what the problem was...

On 31.07.07 08:41, Diego Pomatta wrote:
> But is not qmail's job to detect spam or tell the sender what the
> problem was; qmail is just the MTA, and a damn fine one imho.
> A filter/scanner/anti-spam tool has to do that.

I hope I explained well enough: qmail does not pass message from scanner to
user. It only sends its own message which tells nothing. That's bad.

--
Matus UHLAR - fantomas, uhlar@... ; http://www.fantomas.sk/
Warning: I wish NOT to receive e-mail advertising to this address.
Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
My mind is like a steel trap - rusty and illegal in 37 states.

[Kris Deugau]

Diego Pomatta wrote:
> But is not qmail's job to detect spam

True.

> or tell the sender what the
> problem was;

True only for your local site policy;  most people who reject spam would
like to let the sender know so legitimate senders can rearrange their
message to try again.  More generally, it's usually a good idea to
include *some* kind of information about why you rejected the message if
you reject an email message at the MTA layer.

qmail makes this much more difficult that pretty much any other MTA.

qmail, as provided by DJB, is nearly unusable in today's email
environment IMO.

> qmail is just the MTA, and a damn fine one imho.
> A filter/scanner/anti-spam tool has to do that.

If you're going to notify senders about spam or virus content, the time
to do it is before your mail system has sent a "250 OK" reply to the
message's DATA segment.  Accepting the message then constructing a
(new!) rejection message to send back generates backscatter, and is
likely to get your system blacklisted locally by sysadmins everywhere if
you do this.

Earlier you also wrote:
 > If you mean incoming, IN MY CASE I drop spam without further notice to
 > the sender or the recipient. I deal with the false possitives
 > personally, and configure SA accordingly. Only 2 false possitives
 > since SA is in effect, though. And it was actually mail I would
 > consider spam, but the user in question wanted to receive it anyway.

2 FPs over what time period?  How much overall mail flow?  What type of
system is it;  how many accounts?

How do you find out about a false positive if you discard anything
tagged as spam?

I don't drop anything but confirmed viruses on my *personal* mail
system, never mind the systems I'm responsible for at work;  I shudder
to think of the cries of outrage if I silently dropped spam on the ISP
mail systems I administer.  (There *have* been business-related FPs,
more than once.)  I *do* *divert* messages considered spam for most
customers to a spam folder, and old spam is expired on a daily basis.

-kgd

[Diego Pomatta]

Kris Deugau escribió:
> I don't drop anything but confirmed viruses on my *personal* mail
> system, never mind the systems I'm responsible for at work;  I shudder
> to think of the cries of outrage if I silently dropped spam on the ISP
> mail systems I administer.  (There *have* been business-related FPs,
> more than once.)  I *do* *divert* messages considered spam for most
> customers to a spam folder, and old spam is expired on a daily basis.
>
Yes of course. This is not an ISP, otherwise we would tag spam.
Let's just say that we know what we are doing, and we do it for a reason. ;)

I'm off to lunch!
/Regards

[hamann.w]

>>
>> Diego Pomatta wrote:
>> > But is not qmail's job to detect spam
>>
>> True.
>>
>> > or tell the sender what the
>> > problem was;
>>
>> True only for your local site policy;  most people who reject spam would
>> like to let the sender know so legitimate senders can rearrange their
>> message to try again.  More generally, it's usually a good idea to
>> include *some* kind of information about why you rejected the message if
>> you reject an email message at the MTA layer.
>>
>> qmail makes this much more difficult that pretty much any other MTA.

as another qmail user, may I put this straight: qmail does not pass an individual message
to the sender, just a selection of predetermined messages (temporary problem,
prohibited, whatnot)
You are free to add something like
"554 your message is considered excessively spammy"
to the list of predetermined messages. You cannot send the actual spam score

>>
>> qmail, as provided by DJB, is nearly unusable in today's email
>> environment IMO.
The fact that we need spamassassin, antivirus, and the like to integrate into
other mailers seems to indicate that they are not much better ....

>> > qmail is just the MTA, and a damn fine one imho.
>> > A filter/scanner/anti-spam tool has to do that.
>>
>> If you're going to notify senders about spam or virus content, the time
>> to do it is before your mail system has sent a "250 OK" reply to the
>> message's DATA segment.  Accepting the message then constructing a
>> (new!) rejection message to send back generates backscatter, and is
>> likely to get your system blacklisted locally by sysadmins everywhere if
>> you do this.
>>
>>

It is genereally known as bad practice, and has been told over and over again,
to bounce messages.
If the discussion is about the response to the DATA phase (as it should be),
you are free to modify that piece of perl code that drives mail scanning in a way that it
sends on the message (with subject changed, or otherwise modified) AND tell the sender
that it has permfailed

Wolfgang Hamann

[mouss-2]

Rick Macdougall wrote:
>
> simscan correctly uses an SMTP REJECT (55x code during the smtp
> conversation) and it is also possible to use custom reject messages
> with simscan so the sender, if any, knows exactly why the message was
> rejected.
>
> I have yet to see a good implementation of this in Postfix or
> Sendmail, and is one of the reasons I stick with Qmail.

running SA at smtp time requires that the client does not timeout. so
you'd better scan fast! you're also more subject to DOS (your smtp
listeners are busy). compare this to queue and filter...

anyway, you can still do smtp time filtering with postfix and sendmail.
google for milter and proxy_filter.


>
> Having to /dev/null spam and/or viruses to the end user is even worse
> IMHO (as an ISP, it might be acceptable in an office env where you can
> train the users to look at spam or virus folders).

Reject (not bounce) is acceptable, but you'd better configure your
filter correctly. If I get a reject from you for a legitimate mail I
sent, you will just make yourself ridiculous... It also delays
legitimate mail (because I have to resend).

delivering to a Junk folder and letting the recipient review this folder
maybe annoying, but if the recipient is not too dumb (or too busy), then
he can "save ham transparently".

anyway, each site has its own policy for handling spam. What we ask is
to avoid bad practices that result in annoying others (backscatter, C/R,
... etc). The rest (discard, reject at smtp time, quarantine, ...) is up
to you, your boss, your users, your customers, ... etc.

[Rick Macdougall-2]

Hi,

mouss wrote:

> Rick Macdougall wrote:
>>
>> simscan correctly uses an SMTP REJECT (55x code during the smtp
>> conversation) and it is also possible to use custom reject messages
>> with simscan so the sender, if any, knows exactly why the message was
>> rejected.
>>
>> I have yet to see a good implementation of this in Postfix or
>> Sendmail, and is one of the reasons I stick with Qmail.
>
> running SA at smtp time requires that the client does not timeout. so
> you'd better scan fast! you're also more subject to DOS (your smtp
> listeners are busy). compare this to queue and filter...

We do, 4 back end spamd machines for 4 external MX machines.

>>
>> Having to /dev/null spam and/or viruses to the end user is even worse
>> IMHO (as an ISP, it might be acceptable in an office env where you can
>> train the users to look at spam or virus folders).
>
> Reject (not bounce) is acceptable, but you'd better configure your
> filter correctly. If I get a reject from you for a legitimate mail I
> sent, you will just make yourself ridiculous... It also delays
> legitimate mail (because I have to resend).
>

We only REJECT very high scoring spams or viruses, I haven't seen an FP
on the rejecting yet.

>
> anyway, each site has its own policy for handling spam. What we ask is
> to avoid bad practices that result in annoying others (backscatter, C/R,
> ... etc). The rest (discard, reject at smtp time, quarantine, ...) is up
> to you, your boss, your users, your customers, ... etc.

Agree completely.

Regards,

Rick

[John Hardin]

On Tue, 31 Jul 2007, mouss wrote:

> running SA at smtp time requires that the client does not timeout.
> so you'd better scan fast! you're also more subject to DOS (your
> smtp listeners are busy). compare this to queue and filter...

okay, here's a sick idea:

(1) MTA completes the SMTP exchange and responds with a 4xx after DATA
finishes.

(2) MTA passes message off to SA, then stores a hash of
message-ID/score. MTA then discards the message.

(3) When the remote MTA retries (if it retries) then the MTA looks up
the score in the hash and decides whether to 200 or 5xx the message.

All of the benefits of both methods! :)

--
 John Hardin KA7OHZ                    http://www.impsec.org/~jhardin/
 jhardin@...    FALaholic #11174     pgpk -a jhardin@...
 key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C  AF76 D822 E6E6 B873 2E79
-----------------------------------------------------------------------
  Perfect Security is unattainable; beware those who would try to sell
  it to you, regardless of the cost, for they are trying to sell you
  your own slavery.
-----------------------------------------------------------------------
 4 days until The 272nd anniversary of John Peter Zenger's acquittal

[Rick Macdougall-2]

John D. Hardin wrote:

> On Tue, 31 Jul 2007, mouss wrote:
>
>> running SA at smtp time requires that the client does not timeout.
>> so you'd better scan fast! you're also more subject to DOS (your
>> smtp listeners are busy). compare this to queue and filter...
>
> okay, here's a sick idea:
>
> (1) MTA completes the SMTP exchange and responds with a 4xx after DATA
> finishes.
>
> (2) MTA passes message off to SA, then stores a hash of
> message-ID/score. MTA then discards the message.
>
> (3) When the remote MTA retries (if it retries) then the MTA looks up
> the score in the hash and decides whether to 200 or 5xx the message.
>
> All of the benefits of both methods! :)
>

Sort of like grey listing, which I do run on my personal domain, but I
wouldn't use that method because of the inherent delay caused by the 4xx
retry.

Neat idea though.

Regards,

Rick

On Tue, 31 Jul 2007 at 18:20 -0400, rickm@... confabulated:

> John D. Hardin wrote:
>> On Tue, 31 Jul 2007, mouss wrote:
>>
>>> running SA at smtp time requires that the client does not timeout.
>>> so you'd better scan fast! you're also more subject to DOS (your
>>> smtp listeners are busy). compare this to queue and filter...
>>
>> okay, here's a sick idea:
>>
>> (1) MTA completes the SMTP exchange and responds with a 4xx after DATA
>> finishes.
>>
>> (2) MTA passes message off to SA, then stores a hash of message-ID/score.
>> MTA then discards the message.
>>
>> (3) When the remote MTA retries (if it retries) then the MTA looks up the
>> score in the hash and decides whether to 200 or 5xx the message.
>>
>> All of the benefits of both methods! :)
>>
>
> Sort of like grey listing, which I do run on my personal domain, but I
> wouldn't use that method because of the inherent delay caused by the 4xx
> retry.
>
> Neat idea though.

I agree, neat idea. However, all email messages coming into the server
would be delayed. Unlike greylisting, where the connection is accepted
after the initial 4xx rejection.

-------
   _|_
  (_| |

[Per Jessen]

Rick Macdougall wrote:

> Sort of like grey listing, which I do run on my personal domain, but I
> wouldn't use that method because of the inherent delay caused by the
> 4xx retry.

Only happens once though.


/Per Jessen, Zürich

[mouss-2]

John D. Hardin wrote:

> On Tue, 31 Jul 2007, mouss wrote:
>
>  
>> running SA at smtp time requires that the client does not timeout.
>> so you'd better scan fast! you're also more subject to DOS (your
>> smtp listeners are busy). compare this to queue and filter...
>>    
>
> okay, here's a sick idea:
>
> (1) MTA completes the SMTP exchange and responds with a 4xx after DATA
> finishes.
>
> (2) MTA passes message off to SA, then stores a hash of
> message-ID/score. MTA then discards the message.
>
> (3) When the remote MTA retries (if it retries) then the MTA looks up
> the score in the hash and decides whether to 200 or 5xx the message.
>
> All of the benefits of both methods! :)
>  

This can be improved at the cost of code complexity: run SA, and if it
does not return in "due time", then return 4xx. So if you scan fast, you
reject or accept. otherwise, you tempfail.

Either way, this requires "some work". and in general, this is not worth
the pain. mostly when users "want their mail now" (and don't say "this
is not instant messaging", they just don't understand what you're
talking about!). This is why I favour: filter, tag and deliver: if user
wants, "spam" goes to Junk folder or elsewhere (special address,
quarantine, ..;etc). It's his mail, he will know what to do with!