554 deferral or failure?

[frederickmc-2]

Hi,
In my logs, for outgoing mail, I have a lot of lines where the receiving server responded with 554, yet it is logged as
a deferral rather than a failure. I'm seeing this for 571 also. I thought that all smtp codes >=500 were supposed to be
permanant failures.

I'm running netqmail-1.06 with the following patches:
http://pyropus.ca/software/misc/qmail-1.03-domainbindings-1.2.patch
qmail-smtpd-viruscan-1.3.patch
big-concurrency.patch
big-ext-todo-20030101.patch
doublebounce-trim.patch

In qmail-remote.c (line 227 among other places) I see:
if (code >= 500) quit("DConnected to "," but sender was rejected");

and in qmail-send.c
         case 'D':
           log3("delivery ",strnum3,": failure: ");

which indicate to me that it should be a failure rather than deferral.

Any suggestions?

Thanks,
Fred Mc

[Adi Pircalabu]

On Sun, 1 Nov 2009 14:39:02 -0800 Fred McIntyre wrote:

> In my logs, for outgoing mail, I have a lot of lines where the
> receiving server responded with 554, yet it is logged as a deferral
> rather than a failure. I'm seeing this for 571 also. I thought that
> all smtp codes >=500 were supposed to be permanant failures.
[...]
> In qmail-remote.c (line 227 among other places) I see:
> if (code >= 500) quit("DConnected to "," but sender was rejected");
>
> and in qmail-send.c
>          case 'D':
>            log3("delivery ",strnum3,": failure: ");
>
> which indicate to me that it should be a failure rather than deferral.

http://www.faqs.org/rfcs/rfc821.html
For SMTP, a failure is a failure, regardless of whether it's permanent
(5xx) or temporary (4xx) one.
It's also common sense to use "failure" for every type of event which
did not finished successfully. Failure to understand this is still a
failure :)

--
Adi Pircalabu

[Markus Stumpf-4]

On Sun, Nov 01, 2009 at 02:39:02PM -0800, Fred McIntyre wrote:
> I thought that all smtp codes >=500 were supposed to be
> permanant failures.

They are.

> In qmail-remote.c (line 227 among other places) I see:

Hmmm, are you sure that the qmail-remote.c you are looking at is the
same from which the executable was created?

I have never seen the described behaviour and none of the patches seems
to touch qmail-remote, so this would be my only guess.

        \Maex

[Markus Stumpf-4]

On Mon, Nov 02, 2009 at 10:29:57AM +1100, Adi Pircalabu wrote:
> For SMTP, a failure is a failure, regardless of whether it's permanent
> (5xx) or temporary (4xx) one.
> It's also common sense to use "failure" for every type of event which
> did not finished successfully. Failure to understand this is still a
> failure :)

You are so funny.

        \Maex

[Kyle Wheeler-2]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

On Sunday, November  1 at 02:39 PM, quoth Fred McIntyre:
> In my logs, for outgoing mail, I have a lot of lines where the
> receiving server responded with 554, yet it is logged as a deferral
> rather than a failure. I'm seeing this for 571 also. I thought that
> all smtp codes >=500 were supposed to be permanant failures.

It depends on when the error is given. For example, if a 554 error is
given upon connection, what does it mean?

Okay, it probably means that the recipient mail server is run by an
idiot, but (unfortunately) there are idiots out there (for details,
see http://www.memoryhole.net/qmail/#rfc2821). Anyway, should qmail
assume that the receiving server knew which message I was intending to
send it (despite knowing nothing about it other than that it was
coming from my server), and that the message I was intending to send
should be treated as having received a permanent failure error? The
answer is: no, of course not. Just because it's a 5xx error doesn't
mean that the *message* was permanently rejected.

Here's another example: when I send the SMTP command "QUIT", the
server I'm connected to is SUPPOSED to respond with a 221 code. By
that point it's already claimed to have successfully accepted all
messages that I've sent. But if the server respond with a 554 code
instead, what does THAT mean? Did any of the messages get sent? Did
any fail? (I've never actually seen this happen, but theoretically it
could.)

There are lots of opportunities for generating a 5xx code during an
SMTP connection that either make no sense or have NOTHING to do with
the message(s) you're attempting to send.

Make sense?

~Kyle
- --
It is better to dwell in the wilderness than with a contentious and an
angry woman.
                                                      -- Proverbs 21:19
-----BEGIN PGP SIGNATURE-----
Comment: Thank you for using encryption!

iQIcBAEBCAAGBQJK7lt8AAoJECuveozR/AWeNoUP/RG9Y/TPfe+qFKwmaaE7Uhvx
1tOm/EDn1KVd07UgdJE4//jV2Vm4o+Ddg119szqjYjasTnf1JAvn5RT9+4mLfZrS
HzOEYE/iUBIGOUuiexidcRnTK+1TIRjn+3m7NOFKEdB3LV8Ci7/I5KBZPEy9dYxw
wYcpmlaha8+vL3/5BONkVSpTr+3M3Naj7N58dT3RalukEYXVgWEBaLKDWm5R39nb
eQQlOljAGz2jtPb0X9sZG2wxkcYDSioAzicW1RJ1w36M4HJjZBLF7qPZ+lPH5UMY
h+ZiYlGPd28j6zUt4WkDM7FuIOfnHoB9oWhOpwXPHLpCW/r7D99zG0B87QFvcwSF
4qG69Skr96ahdiZ51gF3Mi0D57OG7CFh/YA/miH3xukJaxDImwPK2VVjS6LMR54e
5SpFZuqL42z49AP6Yzz5pRxu19QCRRBUNCRGYwG56QWuH2bskm7UvxBGGun+9eqZ
kbpnLxA+BEcsdBfb18igEvXkRcTuXnkBzqIWASRbAv/N1rPIXApA5qR1fia5Odl8
iVwbd+9P8ruGXv9kRcvzWqzLp+SUkd97mWO3HRyff2QEiwBeFI8gU2fOkKmDGcaU
Lk+dgLlUduva+CE+GqCiV6875ue0qVSUbsZ299Nay9RsDdiqvLK6uW9eafOCAE5y
zAqEnwxk7095RuwUTbLH
=mCox
-----END PGP SIGNATURE-----

[John Levine]

>It depends on when the error is given. For example, if a 554 error is
>given upon connection, what does it mean?

According to RFC 5321 it means "No SMTP service here", or to put it
more concisely, FOAD.  Whatever you thought you were going to send,
forget it, treat it as a failure and don't try again.

>answer is: no, of course not. Just because it's a 5xx error doesn't
>mean that the *message* was permanently rejected.

Well, yeah, actually, it does.  Typically it means that the host
has blacklisted you and doesn't want your mail.

>Here's another example: when I send the SMTP command "QUIT", the
>server I'm connected to is SUPPOSED to respond with a 221 code. By
>that point it's already claimed to have successfully accepted all
>messages that I've sent. But if the server respond with a 554 code
>instead, what does THAT mean?

That's broken, QUIT can only return 221.  Treat it any way you want.

R's,
John

[Kyle Wheeler-2]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

On Monday, November  2 at 05:06 AM, quoth John Levine:
>> It depends on when the error is given. For example, if a 554 error is
>> given upon connection, what does it mean?
>
> According to RFC 5321 it means "No SMTP service here", or to put it
> more concisely, FOAD.  Whatever you thought you were going to send,
> forget it, treat it as a failure and don't try again.

RFC 5321 is a draft barely a year old, so you'll forgive me for not
being intimately familiar with its contents (heck, DKIM is more
venerable). :)

The behavior that I was describing is often used by misguided
administrators following a policy whereby their most-preferable
(lowest-ranked) MX record is a fake "SMTP" server that responds to all
commands with a 554 error. The idea (and, as far as I can see, RFC
5321 merely codifies this idea) is that the sender should then
re-attempt the delivery by contacting the next-most-preferable MX. It
should not cause a termination of the message you were attempting to
deliver, but should instead be treated as if the connection could not
be established. (You'd think that simply refusing the connection would
be easier and more efficient, but...)

Section 4.2.2 says:

    554  Transaction failed (or, in the case of a connection-opening
         response, "No SMTP service here")

Now, given that a greeting 554 is technically (according to this RFC)
part of the SMTP protocol, that's kinda like saying "I do not speak
English". But it's explicitly *not* saying FOAD, and as far as I can
see should have the exact same effect as simply refusing the
connection.

But perhaps I'm misreading this new RFC. Where does it say that the
error should be treated as a permanent termination of any and all
messages you were intending to deliver?

>> answer is: no, of course not. Just because it's a 5xx error doesn't
>> mean that the *message* was permanently rejected.
>
> Well, yeah, actually, it does.  Typically it means that the host
> has blacklisted you and doesn't want your mail.

Really? In my experience, it virtually always means the admin of the
recipient domain is following a (broken) modified "nolisting"
anti-spam policy.

>> Here's another example: when I send the SMTP command "QUIT", the
>> server I'm connected to is SUPPOSED to respond with a 221 code. By
>> that point it's already claimed to have successfully accepted all
>> messages that I've sent. But if the server respond with a 554 code
>> instead, what does THAT mean?
>
> That's broken, QUIT can only return 221.  Treat it any way you want.

I agree that it's broken; but my point is that recipient servers can
behave badly in a variety of ways, and in particular can generate
bogus errors at a variety of points that would cause qmail to log an
error message that does not indicate that the message delivery
actually failed (either permanently or temporarily).

~Kyle
- --
Coffee is the common man's gold, and like gold, it brings to every
person the feeling of luxury and nobility.
                                                 -- Sheik Abd-al-Kadir
-----BEGIN PGP SIGNATURE-----
Comment: Thank you for using encryption!

iQIVAwUBSu8OviuveozR/AWeAQhEzA//fATlwkENxAGq6wcxpPNuGZeUkV9EJg4x
3bMLdj0lryXj7xQOjXyY/77xuuHqL20/lwUILCxYtItNct8YQBJj5tjW/a5D3TM2
WOPrS/3nzu4ZthdN0w2sWazR8Zo+vZ/ImqRyRslL1kODyCB6r/G9TNaHmkY2Q+GZ
kku/Ki+/q4JmtHfaxBCiqGRiGbtVuO53SHf7Z1MBU+5chtG/z98bvDUtd7Lmb3RM
skO0DUdgedNb6+az+a//1rM3np3eEIas5XmGSGTN+jwD437+HgPNmnLMSD54g+zV
XT2nmjFbDzo/pmo6rnnGtktgsC7/y6t3G3u55NSNCk0d0E3itWM04wwf8mVdb+f7
PQxcSTVqPfPwZ4/ghIWfVCv2FMOGAjJIxclfT+pKTsFqAs6I75MO2YNgBPXL4f2R
Wu+10zo2e0khTH1An3CAejzahHVZIHBCzKb5Dj5SeF/xCmvvcPt5c7J+7kjdE7O6
rlZB+XUo7eDoXfmCF7NxipNlRlgjtMDBuk84wd10QyhgHqwFCkasShFk1QkJlbYP
MxM4KIkwRh1UaXhm6GckHGEEctEeaZBvoCDmL0LFkcXQw4vx6W99/Y8prcDAScms
c9pE2rXdqflOHgJ5NJw/5DvZXoQNOTm9Hy5a82qVfGOZb5PTOQmp5uOmT1r4refj
DdF8psZGww0=
=fpN2
-----END PGP SIGNATURE-----