Discrepancy between public servers

[Cedric Knight, GreenNet]

Hi

I run dccifd with SpamAssassin using the public DCC servers, and a user
has recently reported a non-bulk email getting caught as spam.  I
checked and it seems odd to me that there is such a large discrepancy
for the offending checksum between different DCC servers:

X-DCC-Misty-Metrics: mail 1170; Body=1 Fuz1=1 Fuz2=many
                                                      checksum  server
                 env_From: 28cec8de 444a5192 3d08f152 651fd1fb       0
                     From: 79af87a5 8716075c 1f9667c1 95b6f0b7       0
               Message-ID: d43994f1 08b825db 67624e9a 6b298130       0
                 Received: 5670dfb0 82821e39 71afee36 b217b67b       0
                     Body: 7e1aab3a 07098a85 b63e65cc 2f25ac14       1
                     Fuz1: ebc11451 b12abc2d 5f876d7c 8afcae55       1
                     Fuz2: 67279754 87ed81a3 be38bdad 1e7e51af    many

X-DCC-wuwien-Metrics: mail 1290; Body=0 Fuz1=0 Fuz2=0
                                                      checksum  server
                 env_From: 28cec8de 444a5192 3d08f152 651fd1fb       0
                     From: 79af87a5 8716075c 1f9667c1 95b6f0b7       0
               Message-ID: d43994f1 08b825db 67624e9a 6b298130       0
                 Received: 5670dfb0 82821e39 71afee36 b217b67b       0
                     Body: 7e1aab3a 07098a85 b63e65cc 2f25ac14       0
                     Fuz1: ebc11451 b12abc2d 5f876d7c 8afcae55       0
                     Fuz2: 67279754 87ed81a3 be38bdad 1e7e51af       0

X-DCC-z.dcc-servers-Metrics: mail 1049; Body=0 Fuz1=0 Fuz2=0
                                                      checksum  server
                 env_From: 28cec8de 444a5192 3d08f152 651fd1fb       0
                     From: 79af87a5 8716075c 1f9667c1 95b6f0b7       0
               Message-ID: d43994f1 08b825db 67624e9a 6b298130       0
                 Received: 5670dfb0 82821e39 71afee36 b217b67b       0
                     Body: 7e1aab3a 07098a85 b63e65cc 2f25ac14       0
                     Fuz1: ebc11451 b12abc2d 5f876d7c 8afcae55       0
                     Fuz2: 67279754 87ed81a3 be38bdad 1e7e51af       0

Variations of a factor of ten or so, or between 0 and 1, I could
understand, but how can the same checksum score 0 on most servers (at
most 3), and 'many' on Misty?  Could there be some corruption in the
database or the flood?

I've checked on some other hits, and it's not unique to that one sample:

$dccproc -Q -d -C <fp-dcc-bbc.eml
X-DCC-INFN-TO-Metrics: mail 1233; Body=0 Fuz1=0 Fuz2=0
                                                      checksum  server
                 env_From: a1f75b4b 45bac58c 7d0dc870 46b534f5       0
                     From: 238bdb31 2f80713d 38a823c6 e5c4cb0f       0
               Message-ID: 0cffe9bf f4ebc1d7 51f0c426 1be6e3a0       0
                 Received: ece268b8 8689246a 2f3bef01 b772b461       0
                     Body: e6b1758e b10d7a22 1077f4d5 1784d7f6       0
                     Fuz1: 41428ac8 318bc1fd 989470b4 f99a689a       0
                     Fuz2: f9747715 3248962f b345bd8d b5dbe63d       0

$ cdcc "add 71.246.8.99 RTT-4000 ms"
$ dccproc -Q -d -C <fp-dcc-bbc.eml
note recvfrom(???,0): Connection refused
X-DCC-Misty-Metrics: mail 1170; Body=0 Fuz1=0 Fuz2=many
                                                      checksum  server
                 env_From: a1f75b4b 45bac58c 7d0dc870 46b534f5       0
                     From: 238bdb31 2f80713d 38a823c6 e5c4cb0f       0
               Message-ID: 0cffe9bf f4ebc1d7 51f0c426 1be6e3a0       0
                 Received: ece268b8 8689246a 2f3bef01 b772b461       0
                     Body: e6b1758e b10d7a22 1077f4d5 1784d7f6       0
                     Fuz1: 41428ac8 318bc1fd 989470b4 f99a689a       0
                     Fuz2: f9747715 3248962f b345bd8d b5dbe63d    many

BTW I wrote to dcc@... as the contact listed for the server on
http://www.rhyolite.com/dcc/ but it bounced "550 5.1.1
<dcc@...>... User unknown"

Thanks for any help

CK
_______________________________________________
DCC mailing list      DCC@...
http://www.rhyolite.com/mailman/listinfo/dcc

[Vernon Schryver]

> From: Cedric Knight <cedric@...>

> I run dccifd with SpamAssassin using the public DCC servers, and a user
> has recently reported a non-bulk email getting caught as spam.

The "many" checksum value implies that a very similar message was
reported by a DCC client as spam.  The FUZ2 checksum ignores binary
bits, and so the similarity might be free mail provider advertising
or a user signature.  In other words, I am sure that as far the DCC
network is concerned, the message was bulk.

>                                                                 I
> checked and it seems odd to me that there is such a large discrepancy
> for the offending checksum between different DCC servers:

>                      Body: 7e1aab3a 07098a85 b63e65cc 2f25ac14       1
>                      Fuz1: ebc11451 b12abc2d 5f876d7c 8afcae55       1
>                      Fuz2: 67279754 87ed81a3 be38bdad 1e7e51af    many

>                      Body: 7e1aab3a 07098a85 b63e65cc 2f25ac14       0
>                      Fuz1: ebc11451 b12abc2d 5f876d7c 8afcae55       0
>                      Fuz2: 67279754 87ed81a3 be38bdad 1e7e51af       0

No, database corruption has very different symptoms.


> I've checked on some other hits, and it's not unique to that one sample:

That is more evidence that no database corruption is involved.

The message body checksum counts of 0, 0, and many for the second sample
suggest a database record that more than a day old that has been trimmed
of uninteresting counts.

The likely situation is that the mail messages are being reported
by only a few DCC clients and so reports of their checksums are not
being flooded throughout the DCC network.  
The mail systems that are reporting the mail without counts of "many"
are probably close as the packets fly to dcc.misty.com.
Because dcc.misty.com has more RAM than most of the public DCC servers
but receives less traffic than some other public servers, it uses
longer database expirations.  That would make it remember reports of
less bulky mail longer than other servers.

I've heard reports that seem similar.  One case involves a relatively
low volume newsletter operator who insists that his mail is not
spam, although some of the targets of his mail have evidently wired
their mail systems to report his mail with counts of "many" to the
DCC network.  Are the relevant messages in this case from a mailing list?



> $ cdcc "add 71.246.8.99 RTT-4000 ms"

I like creating new map files, as in
  rm -f /tmp/map
  cdcc -h /tmp "new map; add dcc.misty.com"

> $ dccproc -Q -d -C <fp-dcc-bbc.eml

dccproc knows about -i, but for such tests feeding the checksums to
`/var/dcc/libexec/dccsight -dCQ` seems easier.


> note recvfrom(???,0): Connection refused

The form of that debugging message that is supposed to indicate the
receipt of an ICMP Unreachable packet indicates that the current
version of the DCC software is not being used.
Besides, the "???" is obviously bogus, and I vaguely recall fixing
a relevant bug some time ago.


Vernon Schryver    vjs@...
_______________________________________________
DCC mailing list      DCC@...
http://www.rhyolite.com/mailman/listinfo/dcc