good Spamassassin Summary report

[Dan Schaefer-3]

I have searched far and wide for a good Spamassassin report using
numerous keywords in Google searches, but I can't find the one that fits
my needs. I am looking for a script that can be run via cron job on a
daily basis. I would pass the script the location of the mail log. The
output will show me for each rule, how many times that rule passed the
test. It will only show me the rules where the count is more than 0. For
example:

ALL_TRUSTED 287
BAYES_00   67
BAYES_10   43
BAYES_20   23
...
RCVD_IN_PBL   25


If you have found something similar to this, good. If you have created
your own script to do this, better. If Spamassassin has this script
created already and I missed it, even better.

Thanks in advance,
Dan Schaefer
Application Developer
Performance Administration Corp.

[Rick Macdougall-2]

Daniel Schaefer wrote:

> I have searched far and wide for a good Spamassassin report using
> numerous keywords in Google searches, but I can't find the one that fits
> my needs. I am looking for a script that can be run via cron job on a
> daily basis. I would pass the script the location of the mail log. The
> output will show me for each rule, how many times that rule passed the
> test. It will only show me the rules where the count is more than 0. For
> example:
>
> ALL_TRUSTED 287
> BAYES_00   67
> BAYES_10   43
> BAYES_20   23
> ...
> RCVD_IN_PBL   25
>
>

Hi,

Does this do what you want ?

http://www.rulesemporium.com/programs/sa-stats.txt

Sample Output

Time Spent Running SA:         1.68 hours
Time Spent Processing Spam:    0.29 hours
Time Spent Processing Ham:     1.39 hours

TOP SPAM RULES FIRED
----------------------------------------------------------------------
RANK    RULE NAME                       COUNT  %OFMAIL %OFSPAM  %OFHAM
----------------------------------------------------------------------
    1    HTML_MESSAGE                      824    77.07   88.13   74.20
    2    RAZOR2_CHECK                      772    19.61   82.57    3.32
    3    RAZOR2_CF_RANGE_51_100            753    18.21   80.53    2.08
    4    RAZOR2_CF_RANGE_E8_51_100         713    17.19   76.26    1.91
    5    URIBL_BLACK                       652    16.03   69.73    2.13
    6    MIME_HTML_ONLY                    609    29.64   65.13   20.45

Regards,

Rick

[Dan Schaefer-3]

Yes, actually it is exactly what I'm looking for. I saw another sa-stats
script that only showed the %'s for HAM and SPAM and the average score
and what not. Thank you sir for sending me this.

Dan Schaefer
Application Developer
Performance Administration Corp.



Rick Macdougall wrote:

> Daniel Schaefer wrote:
>> I have searched far and wide for a good Spamassassin report using
>> numerous keywords in Google searches, but I can't find the one that
>> fits my needs. I am looking for a script that can be run via cron job
>> on a daily basis. I would pass the script the location of the mail
>> log. The output will show me for each rule, how many times that rule
>> passed the test. It will only show me the rules where the count is
>> more than 0. For example:
>>
>> ALL_TRUSTED 287
>> BAYES_00   67
>> BAYES_10   43
>> BAYES_20   23
>> ...
>> RCVD_IN_PBL   25
>>
>>
>
> Hi,
>
> Does this do what you want ?
>
> http://www.rulesemporium.com/programs/sa-stats.txt
>
> Sample Output
>
> Time Spent Running SA:         1.68 hours
> Time Spent Processing Spam:    0.29 hours
> Time Spent Processing Ham:     1.39 hours
>
> TOP SPAM RULES FIRED
> ----------------------------------------------------------------------
> RANK    RULE NAME                       COUNT  %OFMAIL %OFSPAM  %OFHAM
> ----------------------------------------------------------------------
>    1    HTML_MESSAGE                      824    77.07   88.13   74.20
>    2    RAZOR2_CHECK                      772    19.61   82.57    3.32
>    3    RAZOR2_CF_RANGE_51_100            753    18.21   80.53    2.08
>    4    RAZOR2_CF_RANGE_E8_51_100         713    17.19   76.26    1.91
>    5    URIBL_BLACK                       652    16.03   69.73    2.13
>    6    MIME_HTML_ONLY                    609    29.64   65.13   20.45
>
> Regards,
>
> Rick
>

[Dan Schaefer-3]

I guess there's one thing missing. I can't enter a date range...(today,
yesterday, etc).

Dan Schaefer
Application Developer
Performance Administration Corp.



Daniel Schaefer wrote:

> Yes, actually it is exactly what I'm looking for. I saw another
> sa-stats script that only showed the %'s for HAM and SPAM and the
> average score and what not. Thank you sir for sending me this.
>
> Dan Schaefer
> Application Developer
> Performance Administration Corp.
>
>
>
> Rick Macdougall wrote:
>> Daniel Schaefer wrote:
>>> I have searched far and wide for a good Spamassassin report using
>>> numerous keywords in Google searches, but I can't find the one that
>>> fits my needs. I am looking for a script that can be run via cron
>>> job on a daily basis. I would pass the script the location of the
>>> mail log. The output will show me for each rule, how many times that
>>> rule passed the test. It will only show me the rules where the count
>>> is more than 0. For example:
>>>
>>> ALL_TRUSTED 287
>>> BAYES_00   67
>>> BAYES_10   43
>>> BAYES_20   23
>>> ...
>>> RCVD_IN_PBL   25
>>>
>>>
>>
>> Hi,
>>
>> Does this do what you want ?
>>
>> http://www.rulesemporium.com/programs/sa-stats.txt
>>
>> Sample Output
>>
>> Time Spent Running SA:         1.68 hours
>> Time Spent Processing Spam:    0.29 hours
>> Time Spent Processing Ham:     1.39 hours
>>
>> TOP SPAM RULES FIRED
>> ----------------------------------------------------------------------
>> RANK    RULE NAME                       COUNT  %OFMAIL %OFSPAM  %OFHAM
>> ----------------------------------------------------------------------
>>    1    HTML_MESSAGE                      824    77.07   88.13   74.20
>>    2    RAZOR2_CHECK                      772    19.61   82.57    3.32
>>    3    RAZOR2_CF_RANGE_51_100            753    18.21   80.53    2.08
>>    4    RAZOR2_CF_RANGE_E8_51_100         713    17.19   76.26    1.91
>>    5    URIBL_BLACK                       652    16.03   69.73    2.13
>>    6    MIME_HTML_ONLY                    609    29.64   65.13   20.45
>>
>> Regards,
>>
>> Rick
>>

[Martin Gregorie-2]

On Fri, 2009-07-03 at 12:03 -0400, Daniel Schaefer wrote:
> If you have found something similar to this, good. If you have created
> your own script to do this, better. If Spamassassin has this script
> created already and I missed it, even better.
>
I wrote my own but it is somewhat specialized because:

- it runs as part of the logwatch report rather than being a cron job.
  This is the easiest way I know to restrict the scan to the last 24
  hours of the maillog.

- one section of its report comes from maillog entries generated by my
  spamkiller utility, but this just shows totals for clean, spam and
  total messages.

- it is designed to monitor only my own custom rules. It reads local.cf
  to get a list of them and ignores everything else.

- by default it only reports the top ten firing rules.

- it has options to list all rules alphabetically or ranked by hit rate.

- written in Perl, but what else would you expect from an SA reporting
  tool?
 

Martin

[Dan Schaefer-3]

Cool. Having it as part of the Logwatch report would be just fine with
me. I have created a short logwatch script to count and show me a
running total of each spam score number, but your script I'm sure is a
lot better than mine. Would you be willing to release yours to the open
source community, or would you ask for some sort of compensation?

Dan Schaefer
Application Developer
Performance Administration Corp.



Martin Gregorie wrote:

> On Fri, 2009-07-03 at 12:03 -0400, Daniel Schaefer wrote:
>  
>> If you have found something similar to this, good. If you have created
>> your own script to do this, better. If Spamassassin has this script
>> created already and I missed it, even better.
>>
>>    
> I wrote my own but it is somewhat specialized because:
>
> - it runs as part of the logwatch report rather than being a cron job.
>   This is the easiest way I know to restrict the scan to the last 24
>   hours of the maillog.
>
> - one section of its report comes from maillog entries generated by my
>   spamkiller utility, but this just shows totals for clean, spam and
>   total messages.
>
> - it is designed to monitor only my own custom rules. It reads local.cf
>   to get a list of them and ignores everything else.
>
> - by default it only reports the top ten firing rules.
>
> - it has options to list all rules alphabetically or ranked by hit rate.
>
> - written in Perl, but what else would you expect from an SA reporting
>   tool?
>  
>
> Martin
>
>
>  

[John Hardin]

On Fri, 3 Jul 2009, Daniel Schaefer wrote:

> I guess there's one thing missing. I can't enter a date range...(today,
> yesterday, etc).

grep the desired date range out to a temporary log file copy and run the
analyzer against that.

>> >  http://www.rulesemporium.com/programs/sa-stats.txt

--
  John Hardin KA7OHZ                    http://www.impsec.org/~jhardin/
  jhardin@...    FALaholic #11174     pgpk -a jhardin@...
  key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C  AF76 D822 E6E6 B873 2E79
-----------------------------------------------------------------------
   USMC Rules of Gunfighting #6: If you can choose what to bring to a
   gunfight, bring a long gun and a friend with a long gun.
-----------------------------------------------------------------------
  Tomorrow: the 233rd anniversary of the Declaration of Independence

[Martin Gregorie-2]

On Fri, 2009-07-03 at 12:45 -0400, Daniel Schaefer wrote:
> Cool. Having it as part of the Logwatch report would be just fine with
> me. I have created a short logwatch script to count and show me a
> running total of each spam score number, but your script I'm sure is a
> lot better than mine. Would you be willing to release yours to the open
> source community, or would you ask for some sort of compensation?
>
I'd be happy to open source it as a way of giving something back to the
SA community. It needs a small modification to get the ham/spam stats
from spamd log lines rather than from my spamkiller log lines, but I
think that should be straight forward.

What's the normal way to pass scripts into the SA open source
collection?  Is there a central repository or is stuff just released on
personal websites? Either way works for me.


Martin

[Martin Gregorie-2]

My logwatch add-in has now been modified to work entirely from spamd log
messages, installed, tested and is now running as part of my standard
logwatch setup. It is available for download from:

http://www.gregorie.org/computing/spamscan/spamscan.tgz