3.3.0 plans

[Justin Mason]

> Checking if your kit is complete...
> Looks good
> 'ENABLE_SSL' is not a known MakeMaker parameter name.
> 'SYSCONFDIR' is not a known MakeMaker parameter name.
> Writing Makefile for Mail::SpamAssassin
> Makefile written by ExtUtils::MakeMaker 6.42
> + /usr/bin/make 'OPTIMIZE=-O2 -g' -j3
>
> spamassassin-3.2.5 does not complain about ENABLE_SSL or SYSCONFDIR in
> MakeMaker.  What changed?  Do we need to adapt?

no, that should be fine.  (we should probably fix it ;)

btw, this should work --

perl Makefile.PL PREFIX=/tmp/sa
make
make install
/tmp/sa/bin/sa-update

does that work for you?  even without an /etc/mail/spamassassin or
/usr/share/spamassassin?  if so it's definitely the .spec file IMO.

--j.

[Justin Mason]

On Mon, Jul 6, 2009 at 22:40, Warren Togami<wtogami@...> wrote:

> On 07/06/2009 05:32 PM, Justin Mason wrote:
>>
>> On Mon, Jul 6, 2009 at 21:05, Warren Togami<wtogami@...>  wrote:
>>>
>>> On 07/06/2009 04:00 PM, Mark Martinec wrote:
>>>>
>>>> Warren,
>>>>
>>>>> On 07/03/2009 10:37 PM, Matt Kettler wrote:
>>>>>>
>>>>>> The public alpha release was announced yesterday on the users mailing
>>>>>> list:
>>>>>>
>>>>>>
>>>>>>
>>>>>> http://mail-archives.apache.org/mod_mbox/spamassassin-users/200907.mbox/%
>>>>>> 3C6c399e450907021522k1678f0ffn454a1e670f064b40@...%3E
>>>>>
>>>>> I'm having trouble getting it to run on Fedora 11.
>>>>>
>>>>> [root@newcaprica spamassassin]#
>>>>>   sa-update config: no configuration text or files found!
>>>>>   do you need to run 'sa-update'?
>>>>> check: no loaded plugin implements 'check_main': cannot scan!
>>>>> Check the necessary '.pre' files are in the config directory.
>>>>
>>>> Yes, and?  Did you run sa-update???
>>>>
>>>> The 3.3 no longer comes with rules in the same package.
>>>> These must be installed separately with 'sa-update',
>>>> which either fetches them from the net, or can install
>>>> them from a tar - which is in the same directory
>>>> as 3.3.0-alpha1 is.
>>>>
>>>>   Mark
>>>
>>> sa-update is failing due to the lack of the /etc/mail/spamassassin/*.pre
>>> files.
>>>
>>
>> this sounds like our RPM spec file is buggy -- does it work if
>> installed from the tgz?
>>
>> --j.
>
> This isn't the upstream RPM spec file.  This is Fedora's spec file.
>
> How can it copy the *.pre files into the RPM if the *.pre files do not exist
> in the tarball's rules/ directory?

crap, you're right. :(
https://issues.apache.org/SpamAssassin/show_bug.cgi?id=6147

there's a patch there that appears to fix it.  does it work for you?

--j.

[Warren Togami]

On 07/06/2009 06:12 PM, Justin Mason wrote:
>
> crap, you're right. :(
> https://issues.apache.org/SpamAssassin/show_bug.cgi?id=6147
>
> there's a patch there that appears to fix it.  does it work for you?
>
> --j.

A patch only for MANIFEST* going to copy *.pre files from the tarball,
when the tarball lacks those files?

It seems MANIFEST* only effects what goes from the SCM into tarball.

Anyhow, I'm adding the *.pre files manually for the initial package.

Warren Togami
wtogami@...

[Justin Mason]

On Mon, Jul 6, 2009 at 23:36, Warren Togami<wtogami@...> wrote:

> On 07/06/2009 06:12 PM, Justin Mason wrote:
>>
>> crap, you're right. :(
>> https://issues.apache.org/SpamAssassin/show_bug.cgi?id=6147
>>
>> there's a patch there that appears to fix it.  does it work for you?
>>
>> --j.
>
> A patch only for MANIFEST* going to copy *.pre files from the tarball, when
> the tarball lacks those files?
>
> It seems MANIFEST* only effects what goes from the SCM into tarball.

yep, you'd have to rebuild the tarball.

> Anyhow, I'm adding the *.pre files manually for the initial package.

ok

--j.

[Warren Togami]

On 07/07/2009 04:57 AM, Justin Mason wrote:

> On Mon, Jul 6, 2009 at 23:36, Warren Togami<wtogami@...>  wrote:
>> On 07/06/2009 06:12 PM, Justin Mason wrote:
>>> crap, you're right. :(
>>> https://issues.apache.org/SpamAssassin/show_bug.cgi?id=6147
>>>
>>> there's a patch there that appears to fix it.  does it work for you?
>>>
>>> --j.
>> A patch only for MANIFEST* going to copy *.pre files from the tarball, when
>> the tarball lacks those files?
>>
>> It seems MANIFEST* only effects what goes from the SCM into tarball.
>
> yep, you'd have to rebuild the tarball.
>
>> Anyhow, I'm adding the *.pre files manually for the initial package.
>
> ok

But now I'm unable to push this packaged due to the tainting issue.  Is
this is in fact a perl bug, not spamassassin's fault?  This is the
latest version of upstream perl.

Warren Togami
wtogami@...

[Justin Mason]

On Tue, Jul 7, 2009 at 10:07, Warren Togami<wtogami@...> wrote:

> On 07/07/2009 04:57 AM, Justin Mason wrote:
>>
>> On Mon, Jul 6, 2009 at 23:36, Warren Togami<wtogami@...>  wrote:
>>>
>>> On 07/06/2009 06:12 PM, Justin Mason wrote:
>>>>
>>>> crap, you're right. :(
>>>> https://issues.apache.org/SpamAssassin/show_bug.cgi?id=6147
>>>>
>>>> there's a patch there that appears to fix it.  does it work for you?
>>>>
>>>> --j.
>>>
>>> A patch only for MANIFEST* going to copy *.pre files from the tarball,
>>> when
>>> the tarball lacks those files?
>>>
>>> It seems MANIFEST* only effects what goes from the SCM into tarball.
>>
>> yep, you'd have to rebuild the tarball.
>>
>>> Anyhow, I'm adding the *.pre files manually for the initial package.
>>
>> ok
>
> But now I'm unable to push this packaged due to the tainting issue.  Is this
> is in fact a perl bug, not spamassassin's fault?  This is the latest version
> of upstream perl.

I haven't looked into it, but iirc, it's a perl bug. (at least a bug
in a module bundled with perl.)  We trigger it due to our extensive
use of -T (taint mode).  As Mark notes, we may be able to work around
it in our code though.

--j.

[Warren Togami]

Thanks for kicking this off.  My RPM packages with packaged rules seem
to be working well on RHEL4, RHEL5 and Fedora 10+.

Do we have enough variety of users submitting nightly mass check data?

What remains to be fixed before 3.3.0?

What is a theoretical schedule for 3.3.0?

Warren Togami
wtogami@...

[Justin Mason]

On Wed, Jul 15, 2009 at 04:14, Warren Togami<wtogami@...> wrote:
> Thanks for kicking this off.  My RPM packages with packaged rules seem to be
> working well on RHEL4, RHEL5 and Fedora 10+.

excellent!

> Do we have enough variety of users submitting nightly mass check data?

we still need some more, I think.

MSECS      SPAM%     HAM%     S/O    RANK   SCORE  NAME WHO/AGE
0.00000  46.7930   0.0431   0.999    0.96    0.01  T_CN_URL
0.00000   1.3699   0.0000   1.000    0.46    0.01  T_CN_URL bb-jhardin
0.00000  68.2628   0.0000   1.000    0.99    0.01  T_CN_URL bb-jm
0.00000  44.9004   0.0132   1.000    0.99    0.01  T_CN_URL dos
0.00000  19.0590   0.0160   0.999    0.94    0.01  T_CN_URL jm
0.00000   2.7273   0.0000   1.000    0.75    0.01  T_CN_URL wtogami
0.00000   5.6111   0.2874   0.951    0.66    0.01  T_CN_URL zmi

That's not bad, but it could be better.  In numbers, the nonspam is
overwhelmingly coming from my corpus and Daryl's (which has some FPs,
I need to mail those on to you D).


> What remains to be fixed before 3.3.0?

can you see this?
https://issues.apache.org/SpamAssassin/buglist.cgi?cmdtype=runnamed&namedcmd=33bugs

if not, try this:

https://issues.apache.org/SpamAssassin/buglist.cgi?query_format=advanced&short_desc_type=allwordssubstr&short_desc=&target_milestone=3.3.0&long_desc_type=allwordssubstr&long_desc=&bug_file_loc_type=allwordssubstr&bug_file_loc=&status_whiteboard_type=allwordssubstr&status_whiteboard=&keywords_type=allwords&keywords=&bug_status=NEW&bug_status=ASSIGNED&bug_status=REOPENED&emailtype1=substring&email1=&emailtype2=substring&email2=&bugidtype=include&bug_id=&chfieldfrom=&chfieldto=Now&chfieldvalue=&cmdtype=doit&order=Reuse+same+sort+as+last+time&known_name=33bugs&query_based_on=33bugs&field0-0-0=noop&type0-0-0=noop&value0-0-0=

very little needs to be done.  Right now I'm thinking just the P1s and P2s:

6143     nor   P1   NEW     Rule2XSBody segfaults due to rule
containing NUL chars
6144 blo P1 NEW write changes list for 3.3.0 release
6155 blo P1 NEW generate new scores for 3.3.0 release
4949 nor P2 NEW spamd shutting down when unable to fork new processes
6003 nor P2 NEW whitelist_from_rcvd propagates to other users
6132 enh P2 REOP FreeMail plugin
6150 maj P2 REOP spamd fails: /usr/bin/spamd line 2504

> What is a theoretical schedule for 3.3.0?

I haven't come up with real dates, but I think we could have this
released (ie out of beta) in 2 months.

--j.

[Warren Togami]

On 07/15/2009 06:41 AM, Justin Mason wrote:

>> Do we have enough variety of users submitting nightly mass check data?
>
> we still need some more, I think.
>
> MSECS      SPAM%     HAM%     S/O    RANK   SCORE  NAME WHO/AGE
> 0.00000  46.7930   0.0431   0.999    0.96    0.01  T_CN_URL
> 0.00000   1.3699   0.0000   1.000    0.46    0.01  T_CN_URL bb-jhardin
> 0.00000  68.2628   0.0000   1.000    0.99    0.01  T_CN_URL bb-jm
> 0.00000  44.9004   0.0132   1.000    0.99    0.01  T_CN_URL dos
> 0.00000  19.0590   0.0160   0.999    0.94    0.01  T_CN_URL jm
> 0.00000   2.7273   0.0000   1.000    0.75    0.01  T_CN_URL wtogami
> 0.00000   5.6111   0.2874   0.951    0.66    0.01  T_CN_URL zmi
>

What do each of these columns mean?

Warren

[Justin Mason]

On Wed, Jul 15, 2009 at 11:51, Warren Togami<wtogami@...> wrote:

> On 07/15/2009 06:41 AM, Justin Mason wrote:
>>>
>>> Do we have enough variety of users submitting nightly mass check data?
>>
>> we still need some more, I think.
>>
>> MSECS      SPAM%     HAM%     S/O    RANK   SCORE  NAME WHO/AGE
>> 0.00000  46.7930   0.0431   0.999    0.96    0.01  T_CN_URL
>> 0.00000   1.3699   0.0000   1.000    0.46    0.01  T_CN_URL bb-jhardin
>> 0.00000  68.2628   0.0000   1.000    0.99    0.01  T_CN_URL bb-jm
>> 0.00000  44.9004   0.0132   1.000    0.99    0.01  T_CN_URL dos
>> 0.00000  19.0590   0.0160   0.999    0.94    0.01  T_CN_URL jm
>> 0.00000   2.7273   0.0000   1.000    0.75    0.01  T_CN_URL wtogami
>> 0.00000   5.6111   0.2874   0.951    0.66    0.01  T_CN_URL zmi
>>
>
> What do each of these columns mean?

see http://wiki.apache.org/spamassassin/MassCheck .  But the only bit
that's relevant to this discussion is the "WHO" part -- ie those
contributor accounts at the end of each line.

--j.

[Justin Mason]

On Wed, Jul 15, 2009 at 12:17, Justin Mason<jm@...> wrote:

> On Wed, Jul 15, 2009 at 11:51, Warren Togami<wtogami@...> wrote:
>> On 07/15/2009 06:41 AM, Justin Mason wrote:
>>>>
>>>> Do we have enough variety of users submitting nightly mass check data?
>>>
>>> we still need some more, I think.
>>>
>>> MSECS      SPAM%     HAM%     S/O    RANK   SCORE  NAME WHO/AGE
>>> 0.00000  46.7930   0.0431   0.999    0.96    0.01  T_CN_URL
>>> 0.00000   1.3699   0.0000   1.000    0.46    0.01  T_CN_URL bb-jhardin
>>> 0.00000  68.2628   0.0000   1.000    0.99    0.01  T_CN_URL bb-jm
>>> 0.00000  44.9004   0.0132   1.000    0.99    0.01  T_CN_URL dos
>>> 0.00000  19.0590   0.0160   0.999    0.94    0.01  T_CN_URL jm
>>> 0.00000   2.7273   0.0000   1.000    0.75    0.01  T_CN_URL wtogami
>>> 0.00000   5.6111   0.2874   0.951    0.66    0.01  T_CN_URL zmi
>>>
>>
>> What do each of these columns mean?
>
> see http://wiki.apache.org/spamassassin/MassCheck

uh, actually, http://wiki.apache.org/spamassassin/HitFrequencies


--
--j.