Whitelisted from addresses

[K Post]

Last thread "more on NotGreedyWhitelist" didn't get an indication if
the problem was going to be able to be fixed or if anyone else was
seeing the problem.

Here's the issue, with 2.0.1 rc 0.5.13 whitelisted from addresseses in
the header that differ from non-whitelisted "mail from" addresses
don't seem to get a message to be considered to be from a whitelisted.

This was supposed to have been fixed with 0.5.04 and I think I saw
that it was, but now it's happening again.  I don't know what version
this started with again, but it's a real problem here.

Any idea what could be wrong?  This goes way too deep for me to be
able to help pinpoint a recommended change to the code.

Thanks

------------------------------------------------------------------------------
Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day
trial. Simplify your report design, integration and deployment - and focus on
what you do best, core application coding. Discover what's new with
Crystal Reports now.  http://p.sf.net/sfu/bobj-july
_______________________________________________
Assp-test mailing list
Assp-test@...
https://lists.sourceforge.net/lists/listinfo/assp-test

[Thomas Eckardt/eck]

This feature works like expected.

whitelisted state is (was) never reached if:

- any address is redlisted
- NotGreedyWhitelist == 1
- NotGreedyWhitelist == 2 and MAIL FROM is not whitelisted
- if the sender has a local mail address
 
Thomas




K Post <nntp.post@...>
04.11.2009 20:12
Bitte antworten an
ASSP development mailing list <assp-test@...>


An
ASSP development mailing list <assp-test@...>
Kopie

Thema
[Assp-test] Whitelisted from addresses






Last thread "more on NotGreedyWhitelist" didn't get an indication if
the problem was going to be able to be fixed or if anyone else was
seeing the problem.

Here's the issue, with 2.0.1 rc 0.5.13 whitelisted from addresseses in
the header that differ from non-whitelisted "mail from" addresses
don't seem to get a message to be considered to be from a whitelisted.

This was supposed to have been fixed with 0.5.04 and I think I saw
that it was, but now it's happening again.  I don't know what version
this started with again, but it's a real problem here.

Any idea what could be wrong?  This goes way too deep for me to be
able to help pinpoint a recommended change to the code.

Thanks

------------------------------------------------------------------------------
Let Crystal Reports handle the reporting - Free Crystal Reports 2008
30-Day
trial. Simplify your report design, integration and deployment - and focus
on
what you do best, core application coding. Discover what's new with
Crystal Reports now.  http://p.sf.net/sfu/bobj-july
_______________________________________________
Assp-test mailing list
Assp-test@...
https://lists.sourceforge.net/lists/listinfo/assp-test




DISCLAIMER:
*******************************************************
This email and any files transmitted with it may be confidential, legally
privileged and protected in law and are intended solely for the use of the

individual to whom it is addressed.
This email was multiple times scanned for viruses. There should be no
known virus in this email!
*******************************************************


------------------------------------------------------------------------------
Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day
trial. Simplify your report design, integration and deployment - and focus on
what you do best, core application coding. Discover what's new with
Crystal Reports now.  http://p.sf.net/sfu/bobj-july
_______________________________________________
Assp-test mailing list
Assp-test@...
https://lists.sourceforge.net/lists/listinfo/assp-test

[K Post]

Thanks for the quick reply and recognizing how critical this problem is for us.

I've got NotGreedyWhitelist set to 1
check all addresses - one match for white - add all

Yet, I'm still getting plenty of bayesian flags from addresses that
are alrady whitelisted.

This was a problem before 0.5.04 (this is when notgreeywhitelist was
on or off).  I reported the problem and you fixed it in 0.5.04.  In a
version shortly thereafter, maybe even the 05/06 combo release, it
broke again.

Can you check what was changed in 04 and make sure that it's still
that way in .14?  If it is, it must be something else

We've got bayesian set to BLOCK, not score, which is probably why
we're seeing the errors in the block reports when score only people
might not.




On Wed, Nov 4, 2009 at 2:36 PM, Thomas Eckardt/eck
<Thomas.Eckardt@...> wrote:

> This feature works like expected.
>
> whitelisted state is (was) never reached if:
>
> - any address is redlisted
> - NotGreedyWhitelist == 1
> - NotGreedyWhitelist == 2 and MAIL FROM is not whitelisted
> - if the sender has a local mail address
>
> Thomas
>
>
>
>
> K Post <nntp.post@...>
> 04.11.2009 20:12
> Bitte antworten an
> ASSP development mailing list <assp-test@...>
>
>
> An
> ASSP development mailing list <assp-test@...>
> Kopie
>
> Thema
> [Assp-test] Whitelisted from addresses
>
>
>
>
>
>
> Last thread "more on NotGreedyWhitelist" didn't get an indication if
> the problem was going to be able to be fixed or if anyone else was
> seeing the problem.
>
> Here's the issue, with 2.0.1 rc 0.5.13 whitelisted from addresseses in
> the header that differ from non-whitelisted "mail from" addresses
> don't seem to get a message to be considered to be from a whitelisted.
>
> This was supposed to have been fixed with 0.5.04 and I think I saw
> that it was, but now it's happening again.  I don't know what version
> this started with again, but it's a real problem here.
>
> Any idea what could be wrong?  This goes way too deep for me to be
> able to help pinpoint a recommended change to the code.
>
> Thanks
>
> ------------------------------------------------------------------------------
> Let Crystal Reports handle the reporting - Free Crystal Reports 2008
> 30-Day
> trial. Simplify your report design, integration and deployment - and focus
> on
> what you do best, core application coding. Discover what's new with
> Crystal Reports now.  http://p.sf.net/sfu/bobj-july
> _______________________________________________
> Assp-test mailing list
> Assp-test@...
> https://lists.sourceforge.net/lists/listinfo/assp-test
>
>
>
>
> DISCLAIMER:
> *******************************************************
> This email and any files transmitted with it may be confidential, legally
> privileged and protected in law and are intended solely for the use of the
>
> individual to whom it is addressed.
> This email was multiple times scanned for viruses. There should be no
> known virus in this email!
> *******************************************************
>
>
> ------------------------------------------------------------------------------
> Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day
> trial. Simplify your report design, integration and deployment - and focus on
> what you do best, core application coding. Discover what's new with
> Crystal Reports now.  http://p.sf.net/sfu/bobj-july
> _______________________________________________
> Assp-test mailing list
> Assp-test@...
> https://lists.sourceforge.net/lists/listinfo/assp-test
>

------------------------------------------------------------------------------
Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day
trial. Simplify your report design, integration and deployment - and focus on
what you do best, core application coding. Discover what's new with
Crystal Reports now.  http://p.sf.net/sfu/bobj-july
_______________________________________________
Assp-test mailing list
Assp-test@...
https://lists.sourceforge.net/lists/listinfo/assp-test

[Thomas Eckardt/eck]

>I've got NotGreedyWhitelist set to 1
>check all addresses - one match for white - add all

This is not possible.

'check all addresses - one match for white - add all'   sets   '
NotGreedyWhitelist'  to 0

I'll not change anything in sub onwhitelist, as long as nobody else is
reporting a problem there.

Thomas




K Post <nntp.post@...>
04.11.2009 21:08
Bitte antworten an
ASSP development mailing list <assp-test@...>


An
ASSP development mailing list <assp-test@...>
Kopie

Thema
Re: [Assp-test] Antwort: Whitelisted from addresses






Thanks for the quick reply and recognizing how critical this problem is
for us.

I've got NotGreedyWhitelist set to 1
check all addresses - one match for white - add all

Yet, I'm still getting plenty of bayesian flags from addresses that
are alrady whitelisted.

This was a problem before 0.5.04 (this is when notgreeywhitelist was
on or off).  I reported the problem and you fixed it in 0.5.04.  In a
version shortly thereafter, maybe even the 05/06 combo release, it
broke again.

Can you check what was changed in 04 and make sure that it's still
that way in .14?  If it is, it must be something else

We've got bayesian set to BLOCK, not score, which is probably why
we're seeing the errors in the block reports when score only people
might not.




On Wed, Nov 4, 2009 at 2:36 PM, Thomas Eckardt/eck
<Thomas.Eckardt@...> wrote:

> This feature works like expected.
>
> whitelisted state is (was) never reached if:
>
> - any address is redlisted
> - NotGreedyWhitelist == 1
> - NotGreedyWhitelist == 2 and MAIL FROM is not whitelisted
> - if the sender has a local mail address
>
> Thomas
>
>
>
>
> K Post <nntp.post@...>
> 04.11.2009 20:12
> Bitte antworten an
> ASSP development mailing list <assp-test@...>
>
>
> An
> ASSP development mailing list <assp-test@...>
> Kopie
>
> Thema
> [Assp-test] Whitelisted from addresses
>
>
>
>
>
>
> Last thread "more on NotGreedyWhitelist" didn't get an indication if
> the problem was going to be able to be fixed or if anyone else was
> seeing the problem.
>
> Here's the issue, with 2.0.1 rc 0.5.13 whitelisted from addresseses in
> the header that differ from non-whitelisted "mail from" addresses
> don't seem to get a message to be considered to be from a whitelisted.
>
> This was supposed to have been fixed with 0.5.04 and I think I saw
> that it was, but now it's happening again.  I don't know what version
> this started with again, but it's a real problem here.
>
> Any idea what could be wrong?  This goes way too deep for me to be
> able to help pinpoint a recommended change to the code.
>
> Thanks
>
>

------------------------------------------------------------------------------
> Let Crystal Reports handle the reporting - Free Crystal Reports 2008
> 30-Day
> trial. Simplify your report design, integration and deployment - and
focus

> on
> what you do best, core application coding. Discover what's new with
> Crystal Reports now.  http://p.sf.net/sfu/bobj-july
> _______________________________________________
> Assp-test mailing list
> Assp-test@...
> https://lists.sourceforge.net/lists/listinfo/assp-test
>
>
>
>
> DISCLAIMER:
> *******************************************************
> This email and any files transmitted with it may be confidential,

legally
> privileged and protected in law and are intended solely for the use of
the
>
> individual to whom it is addressed.
> This email was multiple times scanned for viruses. There should be no
> known virus in this email!
> *******************************************************
>
>
>
------------------------------------------------------------------------------
> Let Crystal Reports handle the reporting - Free Crystal Reports 2008
30-Day
> trial. Simplify your report design, integration and deployment - and
focus on
> what you do best, core application coding. Discover what's new with
> Crystal Reports now.  http://p.sf.net/sfu/bobj-july
> _______________________________________________
> Assp-test mailing list
> Assp-test@...
> https://lists.sourceforge.net/lists/listinfo/assp-test
>

------------------------------------------------------------------------------
Let Crystal Reports handle the reporting - Free Crystal Reports 2008
30-Day
trial. Simplify your report design, integration and deployment - and focus
on
what you do best, core application coding. Discover what's new with
Crystal Reports now.  http://p.sf.net/sfu/bobj-july
_______________________________________________
Assp-test mailing list
Assp-test@...
https://lists.sourceforge.net/lists/listinfo/assp-test



DISCLAIMER:
*******************************************************
This email and any files transmitted with it may be confidential, legally
privileged and protected in law and are intended solely for the use of the

individual to whom it is addressed.
This email was multiple times scanned for viruses. There should be no
known virus in this email!
*******************************************************


------------------------------------------------------------------------------
Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day
trial. Simplify your report design, integration and deployment - and focus on
what you do best, core application coding. Discover what's new with
Crystal Reports now.  http://p.sf.net/sfu/bobj-july
_______________________________________________
Assp-test mailing list
Assp-test@...
https://lists.sourceforge.net/lists/listinfo/assp-test

[K Post]

Sorry, I meant the first option (so 0, not 1)
check all addresses - one match for white - add all

If set to 'check all addresses - one match for white - add all', one
match in any of this fields is enough to get white and all addresses
will be added to whitelist.

So this means, provided that I'm understanding right, that if just one
of the from addresses matches, that the message will be whitelisted.
That's simply not what I'm seeing.

Are other people possibly scoring instead of blocking on bayesian
which is why they're not seeing the errors that I am.

Clearly, our corpus isn't the greatest, or we wouldn't get these false
positive bayesians, but having this notgreedy feature not working for
us isn't helping the corpus learn...

Could you kindly tell me what error you found with 0.5.04.  The
changelog at Fritz's site says:

ASSP 2.0.1 RC 0.5.04
fixed:
- whitelisted from is not detected

and that's when it started working for me as advertised (short lived though).

I'm sorry to keep pestering on this, but the code that controls this
is way to complicated for me to follow.

Thanks



On Wed, Nov 4, 2009 at 3:21 PM, Thomas Eckardt/eck
<Thomas.Eckardt@...> wrote:

>>I've got NotGreedyWhitelist set to 1
>>check all addresses - one match for white - add all
>
> This is not possible.
>
> 'check all addresses - one match for white - add all'   sets   '
> NotGreedyWhitelist'  to 0
>
> I'll not change anything in sub onwhitelist, as long as nobody else is
> reporting a problem there.
>
> Thomas
>
>
>
>
> K Post <nntp.post@...>
> 04.11.2009 21:08
> Bitte antworten an
> ASSP development mailing list <assp-test@...>
>
>
> An
> ASSP development mailing list <assp-test@...>
> Kopie
>
> Thema
> Re: [Assp-test] Antwort: Whitelisted from addresses
>
>
>
>
>
>
> Thanks for the quick reply and recognizing how critical this problem is
> for us.
>
> I've got NotGreedyWhitelist set to 1
> check all addresses - one match for white - add all
>
> Yet, I'm still getting plenty of bayesian flags from addresses that
> are alrady whitelisted.
>
> This was a problem before 0.5.04 (this is when notgreeywhitelist was
> on or off).  I reported the problem and you fixed it in 0.5.04.  In a
> version shortly thereafter, maybe even the 05/06 combo release, it
> broke again.
>
> Can you check what was changed in 04 and make sure that it's still
> that way in .14?  If it is, it must be something else
>
> We've got bayesian set to BLOCK, not score, which is probably why
> we're seeing the errors in the block reports when score only people
> might not.
>
>
>
>
> On Wed, Nov 4, 2009 at 2:36 PM, Thomas Eckardt/eck
> <Thomas.Eckardt@...> wrote:
>> This feature works like expected.
>>
>> whitelisted state is (was) never reached if:
>>
>> - any address is redlisted
>> - NotGreedyWhitelist == 1
>> - NotGreedyWhitelist == 2 and MAIL FROM is not whitelisted
>> - if the sender has a local mail address
>>
>> Thomas
>>
>>
>>
>>
>> K Post <nntp.post@...>
>> 04.11.2009 20:12
>> Bitte antworten an
>> ASSP development mailing list <assp-test@...>
>>
>>
>> An
>> ASSP development mailing list <assp-test@...>
>> Kopie
>>
>> Thema
>> [Assp-test] Whitelisted from addresses
>>
>>
>>
>>
>>
>>
>> Last thread "more on NotGreedyWhitelist" didn't get an indication if
>> the problem was going to be able to be fixed or if anyone else was
>> seeing the problem.
>>
>> Here's the issue, with 2.0.1 rc 0.5.13 whitelisted from addresseses in
>> the header that differ from non-whitelisted "mail from" addresses
>> don't seem to get a message to be considered to be from a whitelisted.
>>
>> This was supposed to have been fixed with 0.5.04 and I think I saw
>> that it was, but now it's happening again.  I don't know what version
>> this started with again, but it's a real problem here.
>>
>> Any idea what could be wrong?  This goes way too deep for me to be
>> able to help pinpoint a recommended change to the code.
>>
>> Thanks
>>
>>
> ------------------------------------------------------------------------------
>> Let Crystal Reports handle the reporting - Free Crystal Reports 2008
>> 30-Day
>> trial. Simplify your report design, integration and deployment - and
> focus
>> on
>> what you do best, core application coding. Discover what's new with
>> Crystal Reports now.  http://p.sf.net/sfu/bobj-july
>> _______________________________________________
>> Assp-test mailing list
>> Assp-test@...
>> https://lists.sourceforge.net/lists/listinfo/assp-test
>>
>>
>>
>>
>> DISCLAIMER:
>> *******************************************************
>> This email and any files transmitted with it may be confidential,
> legally
>> privileged and protected in law and are intended solely for the use of
> the
>>
>> individual to whom it is addressed.
>> This email was multiple times scanned for viruses. There should be no
>> known virus in this email!
>> *******************************************************
>>
>>
>>
> ------------------------------------------------------------------------------
>> Let Crystal Reports handle the reporting - Free Crystal Reports 2008
> 30-Day
>> trial. Simplify your report design, integration and deployment - and
> focus on
>> what you do best, core application coding. Discover what's new with
>> Crystal Reports now.  http://p.sf.net/sfu/bobj-july
>> _______________________________________________
>> Assp-test mailing list
>> Assp-test@...
>> https://lists.sourceforge.net/lists/listinfo/assp-test
>>
>
> ------------------------------------------------------------------------------
> Let Crystal Reports handle the reporting - Free Crystal Reports 2008
> 30-Day
> trial. Simplify your report design, integration and deployment - and focus
> on
> what you do best, core application coding. Discover what's new with
> Crystal Reports now.  http://p.sf.net/sfu/bobj-july
> _______________________________________________
> Assp-test mailing list
> Assp-test@...
> https://lists.sourceforge.net/lists/listinfo/assp-test
>
>
>
> DISCLAIMER:
> *******************************************************
> This email and any files transmitted with it may be confidential, legally
> privileged and protected in law and are intended solely for the use of the
>
> individual to whom it is addressed.
> This email was multiple times scanned for viruses. There should be no
> known virus in this email!
> *******************************************************
>
>
> ------------------------------------------------------------------------------
> Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day
> trial. Simplify your report design, integration and deployment - and focus on
> what you do best, core application coding. Discover what's new with
> Crystal Reports now.  http://p.sf.net/sfu/bobj-july
> _______________________________________________
> Assp-test mailing list
> Assp-test@...
> https://lists.sourceforge.net/lists/listinfo/assp-test
>

------------------------------------------------------------------------------
Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day
trial. Simplify your report design, integration and deployment - and focus on
what you do best, core application coding. Discover what's new with
Crystal Reports now.  http://p.sf.net/sfu/bobj-july
_______________________________________________
Assp-test mailing list
Assp-test@...
https://lists.sourceforge.net/lists/listinfo/assp-test

[K Post]

I think I found the source of the problem in the code.

I've done file compares between every version of ASSP between 2.0.1 rc
0.5.03 and the current one.  While there was a change in 0.5.04 to fix
notgreedy, that change has remained.  I dunno, maybe .04 fixed a
different problem.

In any case, I would have expected to see something in the logs that
says "Whitelisted sender address:" for the mails with a whitelisted
sender in the header.  However, I only see this when the "mail from"
is a whitelisted address.

I dug into the code and it seems that in the onwhitelist sub that the
%senders array isn't getting populated with data.

I put in some debugging lines to see what I could find out.

 if(! $NotGreedyWhitelist or $NotGreedyWhitelist == 2) {
        mlog($fh,"debug about to get senders")   <<--put in this line


            while($header=~/\n(from|sender|reply-to|errors-to|list-\w+):.*?($EmailAdrRe\@$EmailDomainRe)/igo)
{
                my $s = $2;
                $s = $1 if ($s =~ /^prvs=\d\d\d\d\w{6}=(.*)/);
                $senders{lc $s}=1;
        mlog($fh,"debug sender in notgreedyloop: $s")   <<--put in this line

            }
        }

The "debug about to get senders" line does get written to the log.

However, the "debug sender in notgreedyloop" line never shows  It's as
if there's never a match in the regex so the loop never gets entered.

Subequent debugging lines that I put in, right after that loop do run.

I hope this is enough for you to take a quick look to see what you can see.

Thanks

------------------------------------------------------------------------------
Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day
trial. Simplify your report design, integration and deployment - and focus on
what you do best, core application coding. Discover what's new with
Crystal Reports now.  http://p.sf.net/sfu/bobj-july
_______________________________________________
Assp-test mailing list
Assp-test@...
https://lists.sourceforge.net/lists/listinfo/assp-test

[K Post]

Also in getFrom (used to show other froms in the block report) the regex is
/(from|sender|reply-to|errors-to|list-\w+:).*?($EmailAdrRe\@$EmailDomainRe)/i

I >>think<< the colon (:) is supposed to be outside of the parens.

------------------------------------------------------------------------------
Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day
trial. Simplify your report design, integration and deployment - and focus on
what you do best, core application coding. Discover what's new with
Crystal Reports now.  http://p.sf.net/sfu/bobj-july
_______________________________________________
Assp-test mailing list
Assp-test@...
https://lists.sourceforge.net/lists/listinfo/assp-test

[K Post]

I did some more testing.  It seems that $header is blank.  Any idea
how that's possible?

------------------------------------------------------------------------------
Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day
trial. Simplify your report design, integration and deployment - and focus on
what you do best, core application coding. Discover what's new with
Crystal Reports now.  http://p.sf.net/sfu/bobj-july
_______________________________________________
Assp-test mailing list
Assp-test@...
https://lists.sourceforge.net/lists/listinfo/assp-test

[Thomas Eckardt/eck]

found it -> the call to onwhitelist was wrong.

Thomas

DISCLAIMER:
*******************************************************
This email and any files transmitted with it may be confidential, legally
privileged and protected in law and are intended solely for the use of the

individual to whom it is addressed.
This email was multiple times scanned for viruses. There should be no
known virus in this email!
*******************************************************


------------------------------------------------------------------------------
Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day
trial. Simplify your report design, integration and deployment - and focus on
what you do best, core application coding. Discover what's new with
Crystal Reports now.  http://p.sf.net/sfu/bobj-july
_______________________________________________
Assp-test mailing list
Assp-test@...
https://lists.sourceforge.net/lists/listinfo/assp-test