abuse report management system

[Wojciech Ziniewicz]

Hi,
I am now working for a small telco in Poland (dozens of /24 ,/23 and
/22 prefixes and one /16 class).

Several months ago we  began having problems with our customers and
abuse reports generated on the basis of their spam sending practices..

We are receiving tons of abuse reports . Customers were always
notified about the issue and if problem was not resolved , they were
cutoff.

Problem began when amount of abuse reports from other networks rose up
to several reports per day.

Does anybody know any free or opensource system that could help us
dealing with those abuse reports (trapping abuse@... mails and
sending notification to clients)??


regards.

--
Wojciech Ziniewicz
http://rfc.sunsite.dk/rfc/rfc2324.html


--
To UNSUBSCRIBE, email to debian-isp-REQUEST@...
with a subject of "unsubscribe". Trouble? Contact listmaster@...

[Andy Davidson-2]

On 11 Aug 2009, at 13:44, Wojciech Ziniewicz wrote:

> Does anybody know any free or opensource system that could help us
> dealing with those abuse reports (trapping abuse@... mails and
> sending notification to clients)??

This is dangerous, you wouldn't want your customers to see the  
original abuse request, and you wouldn't like outsiders to be able to  
cut off your customers by filing invented abuse emails that trigger  
the threshold in your automated system.  Serious allegations need to  
be checked with the use of, for example, netflow records, and you may  
require to build a different process based on messages from law  
enforcement agencies or similar.

Abuse handling is a specialist skill and should be done by humans.






--
Regards, Andy Davidson            [phone]           www.netsumo.com
NetSumo Ltd,   Specialist networks consultancy for ISPs,    Whitelabel  
24/7 NOC
/* Opinions are my own and & may not constitute policy of any org I  
work for */





--
To UNSUBSCRIBE, email to debian-isp-REQUEST@...
with a subject of "unsubscribe". Trouble? Contact listmaster@...

[Wojciech Ziniewicz]

2009/8/11 Andy Davidson <andy@...>:
> This is dangerous, you wouldn't want your customers to see the original
> abuse request, and you wouldn't like outsiders to be able to cut off your
> customers by filing invented abuse emails that trigger the threshold in your
> automated system.  Serious allegations need to be checked with the use of,
> for example, netflow records, and you may require to build a different
> process based on messages from law enforcement agencies or similar.
>
> Abuse handling is a specialist skill and should be done by humans.

I dont want any of theese.
What i want is a system that will trap abuse emails and notify me when
certain client reaches the trigger of let's say 10 abuse emails per
month. Then i would analyze the records and decide about warning him
about the sittuation and finally possibly cutting him of. We often do
such things to our bussiness customers - if they send tons of spam
using our ip addrs we can cut them off  after 2 warnings (in Poland
sending spam is a crime) .

regards.
WZ



--
Wojciech Ziniewicz
http://rfc.sunsite.dk/rfc/rfc2324.html


--
To UNSUBSCRIBE, email to debian-isp-REQUEST@...
with a subject of "unsubscribe". Trouble? Contact listmaster@...

[Aaron Thoreson]

procmail, arffilter (http://wordtothewise.com/products/arffilter.html),
perl, mysql

We use procmail to filter the inbound messages through 'arffilter' (to
read ARF - formatted attachments/messages) which then goes on to a perl
script that fetches certain header information.  Then we process it
daily and watch for spikes, new sources, etc.  I agree that a human need
examine the content and make final decisions, as the subject lines and
formatting of various organizations' abuse complaints vary wildly.
You'll be constantly amending your procmail rules and modifying your
perl scripts.

The only affordable system I've seen out there is Word to the Wise's
Abacus system.  I'm sure Remedy or something similar has a product, but
as far as open source, I'm not sure.

Aaron

Wojciech Ziniewicz wrote:

> 2009/8/11 Andy Davidson <andy@...>:
>> This is dangerous, you wouldn't want your customers to see the original
>> abuse request, and you wouldn't like outsiders to be able to cut off your
>> customers by filing invented abuse emails that trigger the threshold in your
>> automated system.  Serious allegations need to be checked with the use of,
>> for example, netflow records, and you may require to build a different
>> process based on messages from law enforcement agencies or similar.
>>
>> Abuse handling is a specialist skill and should be done by humans.
>
> I dont want any of theese.
> What i want is a system that will trap abuse emails and notify me when
> certain client reaches the trigger of let's say 10 abuse emails per
> month. Then i would analyze the records and decide about warning him
> about the sittuation and finally possibly cutting him of. We often do
> such things to our bussiness customers - if they send tons of spam
> using our ip addrs we can cut them off  after 2 warnings (in Poland
> sending spam is a crime) .
>
> regards.
> WZ
>
>
>

--
Aaron Thoreson
Network Group
Midcontinent Communications
aaront@...


--
To UNSUBSCRIBE, email to debian-isp-REQUEST@...
with a subject of "unsubscribe". Trouble? Contact listmaster@...

[Michael Katz-2]

Aaron Thoreson wrote:

> procmail, arffilter (http://wordtothewise.com/products/arffilter.html),
> perl, mysql
>
> We use procmail to filter the inbound messages through 'arffilter' (to
> read ARF - formatted attachments/messages) which then goes on to a perl
> script that fetches certain header information.  Then we process it
> daily and watch for spikes, new sources, etc.  I agree that a human need
> examine the content and make final decisions, as the subject lines and
> formatting of various organizations' abuse complaints vary wildly.
> You'll be constantly amending your procmail rules and modifying your
> perl scripts.
>
> The only affordable system I've seen out there is Word to the Wise's
> Abacus system.  I'm sure Remedy or something similar has a product, but
> as far as open source, I'm not sure.

Please don't kill me for this but we offer this capability with MPP.  It
is not open source but affordable.  We can do all kinds of custom
processing based on email content.

M Katz
http://mailspect.com

>
> Aaron
>
> Wojciech Ziniewicz wrote:
>> 2009/8/11 Andy Davidson <andy@...>:
>>> This is dangerous, you wouldn't want your customers to see the original
>>> abuse request, and you wouldn't like outsiders to be able to cut off
>>> your
>>> customers by filing invented abuse emails that trigger the threshold
>>> in your
>>> automated system.  Serious allegations need to be checked with the
>>> use of,
>>> for example, netflow records, and you may require to build a different
>>> process based on messages from law enforcement agencies or similar.
>>>
>>> Abuse handling is a specialist skill and should be done by humans.
>>
>> I dont want any of theese.
>> What i want is a system that will trap abuse emails and notify me when
>> certain client reaches the trigger of let's say 10 abuse emails per
>> month. Then i would analyze the records and decide about warning him
>> about the sittuation and finally possibly cutting him of. We often do
>> such things to our bussiness customers - if they send tons of spam
>> using our ip addrs we can cut them off  after 2 warnings (in Poland
>> sending spam is a crime) .
>>
>> regards.
>> WZ
>>
>>
>>
>

--
To UNSUBSCRIBE, email to debian-isp-REQUEST@...
with a subject of "unsubscribe". Trouble? Contact listmaster@...

[Michelle Konzack-2]

Hello Wojciech,

Am 2009-08-11 15:13:21, schrieb Wojciech Ziniewicz:
> What i want is a system that will trap abuse emails and notify me when
> certain client reaches the trigger of let's say 10 abuse emails per
> month.

I do this already using a procmail script, which spider the Body  if  an
E-Mail is attached and try to get the Received: header and analyse it.

Because the sending IP is associated (timely) with a  customer,  I  know
perfectly which customer was on which dynamic IP and ...

So you could do the same...

if I encounter an unknown Mail-Agent: or nothing, I trigger  immediately
because maybe it is a trojan or something like this.

For the new ISP i am in creation, I am blocking the port 25 and 589  and
force standard users to pass over my SMTP-Proxy.  However,  users  which
there own proper SMTP-Relays can reconfigure the two ports.

> Then i would analyze the records and decide about warning him
> about the sittuation and finally possibly cutting him of. We often do
> such things to our bussiness customers - if they send tons of spam
> using our ip addrs we can cut them off  after 2 warnings (in Poland
> sending spam is a crime) .

So you have the right to shootem?  ;-)

Thanks, Greetings and nice Day/Evening
    Michelle Konzack
    Systemadministrator
    Tamay Dogan Network
    Debian GNU/Linux Consultant

--
Linux-User #280138 with the Linux Counter, http://counter.li.org/
##################### Debian GNU/Linux Consultant #####################
<http://www.tamay-dogan.net/>                 Michelle Konzack
<http://www.can4linux.org/>                   Apt. 917
<http://www.flexray4linux.org/>               50, rue de Soultz
Jabber linux4michelle@...           67100 Strabourg/France
IRC    #Debian (irc.icq.com)                  Tel. DE: [phone]
ICQ    #328449886                             Tel. FR: [phone]

[Christopher Davis-4]

Greetings all,

> -----Original Message-----
> From: Aaron Thoreson [mailto:aaront@...]
> Sent: Tuesday, August 11, 2009 9:19 AM
> To: debian-isp@...
> Subject: Re: abuse report management system
>
> procmail, arffilter
> (http://wordtothewise.com/products/arffilter.html),
> perl, mysql

Also add RTIR - http://bestpractical.com/rtir/

> Wojciech Ziniewicz wrote:
> > 2009/8/11 Andy Davidson <andy@...>:
> >> This is dangerous, you wouldn't want your customers
> to see the original
> >> abuse request, and you wouldn't like outsiders to be
> able to cut off your
> >> customers by filing invented abuse emails that
> trigger the threshold in your
> >> automated system.  Serious allegations need to be
> checked with the use of,
> >> for example, netflow records, and you may require to
> build a different
> >> process based on messages from law enforcement
> agencies or similar.
> >>
> >> Abuse handling is a specialist skill and should be
> done by humans.
> >

Agreed - this is the responsibility of a proven responsible, capable person as operations are routinely in the grey area not defined while ultimately managing ~<20% of an organizations client portfolio.

The complainant should be acknowledged with a comforting note your organization received the complaint and would receive feedback if additional information is required - I would highly advise _not_ including a copy of the original complaint for security/ethical reasons.  

The individuals addressing these issues need to conceal the complainant's and client's information between all correspondences (everyone gets messy when a food fight breaks out).  Under any approach, being positively infectious with all communications is a must (who is not a customer today might be tomorrow and you wouldn't want to erase your current customers on a whim).  

> > I dont want any of theese.
> > What i want is a system that will trap abuse emails
> and notify me when
> > certain client reaches the trigger of let's say 10
> abuse emails per
> > month. Then i would analyze the records and decide
> about warning him
> > about the sittuation and finally possibly cutting him
> of. We often do
> > such things to our bussiness customers - if they send
> tons of spam
> > using our ip addrs we can cut them off  after 2
> warnings (in Poland
> > sending spam is a crime) .

You will need a reporting system capable of tracking your customers over a vast period of time while keeping all of your records available.  Information is like gold.  Every complaint should be associated to a client (telecom services are a penny in the bank to spam operations) and the client's service if multiple services are rendered (they may resell and obtained a questionable client themselves).  The reporting system is going to be a custom fit for the purpose, individuals utilizing, and organizational demands -- include charts and other aids to assist those who would not otherwise fully understand.

Bests,

Christopher Davis




--
To UNSUBSCRIBE, email to debian-isp-REQUEST@...
with a subject of "unsubscribe". Trouble? Contact listmaster@...

[Henrique de Moraes Holschuh]

On Tue, 11 Aug 2009, Wojciech Ziniewicz wrote:
> Does anybody know any free or opensource system that could help us
> dealing with those abuse reports (trapping abuse@... mails and
> sending notification to clients)??

Yes, OTRS can do it.  Request-tracker can also do it.  They can do a lot
more, though (they're full ticket-based request tracking systems).

All of them will require some manual intervention.  OTRS helps you send
template-based replies, which could help speed up the request processing.  I
don't know if RT (request tracker) can do that.

--
  "One disk to rule them all, One disk to find them. One disk to bring
  them all and in the darkness grind them. In the Land of Redmond
  where the shadows lie." -- The Silicon Valley Tarot
  Henrique Holschuh


--
To UNSUBSCRIBE, email to debian-isp-REQUEST@...
with a subject of "unsubscribe". Trouble? Contact listmaster@...

[Dusty Wilson-2]

On Mon, Sep 7, 2009 at 6:37 PM, Henrique de Moraes
Holschuh<hmh@...> wrote:

> On Tue, 11 Aug 2009, Wojciech Ziniewicz wrote:
>> Does anybody know any free or opensource system that could help us
>> dealing with those abuse reports (trapping abuse@... mails and
>> sending notification to clients)??
>
> Yes, OTRS can do it.  Request-tracker can also do it.  They can do a lot
> more, though (they're full ticket-based request tracking systems).
>
> All of them will require some manual intervention.  OTRS helps you send
> template-based replies, which could help speed up the request processing.  I
> don't know if RT (request tracker) can do that.

We use OTRS and have been using it for years.  I was interested in
migrating to RT since it has a nicer interface.  After using RT in
testing for a while, I decided that I really liked OTRS much better.
It might not be as pretty, but it's definitely got a huge number of
features and feels better-written than RT.

Just as an example: in RT you start with a ticket ID (usually #1) and
it goes up from there.  We use OTRS on several servers and need the
ticket IDs to be unique across them.  It supports a "server ID"
embedded in the ticket ID to deal with this.  Also, if you want the
ticket IDs to be randomized or time-based, you can do that easily.  In
RT, you have to hack the code to make it work.

We do all kinds of hackish things with OTRS that were never intended
to be done.  Plus it's fully integrated in our billing and accounting
systems, so we don't have to do dual-entry.  It does a great job of
keeping track of time and people.  And if you really don't like the
look of it, you can just make your own interface.  Oh, and it's
integrated in our Asterisk-based phone system as well.  A customer can
call in with their customer and ticket numbers and get status reports
and get forwarded to the person in charge of the ticket.  It's really
a beautiful thing.

The only problem I have currently is the newest version of OTRS with
HTML email support makes it difficult to reply inline.  We just have
the HTML email support disabled for outgoing emails for now.  (And
probably should leave it off anyway...)

RT is really nice, but I choose to stick with OTRS.  I really do recommend OTRS.

(don't need to CC me, I'm on the list)


--
To UNSUBSCRIBE, email to debian-isp-REQUEST@...
with a subject of "unsubscribe". Trouble? Contact listmaster@...