ISPmail Lenny tutorial ready

[Christoph Haas-2]

Dear list...

My new ISPmail tutorial for Lenny is out. Yay!

For those who don’t know what I’m talking about: I’ve been maintaining a
tutorial for configuring a Debian-server as a full-fledged email server
(Postfix/Dovecot/Amavis/MySQL: POP3, IMAP, SMTP, spam fighting, virus
scanning, webmail, MySQL backend) since the days of Debian Woody that has a
growing fan base. Some of the new features are a simpler database schema,
server-side Sieve filters, an introduction to logical volume management,
quotas, SPF and DNS handling. And as usual it’s not just copy/paste style
but spiced up with lots of explanations on the basics that a sysadmin needs
to know to run an email server. There are also hints for readers of the
previous Etch tutorial on how to migrate. A few pages of additional
(non-essential) features will be delivered in addition soon (Backup MX with
MySQL replication, mailing lists, DynDNS, policyd etc.).

Comments welcome.

Cheers
 Christoph


--
To UNSUBSCRIBE, email to debian-isp-REQUEST@...
with a subject of "unsubscribe". Trouble? Contact listmaster@...

[Eden Caldas]

cool

where is it?

2009/7/17 Christoph Haas <haas@...>

Dear list...

My new ISPmail tutorial for Lenny is out. Yay!

For those who don’t know what I’m talking about: I’ve been maintaining a
tutorial for configuring a Debian-server as a full-fledged email server
(Postfix/Dovecot/Amavis/MySQL: POP3, IMAP, SMTP, spam fighting, virus
scanning, webmail, MySQL backend) since the days of Debian Woody that has a
growing fan base. Some of the new features are a simpler database schema,
server-side Sieve filters, an introduction to logical volume management,
quotas, SPF and DNS handling. And as usual it’s not just copy/paste style
but spiced up with lots of explanations on the basics that a sysadmin needs
to know to run an email server. There are also hints for readers of the
previous Etch tutorial on how to migrate. A few pages of additional
(non-essential) features will be delivered in addition soon (Backup MX with
MySQL replication, mailing lists, DynDNS, policyd etc.).

Comments welcome.

Cheers
 Christoph


--
To UNSUBSCRIBE, email to debian-isp-REQUEST@...
with a subject of "unsubscribe". Trouble? Contact listmaster@...

[Christoph Haas-2]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Original posting:
> 2009/7/17 Christoph Haas <haas@...>
>> My new ISPmail tutorial for Lenny is out. Yay!

Eden Caldas wrote:
> cool
>
> where is it?

Lol, I'm such a dork. :) Copy/paste from a blog announcement is pretty
useless if the links get killed. It's too hot today.

Try: http://workaround.org/ispmail/lenny

Cheers
 Christoph

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iEYEARECAAYFAkpgqpUACgkQCV53xXnMZYYypACgw6Ot/apNyU5V7Yc6yvSY1c6T
mB8AoLqHGyOP/by+TFi42ieG/KGuM/Y+
=9wdj
-----END PGP SIGNATURE-----


--
To UNSUBSCRIBE, email to debian-isp-REQUEST@...
with a subject of "unsubscribe". Trouble? Contact listmaster@...

[Michelle Konzack-2]

Hallo Christoph,

und wo ist der Link dahin?

I am writing an "Debian ISP Tutorial" using courier, clamav-ng, maildrop
procmail, spamassassin, postgresql, php5, bind9, freeradius, ...

I am ongoing ISP in Baden-Württemberg/Germany (see Who is using  Debian)
and using Equipment from Transmode, CISCO,  Iskratel  (FTTH  DSLAM)  and
Alvarion BreezeACCESS VL.

Is your Tutorial under GPL?

Thanks, Greetings and nice Day/Evening
    Michelle Konzack
    Systemadministrator
    Tamay Dogan Network
    Debian GNU/Linux Consultant


Am 2009-07-17 17:00:19, schrieb Christoph Haas:

> Dear list...
>
> My new ISPmail tutorial for Lenny is out. Yay!
>
> For those who don’t know what I’m talking about: I’ve been maintaining a
> tutorial for configuring a Debian-server as a full-fledged email server
> (Postfix/Dovecot/Amavis/MySQL: POP3, IMAP, SMTP, spam fighting, virus
> scanning, webmail, MySQL backend) since the days of Debian Woody that has a
> growing fan base. Some of the new features are a simpler database schema,
> server-side Sieve filters, an introduction to logical volume management,
> quotas, SPF and DNS handling. And as usual it’s not just copy/paste style
> but spiced up with lots of explanations on the basics that a sysadmin needs
> to know to run an email server. There are also hints for readers of the
> previous Etch tutorial on how to migrate. A few pages of additional
> (non-essential) features will be delivered in addition soon (Backup MX with
> MySQL replication, mailing lists, DynDNS, policyd etc.).
>
> Comments welcome.
>
> Cheers
>  Christoph
>
>
> --
> To UNSUBSCRIBE, email to debian-isp-REQUEST@...
> with a subject of "unsubscribe". Trouble? Contact listmaster@...
>

------------------------ END OF REPLIED MESSAGE ------------------------

--
Linux-User #280138 with the Linux Counter, http://counter.li.org/
##################### Debian GNU/Linux Consultant #####################
<http://www.tamay-dogan.net/>                 Michelle Konzack
<http://www.can4linux.org/>                   c/o Vertriebsp. KabelBW
<http://www.flexray4linux.org/>               Blumenstrasse 2
Jabber linux4michelle@...           77694 Kehl/Germany
IRC #Debian (irc.icq.com)                     Tel. DE: [phone]
ICQ #328449886                                Tel. FR: [phone]

[Henry-Nicolas Tourneur]

Le vendredi 17 juillet 2009 17:00:19, Christoph Haas a écrit :

> Dear list...
>
> My new ISPmail tutorial for Lenny is out. Yay!
>
> For those who don’t know what I’m talking about: I’ve been maintaining a
> tutorial for configuring a Debian-server as a full-fledged email server
> (Postfix/Dovecot/Amavis/MySQL: POP3, IMAP, SMTP, spam fighting, virus
> scanning, webmail, MySQL backend) since the days of Debian Woody that has a
> growing fan base. Some of the new features are a simpler database schema,
> server-side Sieve filters, an introduction to logical volume management,
> quotas, SPF and DNS handling. And as usual it’s not just copy/paste style
> but spiced up with lots of explanations on the basics that a sysadmin needs
> to know to run an email server. There are also hints for readers of the
> previous Etch tutorial on how to migrate. A few pages of additional
> (non-essential) features will be delivered in addition soon (Backup MX with
> MySQL replication, mailing lists, DynDNS, policyd etc.).
>
> Comments welcome.
>
> Cheers
>  Christoph

That looks to be a very very great tutorial, thanks for writing it !
Maybe could you speak about the software called whitelister [0].
It is a very good mixture of RBL and greylisting :)

Have a nice week-end,

Henry-Nicolas Tourneur.

[0] http://packages.debian.org/lenny/whitelister


--
To UNSUBSCRIBE, email to debian-isp-REQUEST@...
with a subject of "unsubscribe". Trouble? Contact listmaster@...

[Thomas Goirand]

Christoph Haas wrote:

> Dear list...
>
> My new ISPmail tutorial for Lenny is out. Yay!
>
> For those who don’t know what I’m talking about: I’ve been maintaining a
> tutorial for configuring a Debian-server as a full-fledged email server
> (Postfix/Dovecot/Amavis/MySQL: POP3, IMAP, SMTP, spam fighting, virus
> scanning, webmail, MySQL backend) since the days of Debian Woody that has a
> growing fan base. Some of the new features are a simpler database schema,
> server-side Sieve filters, an introduction to logical volume management,
> quotas, SPF and DNS handling. And as usual it’s not just copy/paste style
> but spiced up with lots of explanations on the basics that a sysadmin needs
> to know to run an email server. There are also hints for readers of the
> previous Etch tutorial on how to migrate. A few pages of additional
> (non-essential) features will be delivered in addition soon (Backup MX with
> MySQL replication, mailing lists, DynDNS, policyd etc.).
>
> Comments welcome.
>
> Cheers
>  Christoph

Hi,

I had a look at your tutorial, it's funny to see that it's very close to
what our control panel does. In fact, we do absolutely all of what is
described there, except that we don't use MySQL for postfix as we have
found that flat files are faster: the DB is just dumped into flat files.

The issue with having dbs for Postfix is that in many cases, something
can go wrong with your MySQL server. While this is fine for pop/imap,
it's not at all for mail delivery. Also, it's forcing you to have a
wrong db schema as it has to be what postfix expect.

On top of what you describe above, you panel does:
- tumgreyspf
- dkimproxy (filtering and scanning)
- some basic DSL stop rules (major source of spam)
- basic header and body checks
- delivery in a SPAM imap box (using maildrop)
- vacation messages
- mailbox quota
- SMTP and pop3 traffic accounting in real time (if using courier)
- MLMMJ lists management

And of course:
- root interface to add/remove domains
- virtual admin interface to add/remove email
- email panel so the users can change their mailbox parameters

Managing emails "by hand" on the shell is just not practical, IMHO.

Over the years, we have found that the biggest issue in this setup is
amavis. It's a dog: it takes an incredible amount of RAM and CPU for
what it does, and often crashes. Seems that the Lenny version is better
than the etch one (that was crashing for no reasons), but still, I don't
consider it a good product. Has anyone ever work with DSPAM? How is it
compared to amavis?

Also, does one of you know a way to get the traffic information out of
dovecot? We support Dovecot, but don't use it, just because of that.
It's a shame because Dovecot is a WAY faster than courier.

Last thing: I think it's a shame that, when you setup amavis, clamav,
spamassassin and postfix in Debian, they are not configured by default
to work together. There's a lot of scripting needed to do this setup,
and there's no way (because of the Debian policy) to do this in a Debian
package postinst script.

Thomas


--
To UNSUBSCRIBE, email to debian-isp-REQUEST@...
with a subject of "unsubscribe". Trouble? Contact listmaster@...

[Seth Mattinen]

Thomas Goirand wrote:
>
> Also, does one of you know a way to get the traffic information out of
> dovecot? We support Dovecot, but don't use it, just because of that.
> It's a shame because Dovecot is a WAY faster than courier.

Logs, but I don't use the ancient version Debian ships with. Examples
from my logs:

dovecot: POP3(xxx@...): Disconnected: Logged out top=0/0, retr=0/0,
del=0/60, size=28331635
dovecot: IMAP(@xxx.co.uk): Disconnected in IDLE bytes=294/526
IMAP(xxx@...): Disconnected for inactivity bytes=749/10824
dovecot: IMAP(xxx@...): Disconnected: Logged out bytes=285/20363

I just use a program looping on a fifo to read crap out of syslog and
inject it somewhere useful.

~Seth


--
To UNSUBSCRIBE, email to debian-isp-REQUEST@...
with a subject of "unsubscribe". Trouble? Contact listmaster@...

[Seth Mattinen]

Seth Mattinen wrote:

> Thomas Goirand wrote:
>>
>> Also, does one of you know a way to get the traffic information out of
>> dovecot? We support Dovecot, but don't use it, just because of that.
>> It's a shame because Dovecot is a WAY faster than courier.
>
> Logs, but I don't use the ancient version Debian ships with. Examples
> from my logs:
>
> dovecot: POP3(xxx@...): Disconnected: Logged out top=0/0, retr=0/0,
> del=0/60, size=28331635
> dovecot: IMAP(@xxx.co.uk): Disconnected in IDLE bytes=294/526
> IMAP(xxx@...): Disconnected for inactivity bytes=749/10824
> dovecot: IMAP(xxx@...): Disconnected: Logged out bytes=285/20363
>

And here's a POP3 connection where someone actually downloaded a message:

POP3(xxx@...): Disconnected: Logged out top=0/0, retr=1/1782735,
del=0/488, size=90072431

~Seth


--
To UNSUBSCRIBE, email to debian-isp-REQUEST@...
with a subject of "unsubscribe". Trouble? Contact listmaster@...

[Thomas Goirand]

Seth Mattinen wrote:

> Seth Mattinen wrote:
>> Thomas Goirand wrote:
>>>
>>> Also, does one of you know a way to get the traffic information out of
>>> dovecot? We support Dovecot, but don't use it, just because of that.
>>> It's a shame because Dovecot is a WAY faster than courier.
>>
>> Logs, but I don't use the ancient version Debian ships with. Examples
>> from my logs:
>>
>> dovecot: POP3(xxx@...): Disconnected: Logged out top=0/0,
>> retr=0/0, del=0/60, size=28331635
>> dovecot: IMAP(@xxx.co.uk): Disconnected in IDLE bytes=294/526
>> IMAP(xxx@...): Disconnected for inactivity bytes=749/10824
>> dovecot: IMAP(xxx@...): Disconnected: Logged out bytes=285/20363
>>
>
> And here's a POP3 connection where someone actually downloaded a message:
>
> POP3(xxx@...): Disconnected: Logged out top=0/0, retr=1/1782735,
> del=0/488, size=90072431
>
> ~Seth

Do you confirm that the current Debian package (Lenny, SID?) does NOT
include this kind of logging?

Thomas

P.S: Once again, do NOT add me as Cc:, I'm reading the list, and as a
rule, you should not anyway. Thanks.


--
To UNSUBSCRIBE, email to debian-isp-REQUEST@...
with a subject of "unsubscribe". Trouble? Contact listmaster@...

[Seth Mattinen]

Thomas Goirand wrote:

> Seth Mattinen wrote:
>> Seth Mattinen wrote:
>>> Thomas Goirand wrote:
>>>> Also, does one of you know a way to get the traffic information out of
>>>> dovecot? We support Dovecot, but don't use it, just because of that.
>>>> It's a shame because Dovecot is a WAY faster than courier.
>>> Logs, but I don't use the ancient version Debian ships with. Examples
>>> from my logs:
>>>
>>> dovecot: POP3(xxx@...): Disconnected: Logged out top=0/0,
>>> retr=0/0, del=0/60, size=28331635
>>> dovecot: IMAP(@xxx.co.uk): Disconnected in IDLE bytes=294/526
>>> IMAP(xxx@...): Disconnected for inactivity bytes=749/10824
>>> dovecot: IMAP(xxx@...): Disconnected: Logged out bytes=285/20363
>>>
>> And here's a POP3 connection where someone actually downloaded a message:
>>
>> POP3(xxx@...): Disconnected: Logged out top=0/0, retr=1/1782735,
>> del=0/488, size=90072431
>>
>> ~Seth
>
> Do you confirm that the current Debian package (Lenny, SID?) does NOT
> include this kind of logging?

Not sure what you mean, but here's version 1.0.rc15-2etch4:

POP3(xxx): Disconnected: Logged out top=0/0, retr=0/0, del=0/0, size=0
IMAP(xxx): Disconnected: Logged out
IMAP(xxx): Disconnected for inactivity

imap_logout_format is not a valid setting for old versions. If you want
usage logging use a current version, wherever source you choose to get
it from. I rebuild it from unstable.

>
> P.S: Once again, do NOT add me as Cc:, I'm reading the list, and as a
> rule, you should not anyway. Thanks.
>

Yeah well I was drilling holes in concrete all day, I'm tired, I forgot,
and I was just trying to provide a useful answer to your question. I
don't need to be bitched at on list. Thanks.

~Seth


--
To UNSUBSCRIBE, email to debian-isp-REQUEST@...
with a subject of "unsubscribe". Trouble? Contact listmaster@...

[Bogdan-16]

Hi,

Thomas wrote:

> On top of what you describe above, our panel does:
> - tumgreyspf
> - dkimproxy (filtering and scanning)
> - some basic DSL stop rules (major source of spam)
> - basic header and body checks
> - delivery in a SPAM imap box (using maildrop)
> - vacation messages
> - mailbox quota
> - SMTP and pop3 traffic accounting in real time (if using courier)
> - MLMMJ lists management

As a half-year user of DTC, I can confirm every word of Thomas's email :)
It can't be easier than using his control panel to setup a complete hosting-like environment in less than an hour.
I'm thinking of adding Ukrainian translation to DTC, and then maybe starting a custom branch for my (yet single) server.

Bogdan

-- реклама -----------------------------------------------------------
Создай свой сайт бесплатно! www.hostpro.ua


--
To UNSUBSCRIBE, email to debian-isp-REQUEST@...
with a subject of "unsubscribe". Trouble? Contact listmaster@...

[Wojciech Ziniewicz]

2009/7/18 Bogdan <do.IT@...>

As a half-year user of DTC, I can confirm every word of Thomas's email :)
It can't be easier than using his control panel to setup a complete hosting-like environment in less than an hour.
I'm thinking of adding Ukrainian translation to DTC, and then maybe starting a custom branch for my (yet single) server.

What i love ISPmail for is a "tutoriality" of it.

I've learned some basics of creating enterprise linux email system after doing this howto. I'm big fan of Christoph though ;) as I find it very important (i recommended etch tutorial for many friends ,saying that they will understand more than just configuring postfix and spamassasin)

regards.

--
Wojciech Ziniewicz
http://rfc.sunsite.dk/rfc/rfc2324.html

[Christoph Haas]

Thomas Goirand schrieb:
> Christoph Haas wrote:
>> My new ISPmail tutorial for Lenny is out. Yay!
>
> I had a look at your tutorial, it's funny to see that it's very close to
> what our control panel does. In fact, we do absolutely all of what is
> described there, except that we don't use MySQL for postfix as we have
> found that flat files are faster: the DB is just dumped into flat files.

How many virtual users and aliases do you have? I'm not sure how well it
scales with more than a thousand users. I just have 150 accounts here on
a server that I used for testing. And I love the flexibility of putting
a row into a database table and instantly having the change online.
Somehow I don't like batch processes that create text/hash files unless
necessary.

> The issue with having dbs for Postfix is that in many cases, something
> can go wrong with your MySQL server. While this is fine for pop/imap,
> it's not at all for mail delivery.

If the MySQL service becomes unavailable for Postfix then the sending
server will get a 4xx temporary error. In a huge setup I'd probably set
up monitoring to get noticed of such outages quickly. And I'd use a load
balancer in front of two MySQL instances (master-slave replication).

> Also, it's forcing you to have a
> wrong db schema as it has to be what postfix expect.

I like it that the SQL queries are completely customizable. You aren't
forced to anything really.

> On top of what you describe above, you panel does:
> - tumgreyspf

A reader suggested to give postfix-policyd-spf-python a try instead.

> - dkimproxy (filtering and scanning)

I have that on my list. Since SPF hardly helps against spam here I can
as well try that. :)

> - some basic DSL stop rules (major source of spam)
> - basic header and body checks

I find those dangerous. Which do you use?

> - delivery in a SPAM imap box (using maildrop)

With Dovecot you can use a global sieve configuration file to drop spam
into a seperate folder. Maildrop has been a pain here. Seeing that it
has a wrapper that catches segfaults (which it throws frequently)
convinced me that I don't want to use it. And another problem are users
who juse use POP3 because they just get to see their inbox.

> - vacation messages

That's fortunately easy with Dovecot+Sieve. Do you use maildrop for that
purpose?

> - mailbox quota

Dovecot handles that. Although I don't like that the respective user
does not get informed about the limits and just suddenly doesn't get
mail any more.

> - SMTP and pop3 traffic accounting in real time (if using courier)
> - MLMMJ lists management

Can you compare that to mailman? I haven't used MLMMJ yet.

> And of course:
> - root interface to add/remove domains
> - virtual admin interface to add/remove email
> - email panel so the users can change their mailbox parameters
>
> Managing emails "by hand" on the shell is just not practical, IMHO.

True. I'm currently preparing something for the new database layout.

> Over the years, we have found that the biggest issue in this setup is
> amavis. It's a dog: it takes an incredible amount of RAM and CPU for
> what it does, and often crashes. Seems that the Lenny version is better
> than the etch one (that was crashing for no reasons), but still, I don't
> consider it a good product. Has anyone ever work with DSPAM? How is it
> compared to amavis?

I'm totally with you regarding Amavis. It's a resource hog, has cryptic
configuration and sometimes just doesn't work for no reason. dspam is
very badly documented and barely useable. I now have clamav-milter and
spamassassin-milter on my list of things to try.

> Also, does one of you know a way to get the traffic information out of
> dovecot? We support Dovecot, but don't use it, just because of that.
> It's a shame because Dovecot is a WAY faster than courier.

And much easier to configure. Not sure about the traffic information
part though. Do you intend to limit the number of bytes a certain user
transmits?

> Last thing: I think it's a shame that, when you setup amavis, clamav,
> spamassassin and postfix in Debian, they are not configured by default
> to work together. There's a lot of scripting needed to do this setup,
> and there's no way (because of the Debian policy) to do this in a Debian
> package postinst script.

That might indeed be a bit too much to ask for. Postfix does one thing -
and it does it well. You can combine it with many other pieces of
software and it's hard to tell what you want to do with it. Emmanuel
Revah made a step in that direction and is offering DISS
[http://diss.manurevah.com/].

Cheers
 Christoph

[Eden Caldas]

Christoph

Very nice stuff.

I have two questions.

1 - Why don't you use postfixadmin? http://postfixadmin.sourceforge.net/
You can adjust your tutorial to use postfixadmin table styles and you
can mannage quota, users, domains, alias from a web browser.

2 - You said postgrey can also check SPF. How do you make it check for
SPF? Since you're using it for greylisting it'd be cool.

Eden

2009/7/18 Christoph Haas <email@...>

>
> Thomas Goirand schrieb:
> > Christoph Haas wrote:
> >> My new ISPmail tutorial for Lenny is out. Yay!
> >
> > I had a look at your tutorial, it's funny to see that it's very close to
> > what our control panel does. In fact, we do absolutely all of what is
> > described there, except that we don't use MySQL for postfix as we have
> > found that flat files are faster: the DB is just dumped into flat files.
>
> How many virtual users and aliases do you have? I'm not sure how well it
> scales with more than a thousand users. I just have 150 accounts here on
> a server that I used for testing. And I love the flexibility of putting
> a row into a database table and instantly having the change online.
> Somehow I don't like batch processes that create text/hash files unless
> necessary.
>
> > The issue with having dbs for Postfix is that in many cases, something
> > can go wrong with your MySQL server. While this is fine for pop/imap,
> > it's not at all for mail delivery.
>
> If the MySQL service becomes unavailable for Postfix then the sending
> server will get a 4xx temporary error. In a huge setup I'd probably set
> up monitoring to get noticed of such outages quickly. And I'd use a load
> balancer in front of two MySQL instances (master-slave replication).
>
> > Also, it's forcing you to have a
> > wrong db schema as it has to be what postfix expect.
>
> I like it that the SQL queries are completely customizable. You aren't
> forced to anything really.
>
> > On top of what you describe above, you panel does:
> > - tumgreyspf
>
> A reader suggested to give postfix-policyd-spf-python a try instead.
>
> > - dkimproxy (filtering and scanning)
>
> I have that on my list. Since SPF hardly helps against spam here I can
> as well try that. :)
>
> > - some basic DSL stop rules (major source of spam)
> > - basic header and body checks
>
> I find those dangerous. Which do you use?
>
> > - delivery in a SPAM imap box (using maildrop)
>
> With Dovecot you can use a global sieve configuration file to drop spam
> into a seperate folder. Maildrop has been a pain here. Seeing that it
> has a wrapper that catches segfaults (which it throws frequently)
> convinced me that I don't want to use it. And another problem are users
> who juse use POP3 because they just get to see their inbox.
>
> > - vacation messages
>
> That's fortunately easy with Dovecot+Sieve. Do you use maildrop for that
> purpose?
>
> > - mailbox quota
>
> Dovecot handles that. Although I don't like that the respective user
> does not get informed about the limits and just suddenly doesn't get
> mail any more.
>
> > - SMTP and pop3 traffic accounting in real time (if using courier)
> > - MLMMJ lists management
>
> Can you compare that to mailman? I haven't used MLMMJ yet.
>
> > And of course:
> > - root interface to add/remove domains
> > - virtual admin interface to add/remove email
> > - email panel so the users can change their mailbox parameters
> >
> > Managing emails "by hand" on the shell is just not practical, IMHO.
>
> True. I'm currently preparing something for the new database layout.
>
> > Over the years, we have found that the biggest issue in this setup is
> > amavis. It's a dog: it takes an incredible amount of RAM and CPU for
> > what it does, and often crashes. Seems that the Lenny version is better
> > than the etch one (that was crashing for no reasons), but still, I don't
> > consider it a good product. Has anyone ever work with DSPAM? How is it
> > compared to amavis?
>
> I'm totally with you regarding Amavis. It's a resource hog, has cryptic
> configuration and sometimes just doesn't work for no reason. dspam is
> very badly documented and barely useable. I now have clamav-milter and
> spamassassin-milter on my list of things to try.
>
> > Also, does one of you know a way to get the traffic information out of
> > dovecot? We support Dovecot, but don't use it, just because of that.
> > It's a shame because Dovecot is a WAY faster than courier.
>
> And much easier to configure. Not sure about the traffic information
> part though. Do you intend to limit the number of bytes a certain user
> transmits?
>
> > Last thing: I think it's a shame that, when you setup amavis, clamav,
> > spamassassin and postfix in Debian, they are not configured by default
> > to work together. There's a lot of scripting needed to do this setup,
> > and there's no way (because of the Debian policy) to do this in a Debian
> > package postinst script.
>
> That might indeed be a bit too much to ask for. Postfix does one thing -
> and it does it well. You can combine it with many other pieces of
> software and it's hard to tell what you want to do with it. Emmanuel
> Revah made a step in that direction and is offering DISS
> [http://diss.manurevah.com/].
>
> Cheers
>  Christoph
>
>

--
To UNSUBSCRIBE, email to debian-isp-REQUEST@...
with a subject of "unsubscribe". Trouble? Contact listmaster@...

[Thomas Goirand]

Christoph Haas wrote:

> Thomas Goirand schrieb:
>> Christoph Haas wrote:
>>> My new ISPmail tutorial for Lenny is out. Yay!
>> I had a look at your tutorial, it's funny to see that it's very close to
>> what our control panel does. In fact, we do absolutely all of what is
>> described there, except that we don't use MySQL for postfix as we have
>> found that flat files are faster: the DB is just dumped into flat files.
>
> How many virtual users and aliases do you have? I'm not sure how well it
> scales with more than a thousand users. I just have 150 accounts here on
> a server that I used for testing. And I love the flexibility of putting
> a row into a database table and instantly having the change online.
> Somehow I don't like batch processes that create text/hash files unless
> necessary.

No pb with thousands of users...

> If the MySQL service becomes unavailable for Postfix then the sending
> server will get a 4xx temporary error. In a huge setup I'd probably set
> up monitoring to get noticed of such outages quickly. And I'd use a load
> balancer in front of two MySQL instances (master-slave replication).

MySQL replication only works with low loads. In fact, running MySQL over
the network has a HUGE performance penalty.

>> On top of what you describe above, you panel does:
>> - tumgreyspf
>
> A reader suggested to give postfix-policyd-spf-python a try instead.

tumgreyspf doesn't work very well, I'd like to replace it by something
better in fact.

>> - dkimproxy (filtering and scanning)
>
> I have that on my list. Since SPF hardly helps against spam here I can
> as well try that. :)

The issue with DKIM is not about filtering. It's MANDATORY to sign your
email to be able to send to some big providers (namely: yahoo).

>> - some basic DSL stop rules (major source of spam)
>> - basic header and body checks
>
> I find those dangerous. Which do you use?

I filter things like /^dsl.*\..*/, /ppp.*\..*/ and things like that,
known attachments like file.zip, details.zip and such, known X-Mailer
like SmartMailer, Avalanche, etc, subject with medecine names (I wont
write them here to avoid triggering spam filters...), and things like
that. The most efficient is the dsl/ppp/cable one. Nowadays, there is NO
WAY someone with such reverse DNS is sending legitimate emails without
using SASL auth, and it's very efficient.

>> - delivery in a SPAM imap box (using maildrop)
>
> With Dovecot you can use a global sieve configuration file to drop spam
> into a seperate folder. Maildrop has been a pain here. Seeing that it
> has a wrapper that catches segfaults (which it throws frequently)
> convinced me that I don't want to use it. And another problem are users
> who juse use POP3 because they just get to see their inbox.

I agree that maildrop is not so good, but I was stuck with it because of
the bandwidth usage thing. I'll be happy to use dovecot and sieve now.
As for the SPAM box not seen using POP3, well, that's the point of it!!!

>> - vacation messages
>
> That's fortunately easy with Dovecot+Sieve. Do you use maildrop for that
> purpose?

Also yes, as well as loop-detection and other things.

>> - mailbox quota
>
> Dovecot handles that. Although I don't like that the respective user
> does not get informed about the limits and just suddenly doesn't get
> mail any more.

Courier-maildrop sends a (configurable) message in the INBOX whenever
the mailbox reaches 90%, so it's fine.

>> - SMTP and pop3 traffic accounting in real time (if using courier)
>> - MLMMJ lists management
>
> Can you compare that to mailman? I haven't used MLMMJ yet.

Oh yes! MLMMJ is lightyears ahead of mailman, that is to me, something
of the past. MLMMJ is really GREAT, and the dev team doing it very
friendly as well.

>> Over the years, we have found that the biggest issue in this setup is
>> amavis. It's a dog: it takes an incredible amount of RAM and CPU for
>> what it does, and often crashes. Seems that the Lenny version is better
>> than the etch one (that was crashing for no reasons), but still, I don't
>> consider it a good product. Has anyone ever work with DSPAM? How is it
>> compared to amavis?
>
> I'm totally with you regarding Amavis. It's a resource hog, has cryptic
> configuration and sometimes just doesn't work for no reason. dspam is
> very badly documented and barely useable. I now have clamav-milter and
> spamassassin-milter on my list of things to try.

Just to remove amavis ??? Let me know if you do find a way to remove
Amavis and still have Clamav and Spamassassin running, I'm interested in
doing it as well.

>> Also, does one of you know a way to get the traffic information out of
>> dovecot? We support Dovecot, but don't use it, just because of that.
>> It's a shame because Dovecot is a WAY faster than courier.
>
> And much easier to configure. Not sure about the traffic information
> part though. Do you intend to limit the number of bytes a certain user
> transmits?

On our system, we do monitor FTP, POP, IMAP, SMTP and web traffic in
real time, so we can disable some accounts if they abuse.

>> Last thing: I think it's a shame that, when you setup amavis, clamav,
>> spamassassin and postfix in Debian, they are not configured by default
>> to work together. There's a lot of scripting needed to do this setup,
>> and there's no way (because of the Debian policy) to do this in a Debian
>> package postinst script.
>
> That might indeed be a bit too much to ask for. Postfix does one thing -
> and it does it well. You can combine it with many other pieces of
> software and it's hard to tell what you want to do with it. Emmanuel
> Revah made a step in that direction and is offering DISS
> [http://diss.manurevah.com/].

Well, I perfectly know how to script things, our panel does all of the
setup by itself, and there's nothing to be done by hand by the users.
It's just that it's not really Debian policy compliant, and it cannot be
sent in a postinst.

Now, it would have been great to have some kind of conf.d folder in
Postfix, but when I asked for it in the dev list of postfix, they swear
at me saying I was a useless c**t to ask for such foolish thing. Well,
not that much, but close! Meaning that there are 2 opposing force: the
Debian policy that forbids one to edit another configuration file, and
the authors refusing to make the software configuration/user friendly.
There's nothing we can do EXCEPT convince the maintainer to modify it's
package to "see" other tools.

It's a great pleasure to exchange opinions about all this, what you did,
etc. Thanks for posting in this list, I appreciate your view a lot!

Thomas


--
To UNSUBSCRIBE, email to debian-isp-REQUEST@...
with a subject of "unsubscribe". Trouble? Contact listmaster@...

[Thomas Goirand]

Eden Caldas wrote:
> 2 - You said postgrey can also check SPF. How do you make it check for
> SPF? Since you're using it for greylisting it'd be cool.
>
> Eden

Not to start a polemic and a huge thread (we all have better things to
do, right?), but SPF is a broken concept that doesn't work with
forwarders. DKIM does work always and is much much better.

The only thing we do is:

if ( SPF_check_failed_or_none )
        use_grey_listing

I think this is the only way to make it acceptable.

Thomas


--
To UNSUBSCRIBE, email to debian-isp-REQUEST@...
with a subject of "unsubscribe". Trouble? Contact listmaster@...

[Christoph Haas]

Eden Caldas wrote:
> Very nice stuff.

Thanks.

> I have two questions.
>=20
> 1 - Why don't you use postfixadmin? http://postfixadmin.sourceforge.net=
/
> You can adjust your tutorial to use postfixadmin table styles and you
> can mannage quota, users, domains, alias from a web browser.

Postfixadmin has been changed to not allow customized SQL queries. Which
means I had to use its exact database layout. And honest I didn't bother
checking out Postfixadmin (again) until everything was finished,
documented, tested and screenshot. I already listed to the people who
asked for a simpler database layout and I followed it and rewrote
several parts of the tutorial. If I had considered Postfixadmin earlier
I might have thought differently.

> 2 - You said postgrey can also check SPF. How do you make it check for
> SPF? Since you're using it for greylisting it'd be cool.

The tumgreyspf policy daemon does that.
[http://workaround.org/ispmail/lenny/dns]

But if I claimed that postgrey does that then I probably made a mistake.
The page on postgrey greylisting should have been removed anyway. I'll
fix that.

Btw I already learned about DKIM and postfix-policyd-spf-python which
are told to be more efficient. I haven't had much luck with SPF anyway
so I think it's a good idea to try that.

Cheers
 Christoph

[Seth Mattinen]

Thomas Goirand wrote:

> Christoph Haas wrote:
>> Thomas Goirand schrieb:
>>> Christoph Haas wrote:
>>>> My new ISPmail tutorial for Lenny is out. Yay!
>>> I had a look at your tutorial, it's funny to see that it's very close to
>>> what our control panel does. In fact, we do absolutely all of what is
>>> described there, except that we don't use MySQL for postfix as we have
>>> found that flat files are faster: the DB is just dumped into flat files.
>> How many virtual users and aliases do you have? I'm not sure how well it
>> scales with more than a thousand users. I just have 150 accounts here on
>> a server that I used for testing. And I love the flexibility of putting
>> a row into a database table and instantly having the change online.
>> Somehow I don't like batch processes that create text/hash files unless
>> necessary.
>
> No pb with thousands of users...
>
>> If the MySQL service becomes unavailable for Postfix then the sending
>> server will get a 4xx temporary error. In a huge setup I'd probably set
>> up monitoring to get noticed of such outages quickly. And I'd use a load
>> balancer in front of two MySQL instances (master-slave replication).
>
> MySQL replication only works with low loads. In fact, running MySQL over
> the network has a HUGE performance penalty.
>

I have a multi-slave setup that lives behind a load balancer that
performs without issues. The master is never touched directly; updates
(i.e. sharing greylisting knowledge) are written to the pool and
redistributed. I'd say as long as you keep your SQL query traffic under
100Mb/sec aggregate you should be fine. I don't know about beyond that
because my load balancers only have 10/100 Ethernet on them.

~Seth


--
To UNSUBSCRIBE, email to debian-isp-REQUEST@...
with a subject of "unsubscribe". Trouble? Contact listmaster@...

[Gregor Hermens-4]

Hi,

Am Samstag, 18. Juli 2009 schrieb Thomas Goirand:
> >> - mailbox quota
> >
> > Dovecot handles that. Although I don't like that the respective user
> > does not get informed about the limits and just suddenly doesn't get
> > mail any more.
>
> Courier-maildrop sends a (configurable) message in the INBOX whenever
> the mailbox reaches 90%, so it's fine.

starting from version 1.1, Dovecot can do that, too:

http://wiki.dovecot.org/Quota/1.1#Quota_warnings

On Lenny you're stuck with 1.0 though...

Gregor
--
     @mazing           fon [phone]
  Gregor Hermens       fax [phone]
Brucker Strasse 12  gregor.hermens@...
D-82216 Gernlinden    http://www.a-mazing.de/


--
To UNSUBSCRIBE, email to debian-isp-REQUEST@...
with a subject of "unsubscribe". Trouble? Contact listmaster@...

[Michelle Konzack-2]

Am 2009-07-19 04:14:33, schrieb Thomas Goirand:
> Christoph Haas wrote:
> > If the MySQL service becomes unavailable for Postfix then the sending
> > server will get a 4xx temporary error. In a huge setup I'd probably set
> > up monitoring to get noticed of such outages quickly. And I'd use a load
> > balancer in front of two MySQL instances (master-slave replication).
>
> MySQL replication only works with low loads. In fact, running MySQL over
> the network has a HUGE performance penalty.

This is, WHY I use PostgreSQL...  It performance miuch better.

Currently I am running a Test-Installation with Courier-Imap-Proxy,  two
Courier-Imap-Storage-Server and two PostgreSQL Database and a Webserver.

I am using 6 "Sun Fire X4100M2"  with  smallest  Quad-Core  Opteron  and
2x2 GByte of memory.

I have 180 Regular-Users on the  System  and  for  testing  arround  400
foreigners.  It handel currently my 4 own and 3 customer domains.

The next step is to install one or  two  inbound  MTA's  which  do  user
customised virus/spam filtering using courierlocalfilter.  Unfortunately
I have not seen a postfix/exim setup which  is  able  to  do  this  with
seperated Inbound-MTA, IMAP-Proxy and IMAP-Storage.

This is, what bigger ISP are using but it seems, each one  has  to  hack
its own stuff (Freenet, 1&1/GMX, Arcor, T-Online, KabelBW, ...)

> I filter things like /^dsl.*\..*/, /ppp.*\..*/ and things like that,
> known attachments like file.zip, details.zip and such, known X-Mailer
> like SmartMailer, Avalanche, etc, subject with medecine names

Here you would produce per day more  then  2000  False-Positives  in  my
network.

Spamfiltering MUST be individuual and if you give th user options,  like
at <freenet.de>, the user can CHOOSE which he/she want. Global filtering
is VERY dangerous.

And of course, you can not use  /^dsl.*\..*/, /ppp.*\..*/  if  you  have
hosted Domains from your customers, because most customers ARE  on  such
lines.

> Courier-maildrop sends a (configurable) message in the INBOX whenever
> the mailbox reaches 90%, so it's fine.

Q:  I am using the courier-suite too,
    but are you able to use Per-User-Quota?

> > Can you compare that to mailman? I haven't used MLMMJ yet.
>
> Oh yes! MLMMJ is lightyears ahead of mailman, that is to me, something
> of the past. MLMMJ is really GREAT, and the dev team doing it very
> friendly as well.

This is NO explanation...  What is better?

I use mailman and courier-mlm and both are working perfectly as I expect

> On our system, we do monitor FTP, POP, IMAP, SMTP and web traffic in
> real time, so we can disable some accounts if they abuse.

Which tools do you use?

> Well, I perfectly know how to script things, our panel does all of the
> setup by itself, and there's nothing to be done by hand by the users.
> It's just that it's not really Debian policy compliant, and it cannot be
> sent in a postinst.

Hmmm, there AFAIK several packages in Debian which modify the behabviour
of other packages in Debian, but there is a  "debconf"  question,  which
ASK  the  administrator,  whether  the  package  should  take  over  the
configuration which the administrator should answer [Ny].

Thanks, Greetings and nice Day/Evening
    Michelle Konzack
    Systemadministrator
    Tamay Dogan Network
    Debian GNU/Linux Consultant


--
Linux-User #280138 with the Linux Counter, http://counter.li.org/
##################### Debian GNU/Linux Consultant #####################
Michelle Konzack   c/o Shared Office KabelBW  ICQ #328449886
+49/177/9351947    Blumenstasse 2             MSN LinuxMichi
+33/6/61925193     77694 Kehl/Germany         IRC #Debian (irc.icq.com)