Questions about initial DBmail setup

[Josh Berkus]

DBMail folks,

I'm setting up DB mail for my personal server (this account).  I've a
number of questions; forgive me that I don't know much at all about
adminning mailservers.  I know a lot about databases, though.

1) How unstable is 2.3.3?  I'd like to try out some of the new features.

2) If I run queries against the mail in the backend database, and update or
delete things, are there parts of dbmail which don't get their data from
the database?

3) What do people use for spam filtering with dbmail?

4) Does anyone use egroupware with dbmail & PostgreSQL?  What do you think
of egroupware?

5) If I added full text search to the backend database, any thoughts on how
I could expose this to the mail client?

--
--Josh

Josh Berkus
PostgreSQL @ Sun
San Francisco
_______________________________________________
DBmail mailing list
DBmail@...
https://mailman.fastxs.nl/mailman/listinfo/dbmail

[Jorge Bastos]

>From my experience if you want to hear about:

>
> 1) How unstable is 2.3.3?  I'd like to try out some of the new
> features.

I'm only using it on testing and seams OK, with some problem I think with
IDLE.


> 2) If I run queries against the mail in the backend database, and
> update or
> delete things, are there parts of dbmail which don't get their data
> from
> the database?

Don't know how to answer :P

>
> 3) What do people use for spam filtering with dbmail?

I use Spamassassin and Clamsmtp with ClamAV.

>
> 4) Does anyone use egroupware with dbmail & PostgreSQL?  What do you
> think
> of egroupware?

Don't use Egroupware

>
> 5) If I added full text search to the backend database, any thoughts on
> how
> I could expose this to the mail client?

No ideia :P


Jorge

_______________________________________________
DBmail mailing list
DBmail@...
https://mailman.fastxs.nl/mailman/listinfo/dbmail

[Paul J Stevens]

Josh Berkus wrote:
> DBMail folks,
>
> I'm setting up DB mail for my personal server (this account).  I've a
> number of questions; forgive me that I don't know much at all about
> adminning mailservers.  I know a lot about databases, though.
>
> 1) How unstable is 2.3.3?  I'd like to try out some of the new features.

Don't use it for anything other than testing. There are still some
pretty fundamental issues to iron out.

>
> 2) If I run queries against the mail in the backend database, and update or
> delete things, are there parts of dbmail which don't get their data from
> the database?

No. Everything is in the database. Unless you are using LDAP for
authentication. That is the *only* exception.


--
  ________________________________________________________________
  Paul Stevens                                      paul at nfg.nl
  NET FACILITIES GROUP                     GPG/PGP: 1024D/11F8CD31
  The Netherlands________________________________http://www.nfg.nl
_______________________________________________
DBmail mailing list
DBmail@...
https://mailman.fastxs.nl/mailman/listinfo/dbmail

[Josh Berkus]

Paul,

> Don't use it for anything other than testing. There are still some
> pretty fundamental issues to iron out.

Thanks for the warning.  2.2 it is.  Now, if only Ubuntu would update
the $&%&@ packages ...

>
>> 2) If I run queries against the mail in the backend database, and update or
>> delete things, are there parts of dbmail which don't get their data from
>> the database?
>
> No. Everything is in the database. Unless you are using LDAP for
> authentication. That is the *only* exception.

Oh?  Is there a non-LDAP option?  I'd love to avoid LDAP -- I thought it
was required.

--Josh

_______________________________________________
DBmail mailing list
DBmail@...
https://mailman.fastxs.nl/mailman/listinfo/dbmail

[Aaron Stone]

On Jun 25, 2008, at 10:16 AM, Josh Berkus wrote:

> Paul,
>
>> Don't use it for anything other than testing. There are still some
>> pretty fundamental issues to iron out.
>
> Thanks for the warning.  2.2 it is.  Now, if only Ubuntu would  
> update the $&%&@ packages ...
>
>>> 2) If I run queries against the mail in the backend database, and  
>>> update or delete things, are there parts of dbmail which don't get  
>>> their data from the database?
>> No. Everything is in the database. Unless you are using LDAP for
>> authentication. That is the *only* exception.
>
> Oh?  Is there a non-LDAP option?  I'd love to avoid LDAP -- I  
> thought it was required.

The non-LDAP option is the default: everything is in the database :-)

Aaron
_______________________________________________
DBmail mailing list
DBmail@...
https://mailman.fastxs.nl/mailman/listinfo/dbmail

[Josh Berkus]

Aaron Stone wrote:

>
> On Jun 25, 2008, at 10:16 AM, Josh Berkus wrote:
>
>> Paul,
>>
>>> Don't use it for anything other than testing. There are still some
>>> pretty fundamental issues to iron out.
>>
>> Thanks for the warning.  2.2 it is.  Now, if only Ubuntu would update
>> the $&%&@ packages ...
>>
>>>> 2) If I run queries against the mail in the backend database, and
>>>> update or delete things, are there parts of dbmail which don't get
>>>> their data from the database?
>>> No. Everything is in the database. Unless you are using LDAP for
>>> authentication. That is the *only* exception.
>>
>> Oh?  Is there a non-LDAP option?  I'd love to avoid LDAP -- I thought
>> it was required.
>
> The non-LDAP option is the default: everything is in the database :-)

Keen!  Hopefully I'll be able to do a write up on replicated e-mail with
dbmail ...

Where do attachments go?

--Josh
_______________________________________________
DBmail mailing list
DBmail@...
https://mailman.fastxs.nl/mailman/listinfo/dbmail

[Naz Gassiep]

Hi Josh :)
Greetings from pg-hackers :)
- Naz.


Josh Berkus wrote:

> Aaron Stone wrote:
>>
>> On Jun 25, 2008, at 10:16 AM, Josh Berkus wrote:
>>
>>> Paul,
>>>
>>>> Don't use it for anything other than testing. There are still some
>>>> pretty fundamental issues to iron out.
>>>
>>> Thanks for the warning.  2.2 it is.  Now, if only Ubuntu would
>>> update the $&%&@ packages ...
>>>
>>>>> 2) If I run queries against the mail in the backend database, and
>>>>> update or delete things, are there parts of dbmail which don't get
>>>>> their data from the database?
>>>> No. Everything is in the database. Unless you are using LDAP for
>>>> authentication. That is the *only* exception.
>>>
>>> Oh?  Is there a non-LDAP option?  I'd love to avoid LDAP -- I
>>> thought it was required.
>>
>> The non-LDAP option is the default: everything is in the database :-)
>
> Keen!  Hopefully I'll be able to do a write up on replicated e-mail
> with dbmail ...
>
> Where do attachments go?
>
> --Josh
> _______________________________________________
> DBmail mailing list
> DBmail@...
> https://mailman.fastxs.nl/mailman/listinfo/dbmail
>

_______________________________________________
DBmail mailing list
DBmail@...
https://mailman.fastxs.nl/mailman/listinfo/dbmail

[Curtis Maurand]

I've been running it  on Ubuntu for a couple of years.  I've got 2.2
running on Ubuntu Server 6.10 LTS and it compiles from source perfectly.

Curtis

Josh Berkus wrote:

> Paul,
>
>> Don't use it for anything other than testing. There are still some
>> pretty fundamental issues to iron out.
>
> Thanks for the warning.  2.2 it is.  Now, if only Ubuntu would update
> the $&%&@ packages ...
>
>>
>>> 2) If I run queries against the mail in the backend database, and
>>> update or delete things, are there parts of dbmail which don't get
>>> their data from the database?
>>
>> No. Everything is in the database. Unless you are using LDAP for
>> authentication. That is the *only* exception.
>
> Oh?  Is there a non-LDAP option?  I'd love to avoid LDAP -- I thought
> it was required.
>
> --Josh
>
> _______________________________________________
> DBmail mailing list
> DBmail@...
> https://mailman.fastxs.nl/mailman/listinfo/dbmail

_______________________________________________
DBmail mailing list
DBmail@...
https://mailman.fastxs.nl/mailman/listinfo/dbmail

[Paul J Stevens]

Josh Berkus wrote:

> Keen!  Hopefully I'll be able to do a write up on replicated e-mail with
> dbmail ...

On postgresql perhaps?

>
> Where do attachments go?

Depends. In 2.2 everything is chopped up into 500k blocks and stuffed in
blobs that are simply concatened on retrieval. So attachments are not
saved separately. Headers are stored in normalized tables.

In 2.3+ attachments are indeed stored as atomic blobs in the mimeparts
table. The rfc822 header part of the complete message, and the
headerpart of attached mimeparts is also stored as blob in the mimeparts
table. Mimeparts are linked into actual messages by the partlists table.
Retrieval put the right blobs in the right order and depth, separated by
the correct boundary.

if you want to look at the storage behaviour of 2.3+ you can use 2.3.2
or 2.3.3 without worrying too much about losing email. I use 2.3.2 on
one key inhouse production system that carries around 25GB storage and
around 10 heavy users. Zero complaints. 2.3.3 has a lot of new and
relatively untested code with known issues and very likely some unknowns
as well. Of course I'm personally very excited about all the new stuff
which I think will help solve some fundamental issues. But I'm also
sometimes a bit overwhelmed with the tasks still before me. Behind every
hill, lies another one, so to speak. More to the point: it looks like I
have to rewrite the whole mime parser from scratch. GMime is not
threadsafe, and adding mutex locks appears to completely kill
performance. But then again, things may be not quite that bad after all.
I havent figured it out quite yet. Which makes it just one of those
things: they take time to work themselves out.




--
  ________________________________________________________________
  Paul Stevens                                      paul at nfg.nl
  NET FACILITIES GROUP                     GPG/PGP: 1024D/11F8CD31
  The Netherlands________________________________http://www.nfg.nl
_______________________________________________
DBmail mailing list
DBmail@...
https://mailman.fastxs.nl/mailman/listinfo/dbmail

[Josh Berkus]

Paul,

> > Keen!  Hopefully I'll be able to do a write up on replicated e-mail
> > with dbmail ...
>
> On postgresql perhaps?

What, like I'd use something else?  ;-)

> In 2.3+ attachments are indeed stored as atomic blobs in the mimeparts
> table. The rfc822 header part of the complete message, and the
> headerpart of attached mimeparts is also stored as blob in the mimeparts
> table. Mimeparts are linked into actual messages by the partlists table.
> Retrieval put the right blobs in the right order and depth, separated by
> the correct boundary.

Are you using BYTEA or LO for PostgreSQL?  The former is vastly easier to
manage.

> More to the point: it looks like I
> have to rewrite the whole mime parser from scratch. GMime is not
> threadsafe, and adding mutex locks appears to completely kill
> performance. But then again, things may be not quite that bad after all.
> I havent figured it out quite yet. Which makes it just one of those
> things: they take time to work themselves out.

Hmmm.  I don't know mime parsers *at all*.  Are you storing the attachments
in MIME form, or in original format?

--
--Josh

Josh Berkus
PostgreSQL @ Sun
San Francisco
_______________________________________________
DBmail mailing list
DBmail@...
https://mailman.fastxs.nl/mailman/listinfo/dbmail

[Paul J Stevens]

Josh Berkus wrote:
> Paul,
>
>>> Keen!  Hopefully I'll be able to do a write up on replicated e-mail
>>> with dbmail ...
>> On postgresql perhaps?
>
> What, like I'd use something else?  ;-)

THing is people talk a lot about pros and cons of running dbmail on a
mysql-replication setup. But I havent heard from anyone doing
replication on postgresql.

>
>> In 2.3+ attachments are indeed stored as atomic blobs in the mimeparts
>> table. The rfc822 header part of the complete message, and the
>> headerpart of attached mimeparts is also stored as blob in the mimeparts
>> table. Mimeparts are linked into actual messages by the partlists table.
>> Retrieval put the right blobs in the right order and depth, separated by
>> the correct boundary.
>
> Are you using BYTEA or LO for PostgreSQL?  The former is vastly easier to
> manage.

bytea.

>
>> More to the point: it looks like I
>> have to rewrite the whole mime parser from scratch. GMime is not
>> threadsafe, and adding mutex locks appears to completely kill
>> performance. But then again, things may be not quite that bad after all.
>> I havent figured it out quite yet. Which makes it just one of those
>> things: they take time to work themselves out.
>
> Hmmm.  I don't know mime parsers *at all*.  Are you storing the attachments
> in MIME form, or in original format?

Mimepart headers and body are stored as-is in the original message, but
separately to maintain atomicity of the bodyparts even when used in
different attachments. Decoding base64 wouldn't add much benefit other
than somewhat better coverage for text searches in message bodies.
Marginal at best.




--
  ________________________________________________________________
  Paul Stevens                                      paul at nfg.nl
  NET FACILITIES GROUP                     GPG/PGP: 1024D/11F8CD31
  The Netherlands________________________________http://www.nfg.nl
_______________________________________________
DBmail mailing list
DBmail@...
https://mailman.fastxs.nl/mailman/listinfo/dbmail

[Josh Berkus]

Paul,

> THing is people talk a lot about pros and cons of running dbmail on a
> mysql-replication setup. But I havent heard from anyone doing
> replication on postgresql.

Yeah, I'm thinking that I could use Bucardo for detached MM replication,
which would let me carry around a duplicate server on my laptop.

For scalable installations, Skytools is attractive, but I don't actually
need one of those.  Maybe when I get used to DBMail, I can talk about
implementing it for PostgreSQL.org.  Although mostly what we need for that
is listmanagers, not mailservers.

> Mimepart headers and body are stored as-is in the original message, but
> separately to maintain atomicity of the bodyparts even when used in
> different attachments. Decoding base64 wouldn't add much benefit other
> than somewhat better coverage for text searches in message bodies.
> Marginal at best.

Hmmm.  If you're storing stuff as MIME, you could actually put it in a TEXT
field in PostgreSQL.  Mind you, that just makes replication easier, it
doesn't help otherwise -- both are stored using TOAST regardless.

--
--Josh

Josh Berkus
PostgreSQL @ Sun
San Francisco
_______________________________________________
DBmail mailing list
DBmail@...
https://mailman.fastxs.nl/mailman/listinfo/dbmail

[Paul J Stevens]

Josh Berkus wrote:
> Hmmm.  If you're storing stuff as MIME, you could actually put it in a TEXT
> field in PostgreSQL.  Mind you, that just makes replication easier, it
> doesn't help otherwise -- both are stored using TOAST regardless.

That won't work afaik. Using TEXT would mean forcing the blob into the correct
encoding. There is not much added benefit I'm aware of, and plenty of downside.


--
  ________________________________________________________________
  Paul Stevens                                      paul at nfg.nl
  NET FACILITIES GROUP                     GPG/PGP: 1024D/11F8CD31
  The Netherlands________________________________http://www.nfg.nl
_______________________________________________
DBmail mailing list
DBmail@...
https://mailman.fastxs.nl/mailman/listinfo/dbmail

[Naz Gassiep]

Perhaps there's something I don't understand, but I thought that messages (including attachments) were all represented using text, i.e., UTF7. Is that not the case? (If this is a n00b question feel free to refer me to an RFC, those things don't scare me).
- Naz.

Paul J Stevens wrote:

Josh Berkus wrote:
  

Hmmm.  If you're storing stuff as MIME, you could actually put it in a TEXT 
field in PostgreSQL.  Mind you, that just makes replication easier, it 
doesn't help otherwise -- both are stored using TOAST regardless.
    

That won't work afaik. Using TEXT would mean forcing the blob into the correct
encoding. There is not much added benefit I'm aware of, and plenty of downside.


  

_______________________________________________
DBmail mailing list
DBmail@...
https://mailman.fastxs.nl/mailman/listinfo/dbmail

[Paul J Stevens]

Naz Gassiep wrote:
> Perhaps there's something I don't understand, but I thought that
> messages (including attachments) were all represented using text, i.e.,
> UTF7. Is that not the case? (If this is a n00b question feel free to
> refer me to an RFC, those things don't scare me).

If everyone followed all the nice RFCs you would be quite correct here. But
thing is email is untrusted, and may very well (in fact will) contain 8bit data.
And not all attachments will actually be encoded in the charset specified in the
Content-Type headers.


--
  ________________________________________________________________
  Paul Stevens                                      paul at nfg.nl
  NET FACILITIES GROUP                     GPG/PGP: 1024D/11F8CD31
  The Netherlands________________________________http://www.nfg.nl
_______________________________________________
DBmail mailing list
DBmail@...
https://mailman.fastxs.nl/mailman/listinfo/dbmail

[Robert Middleswarth]

I followed the wiki for AD but it seems to be very incomplete. I figured
out that I needed unix services installed for password support. But
still not sure about these two fileds

USER_OBJECTCLASS = top,account,dbmailUser
FORW_OBJECTCLASS = top,account,dbmailForwardingAddress

I could add them manually to the Scheme but in the end there is no way
to enter them into AD that I can see? How have other handled them in AD.
We are moving to 2003 R2 with unix ext turned on. How have others be
able to set this up?

Thanks
Robert

PS. I did google several diff ways but couldn't find any info.

_______________________________________________
DBmail mailing list
DBmail@...
https://mailman.fastxs.nl/mailman/listinfo/dbmail

[Brandon Adams-3]

I'm running dbmail 2.2.9 currently against a Win2k3-r2 ADS system.  I
don't have either of those two options set in my dbmail.conf file and
I am able to auth properly against the ADS.  Here is my [LDAP] section
from dbmail.conf.  Hopefully it helps.

[LDAP]
  PORT = 389
  VERSION = 3
  HOSTNAME = domain_controller.mydomain.com
  BASE_DN = OU=Some OU,DC=mydomain,DC=com
  BIND_DN = CN=user,OU=Users,DC=mydomain,DC=com
  BIND_PW = [password]
  SCOPE = SubTree
  CN_STRING = sAMAccountName
  FIELD_PASSWD = userPassword
  FIELD_UID = sAMAccountName
  FIELD_NID = uSNCreated
  MIN_NID = 10000
  MAX_NIC = 20000
  FIELD_CID = gidNumber
  MIN_CID = 10000
  MAX_CID = 20000
  FIELD_MAIL = mail
  FIELD_QUOTA = mailQuota
  FIELD_FWDTARGET = mailForwardingAddress


On Mon, Jun 30, 2008 at 11:06 AM, Robert Middleswarth
<robert@...> wrote:

> I followed the wiki for AD but it seems to be very incomplete. I figured out
> that I needed unix services installed for password support. But still not
> sure about these two fileds
>
> USER_OBJECTCLASS = top,account,dbmailUser
> FORW_OBJECTCLASS = top,account,dbmailForwardingAddress
>
> I could add them manually to the Scheme but in the end there is no way to
> enter them into AD that I can see? How have other handled them in AD. We are
> moving to 2003 R2 with unix ext turned on. How have others be able to set
> this up?
>
> Thanks
> Robert
>
> PS. I did google several diff ways but couldn't find any info.
>
> _______________________________________________
> DBmail mailing list
> DBmail@...
> https://mailman.fastxs.nl/mailman/listinfo/dbmail
>

--

[ Brandon Adams ]
bmadams at gmail dot com
_______________________________________________
DBmail mailing list
DBmail@...
https://mailman.fastxs.nl/mailman/listinfo/dbmail