shared mailing groups

[bdogodev]

I had originally envisaged users employing which ever mail client they
preferred, with the client simply using IMAP to interact with the OGO
server.  Since there are a wide variety of email clients in use, it seems
that there is more diversity of preference among users with regard to this
than with any other application.

The main problem with users using a range of email applications is that
contact information is duplicated (unless I produce the means to reconcile
the address books of different email clients with OGO contacts e.g. using
vCard),  Another problem with a diversity of email clients is that mailing
groups cannot be shared.  Furthermore, it means I can't offer the same
degree of technical support for so many mail clients, as I could for a
single mail client.

As part of my attempt to decide what to do, I'm wondering what would be
involved in implementing shared 'mailing groups' based on OGO contact
data.

Looking at the web interface to OGO, I see that there are mailing lists
that can be created, but these seem not to be sharable.  Moreover, if the
same email address was found on multiple lists, then that email address
received multiple copies of the email sent to the lists.  This would seem
to me to be undesirable behaviour - I would want to resolve the lists of
recipients so that if an address is found on more than one list, that
address would still only receive one copy of the email.

I see that the zOGI interface to OGO does not deal with mail, whilst the
old xml-rpc interface does.  For the moment, I'm prepared to just use the
old interface for sending mail (at some point further down the line, maybe
I'll look at adding mail to the zOGI interface, unless that's something
that zOGI is designed not to do).  From the zOGI interface, it does appear
that Teams can contain Contacts (and not just Accounts), so I'm assuming
that Teams would serve as a good basis for mailing groups.  My idea would
be to extract the Contact details from the Team, resolve unique email
addresses locally, populate the xml-rpc Mail object with the details, and
send the email through the OGO server via that old interface.
 
However, I think I may have misunderstood the nature of the Contacts who
are in Teams.  I searched the OpenGroupware mailing list archive and found
this post (http://www.nabble.com/Team-Mailing-List-td4017149.html#a4017149
) that shows that an email sent to a team will be delivered to each of the
team members if the team has no collective email address configured.
However, in the web UI, teams can only be made up of Accounts.  This leads
me to believe that my understanding of the zOGI interface to Teams is
mistaken.

I'd be grateful to hear what others think about this, and what you do in
regard to mailing groups.    I can see that what I'm trying to do maybe
falls between what zOGI is designed to do, and what the OGO web interface
is designed to do.

A further problem I foresee is giving users access via Funambol -- this
would also mean that they are using the local email client of the phone,
and thus will lose the shared mailing groups (although it's great that
Funambol can take care of the synchronization of contact data).  Which
ever way I look at this, I can't seem to find an obvious solution.  Maybe
I should just go back to the idea of not providing an email client.  That
way at least they wouldn't expect to have shared mailing groups on their
phones.

Regards,

Bernard
--
OpenGroupware.org XML-RPC
xmlrpc@...
http://mail.opengroupware.org/mailman/listinfo/xmlrpc

[Adam Tauno Williams]

> I had originally envisaged users employing which ever mail client they
> preferred, with the client simply using IMAP to interact with the OGO
> server.

Not sure what your mean.  IMAP doesn't really have anything to do with
OGo, that is just whatever IMAP server (hopefully Cyrus) is in the
backend.

> Since there are a wide variety of email clients in use, it seems
> that there is more diversity of preference among users with regard to this
> than with any other application.

Often not 'wise' diversity (witness the number of people who think TB is
a great client, ugh!) but true.

> The main problem with users using a range of email applications is that
> contact information is duplicated (unless I produce the means to reconcile
> the address books of different email clients with OGO contacts e.g. using
> vCard),  

Yep. AFAIK, every major client now supports GroupDAV. (?)

> Another problem with a diversity of email clients is that mailing
> groups cannot be shared.  Furthermore, it means I can't offer the same
> degree of technical support for so many mail clients, as I could for a
> single mail client.
> As part of my attempt to decide what to do, I'm wondering what would be
> involved in implementing shared 'mailing groups' based on OGO contact
> data.

Interesting, I've thought about this before.

> Looking at the web interface to OGO, I see that there are mailing lists
> that can be created, but these seem not to be sharable.

Yea, the mailist support in OGo WebUI, is IMHO, pretty much a hack.

> Moreover, if the
> same email address was found on multiple lists, then that email address
> received multiple copies of the email sent to the lists.  

Did you test this?  If it sends it all as one message the MTA should
deal with this (compress the recipients, and most mailstores suppress
duplicate deliveries anyway).   But maybe I recall that OGo's mailist
thingy sends and individual message to each recipient.

> I see that the zOGI interface to OGO does not deal with mail, whilst the
> old xml-rpc interface does.

Sort of.  Does the old xml-rpcd's mail API even work?  It is really darn
ugly.  The latest versions of zOGI do have the beginnings of mail
support; at least for sending.  So adding something more isn't out of
the question.

>  For the moment, I'm prepared to just use the
> old interface for sending mail (at some point further down the line, maybe
> I'll look at adding mail to the zOGI interface, unless that's something
> that zOGI is designed not to do).

No, the expressed goal of zOGI is to support/expose *ALL* of the
server's functionality.  Of course it isn't there yet.  Another
[unspoken] goal is to allow clients to communicate via HTTP with the OGo
server and require *NO* other connections (as sometimes anything beyond
HTTP is hard to get allowed).  So adding some degree of mail support is
inevitable.

> From the zOGI interface, it does appear
> that Teams can contain Contacts (and not just Accounts), so I'm assuming
> that Teams would serve as a good basis for mailing groups.

If that is true it is an 'unintended feature'.  Teams should only
contain accounts.

>  My idea would
> be to extract the Contact details from the Team, resolve unique email
> addresses locally, populate the xml-rpc Mail object with the details, and
> send the email through the OGO server via that old interface.

Seems reasonable although the server might be able to help.  Look at the
code for the getNotifications API method.
 
> However, I think I may have misunderstood the nature of the Contacts who
> are in Teams.  I searched the OpenGroupware mailing list archive and found
> this post (http://www.nabble.com/Team-Mailing-List-td4017149.html#a4017149
> ) that shows that an email sent to a team will be delivered to each of the
> team members if the team has no collective email address configured.

Hmm.  That is probably a feature specific to the WebUI's mail client.

> However, in the web UI, teams can only be made up of Accounts.  This leads
> me to believe that my understanding of the zOGI interface to Teams is
> mistaken.

Yep, my bug!

> I'd be grateful to hear what others think about this, and what you do in
> regard to mailing groups.    I can see that what I'm trying to do maybe
> falls between what zOGI is designed to do, and what the OGO web interface
> is designed to do.

It seems reasonable, and not terribly hard, to create a new entity:
Group -that can contain contacts & enterprises.

> A further problem I foresee is giving users access via Funambol -- this
> would also mean that they are using the local email client of the phone,
> and thus will lose the shared mailing groups (although it's great that
> Funambol can take care of the synchronization of contact data).

Correct, on that level the groups need to be supported by the MTA.  But
that I think is really trivial (at least if your MTA is Postfix).
Delivery is probably best handled by the MTA anyway for a variety of
reasons.

> Which
> ever way I look at this, I can't seem to find an obvious solution.  Maybe
> I should just go back to the idea of not providing an email client.  That
> way at least they wouldn't expect to have shared mailing groups on their
> phones.
--
OpenGroupware developer: awilliam@...
<http://whitemiceconsulting.blogspot.com/>
OpenGroupare & Cyrus IMAPd documenation @
<http://docs.opengroupware.org/Members/whitemice/wmogag/file_view>

--
OpenGroupware.org XML-RPC
xmlrpc@...
http://mail.opengroupware.org/mailman/listinfo/xmlrpc

[bdogodev]

> > if the
> > same email address was found on multiple lists, then that email
address
> > received multiple copies of the email sent to the lists.
>
> Did you test this?  If it sends it all as one message the MTA should
> deal with this (compress the recipients, and most mailstores suppress
> duplicate deliveries anyway).   But maybe I recall that OGo's mailist
> thingy sends and individual message to each recipient.

In my tests from my OGO server running Postfix to my Gmail account, if the
same email address was in 2 mailing lists, then it received 2 copies of
the email.  I guess the WebUI mailing list feature is just more basic than
one might expect.
 
> > I see that the zOGI interface to OGO does not deal with mail, whilst
the
> > old xml-rpc interface does.
>
> Sort of.  Does the old xml-rpcd's mail API even work?  It is really darn
> ugly.

I tested it and it does work (although the sample code in the old xml-rpc
PDF is wrong.  After fiddling with that a little, it worked.  (For anyone
whose interested, the documentation refers e.g. to methods as
'server.setFrom'
when they need to be 'server.mail.setFrom').

> The latest versions of zOGI do have the beginnings of mail
> support; at least for sending.  So adding something more isn't out of
> the question.

That's good to hear :-)
 
> > From the zOGI interface, it does appear
> > that Teams can contain Contacts (and not just Accounts), so I'm
assuming
> > that Teams would serve as a good basis for mailing groups.
>
> If that is true it is an 'unintended feature'.  Teams should only
> contain accounts.

It could just be my misunderstanding of the sample code on the zOGI site.
I haven't tested it yet.  However, knowing that that is not how Teams
should work, I won't be using that as it may well become 'broken' in the
future.
 
> >  My idea would
> > be to extract the Contact details from the Team, resolve unique email
> > addresses locally, populate the xml-rpc Mail object with the details,
and
> > send the email through the OGO server via that old interface.
>
> Seems reasonable although the server might be able to help.  Look at the
> code for the getNotifications API method.

Sorry, but I don't understand how getNotifications might be useful.  It
only seems to relate to Appointments.

> > I'd be grateful to hear what others think about this, and what you do
in
> > regard to mailing groups.    I can see that what I'm trying to do
maybe
> > falls between what zOGI is designed to do, and what the OGO web
interface
> > is designed to do.
>
> It seems reasonable, and not terribly hard, to create a new entity:
> Group -that can contain contacts & enterprises.

I haven't started to look at the Objective C code in OGO, but I'm relieved
to hear that adding Groups should not be too difficult.
 
> > A further problem I foresee is giving users access via Funambol --
this
> > would also mean that they are using the local email client of the
phone,
> > and thus will lose the shared mailing groups (although it's great that

> > Funambol can take care of the synchronization of contact data).
>
> Correct, on that level the groups need to be supported by the MTA.  But
> that I think is really trivial (at least if your MTA is Postfix).
> Delivery is probably best handled by the MTA anyway for a variety of
> reasons.

That the MTA should resolve the recipient identity sounds right, but maybe
it is expecting too much.  My installation of Postfix didn't seem to
reduce multiple emails to the same recipient to just one email.

The issue I'm referring to with providing access via Funambol, is that the
shared mailing groups would be unavailable on a phone, unless the phone
email clients understand mailing groups and Funambol could sync such
things between the server and the phone.  I'd be very surprised to hear
that such synchronization was possible.


Bernard

--
OpenGroupware.org XML-RPC
xmlrpc@...
http://mail.opengroupware.org/mailman/listinfo/xmlrpc

[Adam Tauno Williams]

> > > From the zOGI interface, it does appear
> > > that Teams can contain Contacts (and not just Accounts), so I'm
> > > assuming
> > > that Teams would serve as a good basis for mailing groups. >
> > If that is true it is an 'unintended feature'.  Teams should only
> > contain accounts.
> It could just be my misunderstanding of the sample code on the zOGI site.
> I haven't tested it yet.  However, knowing that that is not how Teams
> should work, I won't be using that as it may well become 'broken' in the
> future.
> > >  My idea would
> > > be to extract the Contact details from the Team, resolve unique email
> > > addresses locally, populate the xml-rpc Mail object with the details,
> > > and send the email through the OGO server via that old interface.
> > Seems reasonable although the server might be able to help.  Look at the
> > code for the getNotifications API method.
> Sorry, but I don't understand how getNotifications might be useful.  It
> only seems to relate to Appointments.

But the concept is that the server can 'unroll' some of the data for
you.  Rather than just returning the appointments that require
notification it creates a transient entity
<http://code.google.com/p/zogi/wiki/UnderstandingEntities> with the
information you are [obviously] going to need.

> > > I'd be grateful to hear what others think about this, and what you do
> > > in regard to mailing groups.    I can see that what I'm trying to do
> > > maybe falls between what zOGI is designed to do, and what the OGO web
> > > interface is designed to do.
> > It seems reasonable, and not terribly hard, to create a new entity:
> > Group -that can contain contacts & enterprises.
> I haven't started to look at the Objective C code in OGO, but I'm relieved
> to hear that adding Groups should not be too difficult.

The hardest part would be injecting it into the WebDAV space so that
something like Funambol could sync.  But beyond that I think a group's
representation in GroupDAV is just a simple vCard.

> > > A further problem I foresee is giving users access via Funambol --
> > > this  would also mean that they are using the local email client
> > > of the phone,
> > > and thus will lose the shared mailing groups (although it's great
> > > that  Funambol can take care of the synchronization of contact
> > > data).
> > Correct, on that level the groups need to be supported by the MTA.
But
> > that I think is really trivial (at least if your MTA is Postfix).
> > Delivery is probably best handled by the MTA anyway for a variety of
> > reasons.
> That the MTA should resolve the recipient identity sounds right, but maybe
> it is expecting too much.  My installation of Postfix didn't seem to
> reduce multiple emails to the same recipient to just one email.

It should flatten [I believe] if you specify the same destination more
than one *in the same transaction*.  If you perform two operations then
it is oblivious the two messages are related.   This is why it is good
to expand groups in the MTA and not externally.

> The issue I'm referring to with providing access via Funambol, is that the
> shared mailing groups would be unavailable on a phone, unless the phone
> email clients understand mailing groups and Funambol could sync such
> things between the server and the phone.  I'd be very surprised to hear
> that such synchronization was possible.

I think a group is a vCard just like a contact and an enterprise.  The
e-mail address in a group should/would indicate an address that is
expanded by the MTA (which will query to OGo backend).

--
OpenGroupware.org XML-RPC
xmlrpc@...
http://mail.opengroupware.org/mailman/listinfo/xmlrpc

[bdogodev]

> > Sorry, but I don't understand how getNotifications might be useful. It

> > only seems to relate to Appointments.
>
> But the concept is that the server can 'unroll' some of the data for
> you.  Rather than just returning the appointments that require
> notification it creates a transient entity
> <http://code.google.com/p/zogi/wiki/UnderstandingEntities> with the
> information you are [obviously] going to need.

Do you mean the idea is that zOGI would return a Group as a transient
entity?

> > That the MTA should resolve the recipient identity sounds right, but
maybe
> > it is expecting too much.  My installation of Postfix didn't seem to
> > reduce multiple emails to the same recipient to just one email.
>
> It should flatten [I believe] if you specify the same destination more
> than one *in the same transaction*.  If you perform two operations then
> it is oblivious the two messages are related.   This is why it is good
> to expand groups in the MTA and not externally.

I can see how the single transaction could eliminate duplicate emails.  I
don't see how the MTA could expand a group name into a list of email
addresses.  Maybe I'm misunderstanding what you mean by "expand".
 
> > The issue I'm referring to with providing access via Funambol, is that
the
> > shared mailing groups would be unavailable on a phone, unless the
phone
> > email clients understand mailing groups and Funambol could sync such
> > things between the server and the phone.  I'd be very surprised to
hear
> > that such synchronization was possible.
>
> I think a group is a vCard just like a contact and an enterprise.  The
> e-mail address in a group should/would indicate an address that is
> expanded by the MTA (which will query to OGo backend).

I think I might have caught up with your thinking on this.  It occurred to
me later that a Group could just be an email address (e.g.
prospective_customers@...).  Users would send an email to that
address, and that address would be expanded into a list of recipients, and
the email redirected to each member of the list.

Following this idea then users can use whichever email client they want,
whilst the server would still maintain the shared group.  The question is
how to redirect email to that group address, and how to maintain the
group.

One way of doing the expansion might be using sieve.  I can see that sieve
can redirect email to at least one recipient. I will need to see if the
sender remains unchanged in the redirect, and whether or not the
redirection can be to multiple people.  If either of those don't work,
then sieve is a non-starter.

That would mean OGO writing out sieve scripts, which may be completely
inappropriate. I'm sure others have got better ideas about how this should
be implemented.

Bernard
--
OpenGroupware.org XML-RPC
xmlrpc@...
http://mail.opengroupware.org/mailman/listinfo/xmlrpc

[bdogodev]

> One way of doing the expansion might be using sieve.  I can see that
> sieve can redirect email to at least one recipient. I will need to
> see if the sender remains unchanged in the redirect, and whether or
> not the redirection can be to multiple people.  If either of those
> don't work, then sieve is a non-starter.

I checked out the sieve documentation, and technically the above would not
appear to be issues.  However, I suspect there would still be the problem
of multiple emails being sent to someone who was in more than one mailing
group.  I will have to try it out, and see if the redirects happen within
one MTA transaction.  But I don't see why they would, so I'm not hopeful
that this has any potential as part of a solution.

Bernard
--
OpenGroupware.org XML-RPC
xmlrpc@...
http://mail.opengroupware.org/mailman/listinfo/xmlrpc

[Adam Tauno Williams]

On Fri, 2009-07-10 at 21:02 +0100, bdogodev@...
wrote:
> > > Sorry, but I don't understand how getNotifications might be useful. It
> > > only seems to relate to Appointments.
> > But the concept is that the server can 'unroll' some of the data for
> > you.  Rather than just returning the appointments that require
> > notification it creates a transient entity
> > <http://code.google.com/p/zogi/wiki/UnderstandingEntities> with the
> > information you are [obviously] going to need.
> Do you mean the idea is that zOGI would return a Group as a transient
> entity?

For zOGI clients it could fill in all the group information (gather the
e-mail addreses of every company object in the group, etc...)

> > > That the MTA should resolve the recipient identity sounds right, but
> maybe
> > > it is expecting too much.  My installation of Postfix didn't seem to
> > > reduce multiple emails to the same recipient to just one email.
> >
> > It should flatten [I believe] if you specify the same destination more
> > than one *in the same transaction*.  If you perform two operations then
> > it is oblivious the two messages are related.   This is why it is good
> > to expand groups in the MTA and not externally.
> I can see how the single transaction could eliminate duplicate emails.  I
> don't see how the MTA could expand a group name into a list of email
> addresses.  Maybe I'm misunderstanding what you mean by "expand".

Easy.  MTA's do that all the time.  Sounds like a new section of
WMOGAG! :)

> > I think a group is a vCard just like a contact and an enterprise.  The
> > e-mail address in a group should/would indicate an address that is
> > expanded by the MTA (which will query to OGo backend).
> I think I might have caught up with your thinking on this.  It occurred to
> me later that a Group could just be an email address (e.g.
> prospective_customers@...).  Users would send an email to that
> address, and that address would be expanded into a list of recipients, and
> the email redirected to each member of the list.

Yep, no fuss on the client side.

> Following this idea then users can use whichever email client they want,
> whilst the server would still maintain the shared group.  The question is
> how to redirect email to that group address, and how to maintain the
> group.

Yep.

> One way of doing the expansion might be using sieve.  I can see that sieve
> can redirect email to at least one recipient. I will need to see if the
> sender remains unchanged in the redirect, and whether or not the
> redirection can be to multiple people.  If either of those don't work,
> then sieve is a non-starter.
> That would mean OGO writing out sieve scripts, which may be completely
> inappropriate. I'm sure others have got better ideas about how this should
> be implemented.

Nope, it is easy to hook OGo directly into the MTA.  There is already a
section in WMOGAG on how to use OGo to verify recipients.  We'd just do
the reverse.

--
OpenGroupware.org XML-RPC
xmlrpc@...
http://mail.opengroupware.org/mailman/listinfo/xmlrpc

[Adam Tauno Williams]

On Fri, 2009-07-10 at 21:02 +0100, bdogodev@...

> > One way of doing the expansion might be using sieve.  I can see that sieve
> > can redirect email to at least one recipient. I will need to see if the
> > sender remains unchanged in the redirect, and whether or not the
> > redirection can be to multiple people.  If either of those don't work,
> > then sieve is a non-starter.
> > That would mean OGO writing out sieve scripts, which may be completely
> > inappropriate. I'm sure others have got better ideas about how this should
> > be implemented.
> Nope, it is easy to hook OGo directly into the MTA.  There is already a
> section in WMOGAG on how to use OGo to verify recipients.  We'd just do
> the reverse.

Did you ever look into hooking your Postfix into the OGo DB?

--
OpenGroupware.org XML-RPC
xmlrpc@...
http://mail.opengroupware.org/mailman/listinfo/xmlrpc

[bdogodev]

I did look at this some more but wasn't completely satisified with the
result.

Instead of working out how to get Postfix to resolve the group members by
reading the database I just created a file system group to see if Postfix
would solve the duplicate emails issue.

I needed to create files for each mailing group, contain comma-separated
email addresses.  Then I added these lines to /etc/aliases :

test-list1:          :include:/var/mailgroups/test1.lst
test-list2:          :include:/var/mailgroups/test2.lst

then ran  'postalias /etc/aliases'.

When mail was addressed to one of those 2 mailing groups, postfix would
read the list of addresses out of the file system, and send the email
those addresses.  That bit worked well.  If the list changed on the file
system, postfix would even use the changed list.

What I wasn't satisfied with was that Postfix would still send duplicate
emails if an address appears in more than one of the mailing groups for an
email that was sent to multiple mailing groups.

I think that problem will remain even if Postfix was looking to the OGO
database to resolve the addresses within a group.

Bernard

> Adam Tauno Williams
>
>
> On Fri, 2009-07-10 at 21:02 +0100, bdogodev@...
> > > One way of doing the expansion might be using sieve.  I can see
> that sieve
> > > can redirect email to at least one recipient. I will need to see if
the
> > > sender remains unchanged in the redirect, and whether or not the
> > > redirection can be to multiple people.  If either of those don't
work,
> > > then sieve is a non-starter.
> > > That would mean OGO writing out sieve scripts, which may be
completely
> > > inappropriate. I'm sure others have got better ideas about how
> this should
> > > be implemented.
> > Nope, it is easy to hook OGo directly into the MTA.  There is already
a
> > section in WMOGAG on how to use OGo to verify recipients.  We'd just
do
> > the reverse.
>
> Did you ever look into hooking your Postfix into the OGo DB?

--
OpenGroupware.org XML-RPC
xmlrpc@...
http://mail.opengroupware.org/mailman/listinfo/xmlrpc

[Renzo Vettori]

"Our" solution was to have sympa as a mailing list manager and configure some
lists to get subscribers from ldap (ogo auth is against ldap) and sql queries
(ogo contacts/enterprises replicate into our home built cms).

Of course, that was not the only reason to have a mailing list manager ..

Greetings, Renzo Vettori


xmlrpc@... wrote:
> I did look at this some more but wasn't completely satisified with the
> result.
>
> Instead of working out how to get Postfix to resolve the group members by
> reading the database I just created a file system group to see if Postfix
> would solve the duplicate emails issue.
>
> I needed to create files for each mailing group, contain comma-separated
...

--
OpenGroupware.org XML-RPC
xmlrpc@...
http://mail.opengroupware.org/mailman/listinfo/xmlrpc