Debian lenny and svn opensync

[Chris Frey-2]

Hi,

Is anyone else using Debian Lenny successfully with the latest SVN opensync
and evolution plugin?  The results I'm getting are almost approaching random,
with crashes and segfaults and hangs, generating core dumps with backtraces
into libc and libgconf.

I just want to know I'm not crazy. :-)  If other people are using it on
Debian Lenny, I'd dearly love to know your config.

Thanks,
- Chris


------------------------------------------------------------------------------
_______________________________________________
Opensync-devel mailing list
Opensync-devel@...
https://lists.sourceforge.net/lists/listinfo/opensync-devel

[Christian Hilgers-2]

Chris Frey wrote:
> Hi,
>
> Is anyone else using Debian Lenny successfully with the latest SVN opensync
> and evolution plugin?  The results I'm getting are almost approaching random,

from http://www.opensync.org/wiki/status
the evo plugin is still marked alpha

> with crashes and segfaults and hangs, generating core dumps with backtraces
> into libc and libgconf.

dont shoot the messanger :)
Create traces and check them, search the bugs for similar issues.

> I just want to know I'm not crazy. :-)  If other people are using it on
> Debian Lenny, I'd dearly love to know your config.

I am not using it on Lenny stable, but I run three of the build hosts
see http://opensync.org/testing/index.php?project=OpenSync

So aachen is Debian 5.0 unstable/testing mixture.
clio3 is still etch but will be upgraded to lenny as soon
as I have time and dare to do it :)


Christian
--
Christian Hilgers                  |ConSol*
Tel.   [phone]            |Consulting&Solutions Software GmbH
Fax    [phone]            |Berliner Str. 101, 40880 Ratingen
email: Christian.Hilgers@... |WWW: http://www.consol.de


------------------------------------------------------------------------------
_______________________________________________
Opensync-devel mailing list
Opensync-devel@...
https://lists.sourceforge.net/lists/listinfo/opensync-devel

[Daniel Gollub-2]

Hi Chris,

On Thursday 26 March 2009 06:26:28 am Chris Frey wrote:
> Is anyone else using Debian Lenny successfully with the latest SVN opensync
> and evolution plugin?  The results I'm getting are almost approaching
> random, with crashes and segfaults and hangs, generating core dumps with
> backtraces into libc and libgconf.

Interesting - last night someone on IRC had exactly the same problem...
http://pastebin.ca/1372538

Could you try attached patch?


>
> I just want to know I'm not crazy. :-)  If other people are using it on
> Debian Lenny, I'd dearly love to know your config.

Looks like you're not alone - it's working for me. But i'm running openSUSE 11.1.
The guy from last night was also using lenny - but he wasn't sure if his installation was
"clean". But since you're within 12h the second guy i guess their is an issue with the recent
packaging or recent version of evolution or e-d-s.

Which exactly version of evolution and e-d-s and libgconf are you running?

Best Regards,
Daniel

---

Index: src/evolution2_sync.c
===================================================================
--- src/evolution2_sync.c       (revision 5315)
+++ src/evolution2_sync.c       (working copy)
 -213,6 +213,7 @@
        osync_plugin_set_longname(plugin, "Evolution 2.x");
        osync_plugin_set_description(plugin, "Address book, calendar and task list of
Evolution 2");
 
+       osync_plugin_set_start_type(plugin, OSYNC_START_TYPE_PROCESS);
        osync_plugin_set_initialize(plugin, evo2_initialize);
        osync_plugin_set_finalize(plugin, evo2_finalize);
        osync_plugin_set_discover(plugin, evo2_discover);

------------------------------------------------------------------------------
_______________________________________________
Opensync-devel mailing list
Opensync-devel@...
https://lists.sourceforge.net/lists/listinfo/opensync-devel

[Chris Frey-2]

On Thu, Mar 26, 2009 at 11:31:21AM +0100, Daniel Gollub wrote:
> Interesting - last night someone on IRC had exactly the same problem...
> http://pastebin.ca/1372538
>
> Could you try attached patch?

Sorry, I should have mentioned it... that was me on IRC :-)

The patch seemed to work at the time, and I added a ticket for it,
but later on, I got other similar crashes.  Sometimes hard to reproduce
reliably.


> > I just want to know I'm not crazy. :-)  If other people are using it on
> > Debian Lenny, I'd dearly love to know your config.
>
> Looks like you're not alone - it's working for me. But i'm running openSUSE 11.1.
> The guy from last night was also using lenny - but he wasn't sure if his installation was
> "clean". But since you're within 12h the second guy i guess their is an issue with the recent
> packaging or recent version of evolution or e-d-s.
>
> Which exactly version of evolution and e-d-s and libgconf are you running?

Last night, I went through removing e-d-s related packages and installing
fresh ones from lenny.  So now I'm sure I'm using a set of devel libraries
(evolution-data-server-dev, libedataserver1.2, libebook1.2, etc) all
with the same version of 2.22.3-1.1.

All Lenny versions of opensync 0.22 are removed as well, but I don't think
that was having an effect.

Is there a document somewhere that describes the internal architecture
of the opensync library?  It seems to create a lot of threads, which
appear to be used for message passing.  It is almost always in these
threads where the crashes occur (which makes some sense, if the crash
happens in the evolution plugin).

Thanks,
- Chris


------------------------------------------------------------------------------
_______________________________________________
Opensync-devel mailing list
Opensync-devel@...
https://lists.sourceforge.net/lists/listinfo/opensync-devel

[Chris Frey-2]

On Thu, Mar 26, 2009 at 10:21:53AM +0100, Christian Hilgers wrote:
> from http://www.opensync.org/wiki/status
> the evo plugin is still marked alpha

Thanks, good to know. :-)


> > with crashes and segfaults and hangs, generating core dumps with backtraces
> > into libc and libgconf.
>
> dont shoot the messanger :)
> Create traces and check them, search the bugs for similar issues.

Searching for 'libgconf' doesn't turn up any other tickets except mine.


> > I just want to know I'm not crazy. :-)  If other people are using it on
> > Debian Lenny, I'd dearly love to know your config.
>
> I am not using it on Lenny stable, but I run three of the build hosts
> see http://opensync.org/testing/index.php?project=OpenSync
>
> So aachen is Debian 5.0 unstable/testing mixture.
> clio3 is still etch but will be upgraded to lenny as soon
> as I have time and dare to do it :)

I was having issues on etch as well, although not identical, but upgraded
to lenny since I needed to do that anyway, hoping that it was just a
matter of using old libraries.

Thanks for your responses.  I'll see how much further I get today.

- Chris


------------------------------------------------------------------------------
_______________________________________________
Opensync-devel mailing list
Opensync-devel@...
https://lists.sourceforge.net/lists/listinfo/opensync-devel

[Chris Frey-2]

This patch does nothing to handle actual memory freeing, so leaks
may be introduced here... it may help avoid crashes though.
---

I decided to turn on all compiler warnings and try to fix everything that
popped up.  This patch ended up making my evolution plugin test runs
MUCH smoother! :-)

In my Debian Lenny system, the evolution-data-server packages install
/usr/include/evolution-data-server-2.22/libical/ical.h, which contains
the following code at the end:

        #ifndef HANDLE_LIBICAL_MEMORY
        #warning "Please ensure that the memory returned by the functions mentioned at http://bugzilla.gnome.org/show_bug.cgi?id=516408#c1 are free'ed"
        #endif

        #define LIBICAL_MEMFIXES 1

If this change is not supported in the plugin, then according to the
URL above, there's a ring buffer that's freed internally in libical, and
this would explain my near random crash behaviour.

- Chris


 src/evolution2_sync.h |    4 ++++
 tools/list_sources.c  |    4 ++++
 tools/test_uri.c      |    4 ++++
 3 files changed, 12 insertions(+), 0 deletions(-)

diff --git a/src/evolution2_sync.h b/src/evolution2_sync.h
index 22deb72..d7cabf2 100644
--- a/src/evolution2_sync.h
+++ b/src/evolution2_sync.h
@@ -3,6 +3,10 @@
 
 //#include "evo2_sync.h"
 
+#ifndef HANDLE_LIBICAL_MEMORY
+#define HANDLE_LIBICAL_MEMORY 1
+#endif
+
 #include <opensync/opensync.h>
 
 #include <libecal/e-cal.h>
diff --git a/tools/list_sources.c b/tools/list_sources.c
index fb25499..ddebf90 100644
--- a/tools/list_sources.c
+++ b/tools/list_sources.c
@@ -1,3 +1,7 @@
+#ifndef HANDLE_LIBICAL_MEMORY
+#define HANDLE_LIBICAL_MEMORY 1
+#endif
+
 #include <glib.h>
 #include <libecal/e-cal.h>
 #include <libebook/e-book.h>
diff --git a/tools/test_uri.c b/tools/test_uri.c
index 476a6b0..f9a0922 100644
--- a/tools/test_uri.c
+++ b/tools/test_uri.c
@@ -1,3 +1,7 @@
+#ifndef HANDLE_LIBICAL_MEMORY
+#define HANDLE_LIBICAL_MEMORY 1
+#endif
+
 #include <string.h>
 #include <glib.h>
 #include <libecal/e-cal.h>
--
1.6.2.1


------------------------------------------------------------------------------
_______________________________________________
Opensync-devel mailing list
Opensync-devel@...
https://lists.sourceforge.net/lists/listinfo/opensync-devel

[Michael Banck]

On Thu, Mar 26, 2009 at 01:56:43PM -0400, Chris Frey wrote:
> I was having issues on etch as well, although not identical, but upgraded
> to lenny since I needed to do that anyway, hoping that it was just a
> matter of using old libraries.

Just to be sure - you don't have any libraries in /usr/local/lib? Or
rather, is ldd on libopensync*so showing some non-standard paths maybe?


Michael

------------------------------------------------------------------------------
_______________________________________________
Opensync-devel mailing list
Opensync-devel@...
https://lists.sourceforge.net/lists/listinfo/opensync-devel

[Chris Frey-2]

On Thu, Mar 26, 2009 at 05:20:52PM -0400, Chris Frey wrote:
> This patch does nothing to handle actual memory freeing, so leaks
> may be introduced here... it may help avoid crashes though.

I grepped through the evolution plugin source, and didn't find any
of the functions mentioned at:
        http://bugzilla.gnome.org/show_bug.cgi?id=516408#c1

So I believe this patch is safe.

Also, this patch fixed the crashes I was seeing, even without the earlier
patch from Daniel Gollub.  I don't fully understand Daniel's patch, but
fortunately it is optional.

- Chris


------------------------------------------------------------------------------
_______________________________________________
Opensync-devel mailing list
Opensync-devel@...
https://lists.sourceforge.net/lists/listinfo/opensync-devel

[Chris Frey-2]

On Thu, Mar 26, 2009 at 11:34:31PM +0100, Michael Banck wrote:
> Just to be sure - you don't have any libraries in /usr/local/lib? Or
> rather, is ldd on libopensync*so showing some non-standard paths maybe?

My patch seems to have fixed my crash problems, but I double checked
the ldd output just for completeness, and nothing unusual was listed.

Thanks,
- Chris


------------------------------------------------------------------------------
_______________________________________________
Opensync-devel mailing list
Opensync-devel@...
https://lists.sourceforge.net/lists/listinfo/opensync-devel

[Patrick Ohly]

On Thu, 2009-03-26 at 17:20 -0400, Chris Frey wrote:

> In my Debian Lenny system, the evolution-data-server packages install
> /usr/include/evolution-data-server-2.22/libical/ical.h, which contains
> the following code at the end:
>
> #ifndef HANDLE_LIBICAL_MEMORY
> #warning "Please ensure that the memory returned by the functions mentioned at http://bugzilla.gnome.org/show_bug.cgi?id=516408#c1 are free'ed"
> #endif
>
> #define LIBICAL_MEMFIXES 1
>
> If this change is not supported in the plugin, then according to the
> URL above, there's a ring buffer that's freed internally in libical, and
> this would explain my near random crash behaviour.

Not quite: if the plugin does not support this change in libecal's copy
if libical, then it will leak memory. libical will not fall back to
using the ring buffer if HANDLE_LIBICAL_MEMORY is undefined. The only
purpose of that define is to suppress the warning when the user of the
library is doing the right thing.

This particular change in libical was not merged into upstream libical,
which is what libecal 2.26 is using. Instead libical decided to not
break the API of libical and added _r variants of the relevant
functions. The traditional variants still use the ring buffer.

For apps this means they *must* check LIBICAL_MEMFIXES before freeing
some of these strings - or never free them and hope that it won't matter
because a) the strings are small or b) the app is going to be used with
a libical which doesn't need it.

Using the _r variants if available is the best approach. The ring buffer
has been a source of problems because of the indeterministic life time
of the strings, so it is better to not depend on it.

--
Bye, Patrick Ohly
--  
Patrick.Ohly@...
http://www.estamos.de/


------------------------------------------------------------------------------
_______________________________________________
Opensync-devel mailing list
Opensync-devel@...
https://lists.sourceforge.net/lists/listinfo/opensync-devel

[Patrick Ohly]

On Thu, 2009-03-26 at 19:13 -0400, Chris Frey wrote:
> Also, this patch fixed the crashes I was seeing, even without the
> earlier patch from Daniel Gollub.

I suspect that it only seemed to work better. Defining
HANDLE_LIBICAL_MEMORY doesn't change anything at runtime.

--
Bye, Patrick Ohly
--  
Patrick.Ohly@...
http://www.estamos.de/


------------------------------------------------------------------------------
_______________________________________________
Opensync-devel mailing list
Opensync-devel@...
https://lists.sourceforge.net/lists/listinfo/opensync-devel

[Chris Frey-2]

On Mon, Mar 30, 2009 at 09:21:32AM +0200, Patrick Ohly wrote:
> On Thu, 2009-03-26 at 19:13 -0400, Chris Frey wrote:
> > Also, this patch fixed the crashes I was seeing, even without the
> > earlier patch from Daniel Gollub.
>
> I suspect that it only seemed to work better. Defining
> HANDLE_LIBICAL_MEMORY doesn't change anything at runtime.

Well, it could have been user error on my part, but all I know is that
without this patch, it crashed very regularly.  With it, it was actually
possible for me to use opensync.

If you have a more accurate patch, that's great... I'm only sharing
what worked for me, since I'm not expert enough in the plugin code or
opensync to suggest a real fix.  For me, leaking was better than crashing.

If I can help nail down a more accurate fix, please let me know.
I'm happy to do more tests if that helps.

- Chris


------------------------------------------------------------------------------
_______________________________________________
Opensync-devel mailing list
Opensync-devel@...
https://lists.sourceforge.net/lists/listinfo/opensync-devel

[Chris Frey-2]

On Mon, Mar 30, 2009 at 09:20:17AM +0200, Patrick Ohly wrote:
> Not quite: if the plugin does not support this change in libecal's copy
> if libical, then it will leak memory. libical will not fall back to
> using the ring buffer if HANDLE_LIBICAL_MEMORY is undefined. The only
> purpose of that define is to suppress the warning when the user of the
> library is doing the right thing.

I'm trying to find the correct way to get rid of this warning message.
Thanks for this explanation... unfortunately, the trail of libical
is a long and winding one. :-)


> This particular change in libical was not merged into upstream libical,
> which is what libecal 2.26 is using. Instead libical decided to not
> break the API of libical and added _r variants of the relevant
> functions. The traditional variants still use the ring buffer.

I downloaded evolution-data-server 2.26 from
http://ftp.gnome.org/pub/gnome/sources/evolution-data-server/2.26/
and I'm unable to find any libical headers.

In fact, I'm uanble to find LIBICAL_MEMFIXES, and the only mention of
HANDLE_LIBICAL_MEMORY is in the ChangeLog.  It would seem that libical
is no more, and libecal is its successor.


> For apps this means they *must* check LIBICAL_MEMFIXES before freeing
> some of these strings - or never free them and hope that it won't matter
> because a) the strings are small or b) the app is going to be used with
> a libical which doesn't need it.
>
> Using the _r variants if available is the best approach. The ring buffer
> has been a source of problems because of the indeterministic life time
> of the strings, so it is better to not depend on it.

In 2.26, it appears that LIBICAL_MEMFIXES is not defined, and the _r variants
are not exposed in any of the headers files, although some are used in
the .c files.

So.... it seems that the troublesome API is eventually disappearing completely,
and as far as I can tell, so are the _r variants.  I don't know if there is
a define flag that can determine whether _r variants exist or not.

In the meantime, it would seem that the simplest sane fix is to define
HANDLE_LIBICAL_MEMORY, and never free.

Since the evolution plugin doesn't appear to use any of the troublesome
functions, perhaps a change to the header just to avoid the warning is
valid.

What do you think of the patch below?  It's pretty ugly, but would work
for our purposes.

- Chris


diff --git a/src/evolution2_sync.h b/src/evolution2_sync.h
index 22deb72..34625a5 100644
--- a/src/evolution2_sync.h
+++ b/src/evolution2_sync.h
@@ -1,7 +1,26 @@
 #ifndef EVO2_SYNC_H
 #define EVO2_SYNC_H
 
-//#include "evo2_sync.h"
+//
+// Some versions of libical use a ring buffer for the following
+// functions, for which the library manages memory.  Somewhere along
+// the line, this was fixed so that the application was responsible
+// for freeing the strings returned by these functions.  A warning
+// was added to the header that would display if HANDLE_LIBICAL_MEMORY
+// was not defined.
+//
+// Even newer versions of evolution-data-server, which this plugin
+// depends on, get rid of some of these functions, while newer
+// versions of libical add _r variants that implement the
+// application-free functinality.
+//
+// Since this plugin does not use these functions, we disable the
+// warning by defining HANDLE_LIBICAL_MEMORY, then include the
+// headers, and then purposely break the troublesome functions
+// so that if a programmer tries to use them later on, he'll know
+// to handle them with care.
+//
+#define HANDLE_LIBICAL_MEMORY 1
 
 #include <opensync/opensync.h>
 
@@ -9,6 +28,18 @@
 #include <libebook/e-book.h>
 #include <libedataserver/e-data-server-util.h>
 
+#define icalreqstattype_as_string() See_evolution2_sync_h_for_note
+#define icalproperty_as_ical_string() See_evolution2_sync_h_for_note
+#define icalproperty_get_parameter_as_string() See_evolution2_sync_h_for_note
+#define icalproperty_get_value_as_string() See_evolution2_sync_h_for_note
+#define icallangbind_property_eval_string() See_evolution2_sync_h_for_note
+#define icalperiodtype_as_ical_string() See_evolution2_sync_h_for_note
+#define icaltime_as_ical_string() See_evolution2_sync_h_for_note
+#define icalvalue_as_ical_string() See_evolution2_sync_h_for_note
+#define icalcomponent_as_ical_string() See_evolution2_sync_h_for_note
+#define e_cal_component_get_recurid_as_string() See_evolution2_sync_h_for_note
+
+
 typedef struct evo2_location {
  char *name;
  char *uri;
diff --git a/tools/list_sources.c b/tools/list_sources.c
index fb25499..21f66fe 100644
--- a/tools/list_sources.c
+++ b/tools/list_sources.c
@@ -1,3 +1,5 @@
+// see note in ../src/evolution2_sync.h
+#define HANDLE_LIBICAL_MEMORY 1
 #include <glib.h>
 #include <libecal/e-cal.h>
 #include <libebook/e-book.h>
diff --git a/tools/test_uri.c b/tools/test_uri.c
index 476a6b0..1df2a75 100644
--- a/tools/test_uri.c
+++ b/tools/test_uri.c
@@ -1,3 +1,5 @@
+// see note in ../src/evolution2_sync.h
+#define HANDLE_LIBICAL_MEMORY 1
 #include <string.h>
 #include <glib.h>
 #include <libecal/e-cal.h>



------------------------------------------------------------------------------
This SF.net email is sponsored by:
High Quality Requirements in a Collaborative Environment.
Download a free trial of Rational Requirements Composer Now!
http://p.sf.net/sfu/www-ibm-com
_______________________________________________
Opensync-devel mailing list
Opensync-devel@...
https://lists.sourceforge.net/lists/listinfo/opensync-devel

[Patrick Ohly]

On Tue, 2009-04-07 at 16:08 -0400, Chris Frey wrote:

> On Mon, Mar 30, 2009 at 09:20:17AM +0200, Patrick Ohly wrote:
> > This particular change in libical was not merged into upstream libical,
> > which is what libecal 2.26 is using. Instead libical decided to not
> > break the API of libical and added _r variants of the relevant
> > functions. The traditional variants still use the ring buffer.
>
> I downloaded evolution-data-server 2.26 from
> http://ftp.gnome.org/pub/gnome/sources/evolution-data-server/2.26/
> and I'm unable to find any libical headers.
>
> In fact, I'm uanble to find LIBICAL_MEMFIXES, and the only mention of
> HANDLE_LIBICAL_MEMORY is in the ChangeLog.  It would seem that libical
> is no more, and libecal is its successor.

No, quite the opposite. libical is a stand-alone project now, with
active maintainers who merged the patches developed separately earlier
by the various teams who had copied libical.

The upstream project is here:
http://sourceforge.net/projects/freeassociation/

> > For apps this means they *must* check LIBICAL_MEMFIXES before freeing
> > some of these strings - or never free them and hope that it won't matter
> > because a) the strings are small or b) the app is going to be used with
> > a libical which doesn't need it.
> >
> > Using the _r variants if available is the best approach. The ring buffer
> > has been a source of problems because of the indeterministic life time
> > of the strings, so it is better to not depend on it.
>
> In 2.26, it appears that LIBICAL_MEMFIXES is not defined, and the _r variants
> are not exposed in any of the headers files, although some are used in
> the .c files.
>
> So.... it seems that the troublesome API is eventually disappearing completely,
> and as far as I can tell, so are the _r variants.  I don't know if there is
> a define flag that can determine whether _r variants exist or not.

Good question. If not, then some runtime compile + link check is needed
to determine whether they exist. I haven't added that kind of support
myself yet, therefore I haven't looked at how it was done in upstream
libical.

> In the meantime, it would seem that the simplest sane fix is to define
> HANDLE_LIBICAL_MEMORY, and never free.
>
> Since the evolution plugin doesn't appear to use any of the troublesome
> functions, perhaps a change to the header just to avoid the warning is
> valid.
>
> What do you think of the patch below?  It's pretty ugly, but would work
> for our purposes.

Yeah, looks like a good interim solution.

But I wonder why the plugin doesn't need these functions. I definetely
call some of them to import/export events and time zones in
SyncEvolution.

--
Bye, Patrick Ohly
--  
Patrick.Ohly@...
http://www.estamos.de/



------------------------------------------------------------------------------
This SF.net email is sponsored by:
High Quality Requirements in a Collaborative Environment.
Download a free trial of Rational Requirements Composer Now!
http://p.sf.net/sfu/www-ibm-com
_______________________________________________
Opensync-devel mailing list
Opensync-devel@...
https://lists.sourceforge.net/lists/listinfo/opensync-devel

[Chris Frey-2]

On Tue, Apr 07, 2009 at 10:49:27PM +0200, Patrick Ohly wrote:
> Yeah, looks like a good interim solution.

I'll apply it, thanks.


> But I wonder why the plugin doesn't need these functions. I definetely
> call some of them to import/export events and time zones in
> SyncEvolution.

The evolution2 plugin uses e_cal_get_component_as_string() and
e_vcard_to_string(), both of which are documented in the comments to
return a string that needs to be freed.

- Chris


------------------------------------------------------------------------------
This SF.net email is sponsored by:
High Quality Requirements in a Collaborative Environment.
Download a free trial of Rational Requirements Composer Now!
http://p.sf.net/sfu/www-ibm-com
_______________________________________________
Opensync-devel mailing list
Opensync-devel@...
https://lists.sourceforge.net/lists/listinfo/opensync-devel