Re: [RFC] Plugin specifc capabilities II

On Wednesday 01 April 2009 01:52:49 pm Patrick Ohly wrote:
[...]
> > But currently i'm not seeing how this could be done without dataloss. I
> > revisit this again - maybe i missed something...
> >
> > Do you have any idea how merge/demerge in the "native" format would be
> > possible without dataloss?
> > Do you have any other better proposal?
>
> Drop the separate merger/demerger concept. Move that functionality into
> the format conversion.

I looked into that very very close and actually fully implemented this
approach in OpenSync, vformat plugin and the xmlformat plugin. Now i have some
doubts that this approach has only benefits to the current draft in the wiki
... there are also some downsides:

> When converting towards OpenSync, the format plugin works with an item
> in the "foreign", limited format, a capabilities description that
> describes the format used by the peer, and a complete item in the
> internal format. The result is an updated internal item with information
> from the peer mixed in, or a complete copy of the incoming item when
> there is no internal item yet.

I don't see the reason why this couldn't be done with separating the merge and
conversion function. When i tried to implement/port the format plugins to your
proposal i realized that i would just have to call the already implemented
XMLFormat merge code inside the conversion function - or just do the function
call of it at the very end of the conversion.

Simplified example:

osync_bool convert_a_to_b(char **entry_data, unsigned int *entry_size,
                    const char *cached_data, unsigned int cached_size,
                    OSyncCapabilities *caps, const char *config, OSyncError**)
{

  conversion_routine(entry_data, entry_size);
  merge_routine(entry_data, entry_size, cached_data, cached_size, caps);
  return TRUE;
}

Since merge_routine() would always deal with data from the internal format
this would be redundant code - since the merge routine would be
copied/reimplemented in every format plugin which provides format conversion
towards this particular internal format.

If we would keep it separated this would look internally something like this:
(again very simplified)

void incoming_item_from_plugin(OSyncEngine *engine, OSyncChange *change)
{
  OSyncConverter *converter;
  OSyncObjFormat *a = osync_change_get_objformat(change);
  OSyncObjFormat *b = osync_format_env_find_objformat("common");

  converter = osync_format_env_find_converter(engine->formatenv, a, b);


  /* Conversion to forgein/peer format to common format */
  converter->conv_func(change->data, change->size);
 
  /* Get the cached item (which is in the common format already) */
  char *cached_data;
  unsigned int cached_size;
  osync_archive_load_item(engine->archive, &cached_data, &cached_size);

  /* Merge - based on common format data and caps */
  b->merge_func(change->data, change->size, caps, cached_data, cached_size);

  [.... report it to the engine for mapping ...]
}

The advantage here would be that only the "common format" plugin would need to
implement a merge function. (Which is what we have currently only inside the
"xmlformat" plugin with the current approach)

But i see your point with passing along some further capabilities information
to influence the conversion. Like SyncML DevInf UTC or so. As i stated earlier
mails we could have done this with the "const char *config" parameter. But
yeah there are some arguments against it  ...


> When sending towards a peer, the format plugin only needs to pick
> properties from the internal item according to the peers capabilities.
> "Only read what you need" looks more natural to me than the current
> concept of "strip an item, then convert the stripped down version".

Actually this could be done also with separating merger from conversion
function - but completely drop the "demerge" function. So there would be no
demerge call in advance when the format plugin's conversion function get
called.

I guess the initial idea of the "demerge" plugin was to make the conversion
function very simple and generic. The current conversion function doesn't have
to deal with capabilities and just convert everything what they are able to
convert.

> The
> additional advantage is that this conversion has full access to all
> available information, which may be an advantage when going beyond a
> static capabilities-based conversion towards a scripted one which also
> uses dynamic configuration.

Right - that's a real advantage i can't argue against (at least not yet ;)).

If we would keep the "merge" (not demerge!) function separated from conversion
the function - and just drop entirely the "demerge" function from the concept
- we could also have full access and go beyond static caps conversion.

We would just need to pass the OSyncCapabilities* to the conversion function
argument list - additionally. So the conversion functions to and from the
internal/common format have access to device specific information.

Thoughts? Comments?

Best Regards,
Daniel

--
Daniel Gollub                        Geschaeftsfuehrer: Ralph Dehner
FOSS Developer                       Unternehmenssitz:  Vohburg
B1 Systems GmbH                      Amtsgericht:       Ingolstadt
Mobil: +49-(0)-160 47 73 970         Handelsregister:   HRB 3537
EMail: gollub@...          http://www.b1-systems.de

Adresse: B1 Systems GmbH, Osterfeldstraße 7, 85088 Vohburg
http://pgpkeys.pca.dfn.de/pks/lookup?op=get&search=0xED14B95C2F8CA78D


------------------------------------------------------------------------------

_______________________________________________
Opensync-devel mailing list
Opensync-devel@...
https://lists.sourceforge.net/lists/listinfo/opensync-devel