« Return to Thread: [RFC] Remove pendingLimit from OSyncQueue
Re: [RFC] Remove pendingLimit from OSyncQueue
by Daniel Gollub-3
::
Rate this Message:
On Thursday 09 April 2009 10:28:05 am Michael Bell wrote:
> Hi,
>
> I would like to remove pendingLimit from OSyncQueue to fix the dead lock
> of ticket #1078. The pendingLimit of the OSyncQueue and the not fully
> independent messages can produced dead locks. Today the queue already
> causes deadlocks.
>
> What happens?
>
> The OSyncQueue stops dispatching if pendingLimit number of messages
> waiting for an answer. This is correct IPC behaviour if the messages are
> independent but this is not correct in case of OpenSync.
>
> Some protocols (e.g. SyncML) can only flush once for an object type
> (SyncML datastore). This means changes are collected and send if the
> maximum message size of the protocol is reached or all OpenSync messages
> are present. This means the only guaranteed send operation is done if
> the OpenSync message committed_all is handled.
>
> A first workaround was to commit all changes immediately and abort the
> complete synchronization if an error happens by signalling the error to
> the committed_all context. This does not work because the mapping (of
> SyncML) is potentially sent after the changes are received. OpenSync can
> handle mappings only for not committed changes.
That's wrong. See my mail to your batch_commit RFC thread.
> So the decision is simply to add a new mechanism for mapping IDs or to
> remove the pendingLimit which creates a dead lock between two originally
> independent queues.
I don't see the need of a new mapping ID mechanism. We Just need to adapt when
the UID could get updated by the plugin. Right now this can only happen in the
"commit (change) context" - which i thought would be perfectly fine.
If you write in SyncML an entry - when do you get the peers mapping id of the
just changed entry?
I wonder if it's really impossible to update the change UID before reply the
"commit (change) context". (Independent of the pendingLimit issue!)
Maybe it would help to introduce a new osync_context_ interface instead.
We change the definition of osync_context_report_success() within the commit
context. Instead of (additionally) "updating" the UID of a change within the
this commit context - this functions just ACKs that the commit got
handled/scheduled within the plugin. Internally this would just frees the
pendingQeue - no commit report to the Engine - no signalling to the OpenSync
frontend to a write event.
An additional osync_context_ interface could later report the changed UID
after write. For that the plugin would need to increase the ref of the
OSyncContext* and unref it once the change or error got reported with an
osync_context_ interface. We actually could reuse here
osync_context_update_change() which is used also in get_changes().
A plugin which is using commit sink funciton in async way must register a
committed_all() sink function - to signal when all changes got finally
committed.
It's similar to your osync_mapping_alter_interface() proposal with the
difference that there would be only internal changes - no public changes.
Actually my longterm goal is that plugins only use the osync_context_
interface to report changes to the engine ;)
Would this solve the pendingLimit issue?
Best Regards,
Daniel
--
Daniel Gollub Geschaeftsfuehrer: Ralph Dehner
FOSS Developer Unternehmenssitz: Vohburg
B1 Systems GmbH Amtsgericht: Ingolstadt
Mobil: +49-(0)-160 47 73 970 Handelsregister: HRB 3537
EMail: gollub@... http://www.b1-systems.de
Adresse: B1 Systems GmbH, Osterfeldstraße 7, 85088 Vohburg
http://pgpkeys.pca.dfn.de/pks/lookup?op=get&search=0xED14B95C2F8CA78D
------------------------------------------------------------------------------
This SF.net email is sponsored by:
High Quality Requirements in a Collaborative Environment.
Download a free trial of Rational Requirements Composer Now!
http://p.sf.net/sfu/www-ibm-com
_______________________________________________
Opensync-devel mailing list
Opensync-devel@...
https://lists.sourceforge.net/lists/listinfo/opensync-devel
> Hi,
>
> I would like to remove pendingLimit from OSyncQueue to fix the dead lock
> of ticket #1078. The pendingLimit of the OSyncQueue and the not fully
> independent messages can produced dead locks. Today the queue already
> causes deadlocks.
>
> What happens?
>
> The OSyncQueue stops dispatching if pendingLimit number of messages
> waiting for an answer. This is correct IPC behaviour if the messages are
> independent but this is not correct in case of OpenSync.
>
> Some protocols (e.g. SyncML) can only flush once for an object type
> (SyncML datastore). This means changes are collected and send if the
> maximum message size of the protocol is reached or all OpenSync messages
> are present. This means the only guaranteed send operation is done if
> the OpenSync message committed_all is handled.
>
> A first workaround was to commit all changes immediately and abort the
> complete synchronization if an error happens by signalling the error to
> the committed_all context. This does not work because the mapping (of
> SyncML) is potentially sent after the changes are received. OpenSync can
> handle mappings only for not committed changes.
That's wrong. See my mail to your batch_commit RFC thread.
> So the decision is simply to add a new mechanism for mapping IDs or to
> remove the pendingLimit which creates a dead lock between two originally
> independent queues.
I don't see the need of a new mapping ID mechanism. We Just need to adapt when
the UID could get updated by the plugin. Right now this can only happen in the
"commit (change) context" - which i thought would be perfectly fine.
If you write in SyncML an entry - when do you get the peers mapping id of the
just changed entry?
I wonder if it's really impossible to update the change UID before reply the
"commit (change) context". (Independent of the pendingLimit issue!)
Maybe it would help to introduce a new osync_context_ interface instead.
We change the definition of osync_context_report_success() within the commit
context. Instead of (additionally) "updating" the UID of a change within the
this commit context - this functions just ACKs that the commit got
handled/scheduled within the plugin. Internally this would just frees the
pendingQeue - no commit report to the Engine - no signalling to the OpenSync
frontend to a write event.
An additional osync_context_ interface could later report the changed UID
after write. For that the plugin would need to increase the ref of the
OSyncContext* and unref it once the change or error got reported with an
osync_context_ interface. We actually could reuse here
osync_context_update_change() which is used also in get_changes().
A plugin which is using commit sink funciton in async way must register a
committed_all() sink function - to signal when all changes got finally
committed.
It's similar to your osync_mapping_alter_interface() proposal with the
difference that there would be only internal changes - no public changes.
Actually my longterm goal is that plugins only use the osync_context_
interface to report changes to the engine ;)
Would this solve the pendingLimit issue?
Best Regards,
Daniel
--
Daniel Gollub Geschaeftsfuehrer: Ralph Dehner
FOSS Developer Unternehmenssitz: Vohburg
B1 Systems GmbH Amtsgericht: Ingolstadt
Mobil: +49-(0)-160 47 73 970 Handelsregister: HRB 3537
EMail: gollub@... http://www.b1-systems.de
Adresse: B1 Systems GmbH, Osterfeldstraße 7, 85088 Vohburg
http://pgpkeys.pca.dfn.de/pks/lookup?op=get&search=0xED14B95C2F8CA78D
------------------------------------------------------------------------------
This SF.net email is sponsored by:
High Quality Requirements in a Collaborative Environment.
Download a free trial of Rational Requirements Composer Now!
http://p.sf.net/sfu/www-ibm-com
_______________________________________________
Opensync-devel mailing list
Opensync-devel@...
https://lists.sourceforge.net/lists/listinfo/opensync-devel
« Return to Thread: [RFC] Remove pendingLimit from OSyncQueue
| Free embeddable forum powered by Nabble | Forum Help |
