[trunk] [API] dropped osync_plugin_info_get_configdir (#1083)

[Daniel Gollub-2]

Hi,

with r5378 [1] osync_plugin_info_get_configdir() got dropped. No replacement
for this function is planned. If your plugin is using this function, make sure
you already have ported to latest Hashtable API changes.

I'm not aware of any plugin which is already ported to latest trunk API and
still using osync_plugin_info_get_configdir() (expect SyncML plugin which
"fixed" already. IrMC plugin is not yet ported to latest trunk API - only
using this for old hashtable and anchor implementation).

The reason for dropping this function is to avoid that plugin keep storing
files in the plugin configuration directory. So plugins are "force" to use the
OpenSync API, which makes later maintenance and version updates easier. E.g.
lots of plugins used to store hashtables and anchors in different locations.
Or even implemented their own database on top of sqlite3. With the current API
redesigns there should be no plugin interface which requires direct file
access. This makes things also less depending on sqlite3 - and allows later
easily to move to a different database or change the group/member
implementation without breaking the plugin APU.

If you have any question regarding this change - let me know.

JFYI, if your plugin needs to store/cache specific data  (e.g. device
information which get reused on every sync, ...) we plan to provide a new
OSyncDatabase [2] interface which allows to store key=value. (Not planned for
0.39 nor 0.40 - since this implementation wouldn't break any API and can be
added later. If you rely on this interface - please give us feedback!)

[1] http://opensync.org/changeset/5378
[2] http://opensync.org/ticket/1050

Best Regards,
Daniel

------------------------------------------------------------------------------
_______________________________________________
Opensync-devel mailing list
Opensync-devel@...
https://lists.sourceforge.net/lists/listinfo/opensync-devel

[Chris Frey-2]

On Sat, Mar 28, 2009 at 04:10:50PM +0100, Daniel Gollub wrote:
> JFYI, if your plugin needs to store/cache specific data  (e.g. device
> information which get reused on every sync, ...) we plan to provide a new
> OSyncDatabase [2] interface which allows to store key=value. (Not planned for
> 0.39 nor 0.40 - since this implementation wouldn't break any API and can be
> added later. If you rely on this interface - please give us feedback!)

The Barry plugin relies on this function to store its cache and ID map.
These are just simple text files at the moment:  the cache is a list of
known record ID's in the Blackberry, and the idmap is a map of Blackberry
record ID's to opensync ID's (key/value).

Your above paragraph makes it sound like there is no key/value API
currently existing.  What is the best way to handle this?

Thanks,
- Chris


------------------------------------------------------------------------------
_______________________________________________
Opensync-devel mailing list
Opensync-devel@...
https://lists.sourceforge.net/lists/listinfo/opensync-devel

[Daniel Gollub-2]

On Wednesday 01 April 2009 01:52:16 am Chris Frey wrote:

> On Sat, Mar 28, 2009 at 04:10:50PM +0100, Daniel Gollub wrote:
> > JFYI, if your plugin needs to store/cache specific data  (e.g. device
> > information which get reused on every sync, ...) we plan to provide a new
> > OSyncDatabase [2] interface which allows to store key=value. (Not planned
> > for 0.39 nor 0.40 - since this implementation wouldn't break any API and
> > can be added later. If you rely on this interface - please give us
> > feedback!)
>
> The Barry plugin relies on this function to store its cache and ID map.
> These are just simple text files at the moment:  the cache is a list of
> known record ID's in the Blackberry, and the idmap is a map of Blackberry
> record ID's to opensync ID's (key/value).

Whats the purpose of those IDs?
By record you mean the entry/change IDs? Why not use the native Mapping Table
and just use the OSyncChange UID? Is it due to the problem that if you commit
new entries to the BB, the BB creates it's own ID and not using the original
UID reported by OpenSync? If so - there is a way to "workaround" this -
without the need to have your own mapping table. Commit your change - retrieve
the assigned new ID from the BB, and update the OSyncChange with BB's mapping
ID. (But yeah - wild guess - i should check your code ;))

We have this OSyncStateDB (format OSyncAnchor) which is since yesterday also a
key=value interface. In case those are independent of changes/entries.

>
> Your above paragraph makes it sound like there is no key/value API
> currently existing.  What is the best way to handle this?

Yes - this interface still needs to be written. To not break the API the (old)
OSyncDB got remove from the public API. The idea is to reintroduce [2] this
and add such interfaces. Since we add interface and not change or delete we
can do this after 0.39.


[2] http://opensync.org/ticket/1050

Best Regards,
Daniel

------------------------------------------------------------------------------
_______________________________________________
Opensync-devel mailing list
Opensync-devel@...
https://lists.sourceforge.net/lists/listinfo/opensync-devel

[Chris Frey-2]

On Wed, Apr 01, 2009 at 02:02:50AM +0200, Daniel Gollub wrote:
> > The Barry plugin relies on this function to store its cache and ID map.
> > These are just simple text files at the moment:  the cache is a list of
> > known record ID's in the Blackberry, and the idmap is a map of Blackberry
> > record ID's to opensync ID's (key/value).
>
> Whats the purpose of those IDs?
> By record you mean the entry/change IDs?

The record IDs are just numbers that the Blackberry uses to reference
a contact or a calendar item.  They are not guaranteed to be unique, but
they almost always are, and Barry is careful to maintain that
pseudo-uniqueness.


> Why not use the native Mapping Table and just use the OSyncChange UID?
> Is it due to the problem that if you commit
> new entries to the BB, the BB creates it's own ID and not using the original
> UID reported by OpenSync? If so - there is a way to "workaround" this -
> without the need to have your own mapping table. Commit your change - retrieve
> the assigned new ID from the BB, and update the OSyncChange with BB's mapping
> ID. (But yeah - wild guess - i should check your code ;))

That sounds promising.  A lot of this code is just ported from the 0.22
plugin, where I either did things simply for implementation speed, or
just didn't realize the API existed.  I decided back then to keep BB and
opensync ID's strictly separate.

I'll do some more research on OSyncChange UID.  I was under the impression
that opensync could handle alphanumeric ID's, while the BB can only do numeric.

As for adding new records via Barry, we generate the ID, so that's not a
problem.  I was more concerned with preserving the existing ID's from
BB-added records.

Time to study the API some more! :-)



> We have this OSyncStateDB (format OSyncAnchor) which is since yesterday also a
> key=value interface. In case those are independent of changes/entries.

Once an ID is mapped, it stays mapped until the record is deleted.  So if I
understand correctly, this StateDB would work for me.


Let me do some more research before I waste too much of your time. :-)

- Chris


------------------------------------------------------------------------------
_______________________________________________
Opensync-devel mailing list
Opensync-devel@...
https://lists.sourceforge.net/lists/listinfo/opensync-devel

[Daniel Gollub-2]

On Wednesday 01 April 2009 02:22:39 am Chris Frey wrote:

> On Wed, Apr 01, 2009 at 02:02:50AM +0200, Daniel Gollub wrote:
> > > The Barry plugin relies on this function to store its cache and ID map.
> > > These are just simple text files at the moment:  the cache is a list of
> > > known record ID's in the Blackberry, and the idmap is a map of
> > > Blackberry record ID's to opensync ID's (key/value).
> >
> > Whats the purpose of those IDs?
> > By record you mean the entry/change IDs?
>
> The record IDs are just numbers that the Blackberry uses to reference
> a contact or a calendar item.  They are not guaranteed to be unique, but
> they almost always are, and Barry is careful to maintain that
> pseudo-uniqueness.

Hehe - interesting. I had a quick look on barry. The ID things is really crazy
it's defenitly not a UID thing. Does the 1st party software create multiple
records with the same ID? ;)

>
> > Why not use the native Mapping Table and just use the OSyncChange UID?
> > Is it due to the problem that if you commit
> > new entries to the BB, the BB creates it's own ID and not using the
> > original UID reported by OpenSync? If so - there is a way to "workaround"
> > this - without the need to have your own mapping table. Commit your
> > change - retrieve the assigned new ID from the BB, and update the
> > OSyncChange with BB's mapping ID. (But yeah - wild guess - i should check
> > your code ;))
>
> That sounds promising.  A lot of this code is just ported from the 0.22
> plugin, where I either did things simply for implementation speed, or
> just didn't realize the API existed.  I decided back then to keep BB and
> opensync ID's strictly separate.
>
> I'll do some more research on OSyncChange UID.  I was under the impression
> that opensync could handle alphanumeric ID's, while the BB can only do
> numeric.

No - it's just a "string" (const char*) - if you overwrite/change the in your
commit function thats perfectly fine for OpenSync. It will be only changed for
the member which is using your plugin - not for the other plugins. So the
"UID" thing of OSyncChange is not really _unique_ internally. For this we have
"mappings" and "mapping ids" which are stored in  the ..... "mapping table" ;)



dgollub@marvin:~> sqlite3 ~/.opensync/group1/archive.db .dump tbl_changes
BEGIN TRANSACTION;
CREATE TABLE tbl_changes (objtype VARCHAR(64) NOT NULL, id INTEGER PRIMARY KEY
AUTOINCREMENT, uid VARCHAR NOT NULL, memberid INTEGER NOT NULL, mappingid
INTEGER NOT NULL, objengine VARCHAR(64) NOT NULL );
INSERT INTO "tbl_changes" VALUES('contact',17,'4',2,1,'contact');
INSERT INTO "tbl_changes" VALUES('contact',18,'7',2,2,'contact');
INSERT INTO "tbl_changes" VALUES('contact',19,'5',2,3,'contact');
INSERT INTO "tbl_changes" VALUES('contact',25,'foo3',1,1,'contact');
INSERT INTO "tbl_changes" VALUES('contact',26,'foo2',1,2,'contact');
INSERT INTO "tbl_changes" VALUES('contact',27,'foo1',1,3,'contact');

(Hmmm... the naming is not really consisting - the mapping table get
persistent stored with OSyncMappingTable which is using also OSyncArchive and
ends up in  a table "tbl_changes" - maybe some cleanup after 0.39 is required
;))

>
> As for adding new records via Barry, we generate the ID, so that's not a
> problem.  I was more concerned with preserving the existing ID's from
> BB-added records.
>
> Time to study the API some more! :-)
>
> > We have this OSyncStateDB (format OSyncAnchor) which is since yesterday
> > also a key=value interface. In case those are independent of
> > changes/entries.
>
> Once an ID is mapped, it stays mapped until the record is deleted.  So if I
> understand correctly, this StateDB would work for me.

Actually the StateDB is only for "device states". The mapping table is
supposed to be used for stuff like that. Your plugin doesn't need to deal with
the  mapping table directly.

Just delete all your mapping database code and call osync_change_set_uid() in
your plugin commit function when the changetype is ADDED. Once you have
genearted your new uniqe ID. Thats it - after the commit plugin function the
engine would look into this OSyncChagne again and stores your changed UID
isntead of the orignally reported.



>
>
> Let me do some more research before I waste too much of your time. :-)


Sure.

Best Regards,
Daniel



------------------------------------------------------------------------------
_______________________________________________
Opensync-devel mailing list
Opensync-devel@...
https://lists.sourceforge.net/lists/listinfo/opensync-devel