Ubuntu UDS-Boston, Syncing solutions

[DoctorMO]

Hi everyone,

I've pushed the syncing engine and support issue into the UDS as I'm
here all week; I want to talk seriously about how we can start to
integrate conduit and opensync in a coherent set of functionality for
the ubuntu desktop.

I'm sure many of you will be familiar with my emails in the past, I am
unhappy about the current dispersed and non integrated fashion with
which this functionality is being developed. but I want to talk about
it in the context of ubuntu.

So if you have any issues you want me to bring up; or your attending
and would like to take part. please do reply and register on the
blueprint as required.

Best Regards, Martin Owens

-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems?  Stop.
Now Search log events and configuration files using AJAX and a browser.
Download your FREE copy of Splunk now >> http://get.splunk.com/
_______________________________________________
Opensync-devel mailing list
Opensync-devel@...
https://lists.sourceforge.net/lists/listinfo/opensync-devel

[John Stowers]

Martin,

I'm sure John Carr will reply to this mail also. I can only speak for
my own intentions and as such I have generally been doing the
following with my development time (in relation to Conduit + Opensync)

1) Trying to work on areas that opensync does not explicitly target
(for example, not a complete list obviously)
    --> GNOME online desktop type foo
    --> Provide a great GUI
    --> Focus more on online services
    --> Focus on a attractive file sync use case (not rsync level, but
workable for hundreds of files at a time)
2) I am also trying to actively make our Conduit core more
integratable with opensync plugins
    --> Support both hashes and mtimes
    --> Be smart with object comparison and rebuilding relationships
    --> Remove all gnome/gtk dependencies in core
    --> Reduce memory consumption
    --> Etc

I think John Carr has lists similar to above. Although Conduit has not
made its position wrt opensync publically obvious, I think we all
agree that blatantly racing to implement the same features helps few.
I was perhaps guilty of this when I added evolution support using our
own python evolution bindings, but in my defense I think its negligent
that evolution/gnome didnt provide bindings for eds already.

Furthur to that the following is a list of areas that I have avoided
development work on so far. This is both because of time issues, and
because I wanted to agree on how to procede together first.
   --> How can we use HAL to encapsulate the sync capabilities of a
device. Such as have fields in FDI files that describe a mobilie
device as having "Notes", "Contact", etc data to sync
   --> I have thought long and hard about making the command line
tools syncml-obex-client and syncml-obex-server into dbus daemons .
The FDIs in this case could also encode the quirks and specific syncml
config attributes for the phone, if connected over USB (see the
opensuse wiki for a list of these). At this point I would (1)
communicate with the daemon from conduit over DBus. I would also (2)
rewrite the opensync syncml plugin(s) to communicate with these
daemons over dbus. Obviously I have not yet considered how much work
(2) is, and even if its possible. While this still has some code
duplication, its at least an effective middle ground (through
libsyncml). I would appreciate your thoughts on this.

Anyway, thats a bit of food for discussion off the top of my head. Is
there some way I can participate in the Ubuntu discussion remotely?

John Stowers


On 10/31/07, Martin Owens <doctormo@...> wrote:

    Hi everyone,

    I've pushed the syncing engine and support issue into the UDS as I'm
    here all week; I want to talk seriously about how we can start to
    integrate conduit and opensync in a coherent set of functionality for
    the ubuntu desktop.

    I'm sure many of you will be familiar with my emails in the past, I am
    unhappy about the current dispersed and non integrated fashion with
    which this functionality is being developed. but I want to talk about
    it in the context of ubuntu.

    So if you have any issues you want me to bring up; or your attending
    and would like to take part. please do reply and register on the
    blueprint as required.

    Best Regards, Martin Owens
    _______________________________________________
    Conduit-list mailing list
    Conduit-list@...
    http://mail.gnome.org/mailman/listinfo/conduit-list




On 10/31/07, Martin Owens <doctormo@...> wrote:

> Hi everyone,
>
> I've pushed the syncing engine and support issue into the UDS as I'm
> here all week; I want to talk seriously about how we can start to
> integrate conduit and opensync in a coherent set of functionality for
> the ubuntu desktop.
>
> I'm sure many of you will be familiar with my emails in the past, I am
> unhappy about the current dispersed and non integrated fashion with
> which this functionality is being developed. but I want to talk about
> it in the context of ubuntu.
>
> So if you have any issues you want me to bring up; or your attending
> and would like to take part. please do reply and register on the
> blueprint as required.
>
> Best Regards, Martin Owens
>
> -------------------------------------------------------------------------
> This SF.net email is sponsored by: Splunk Inc.
> Still grepping through log files to find problems?  Stop.
> Now Search log events and configuration files using AJAX and a browser.
> Download your FREE copy of Splunk now >> http://get.splunk.com/
> _______________________________________________
> Opensync-devel mailing list
> Opensync-devel@...
> https://lists.sourceforge.net/lists/listinfo/opensync-devel
>

-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems?  Stop.
Now Search log events and configuration files using AJAX and a browser.
Download your FREE copy of Splunk now >> http://get.splunk.com/
_______________________________________________
Opensync-devel mailing list
Opensync-devel@...
https://lists.sourceforge.net/lists/listinfo/opensync-devel

[DoctorMO]

> duplication, its at least an effective middle ground (through
> libsyncml). I would appreciate your thoughts on this.

I will digest your thoughts over sleep and hopefully bring up all the
issues at hand;

> Anyway, thats a bit of food for discussion off the top of my head. Is
> there some way I can participate in the Ubuntu discussion remotely?

There is a one way audio ogg stream from each room (or so I have been
told) and we could use irc to 'hear' your thoughts back.

Best Regards, Martin Owens

-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems?  Stop.
Now Search log events and configuration files using AJAX and a browser.
Download your FREE copy of Splunk now >> http://get.splunk.com/
_______________________________________________
Opensync-devel mailing list
Opensync-devel@...
https://lists.sourceforge.net/lists/listinfo/opensync-devel

[John Stowers]

On 10/31/07, Martin Owens <doctormo@...> wrote:
> > duplication, its at least an effective middle ground (through
> > libsyncml). I would appreciate your thoughts on this.
>
> I will digest your thoughts over sleep and hopefully bring up all the
> issues at hand;

Cool, thanks. Obviously im not in a position to direct the development
of all parties, these examples just came to mind first.

My experience with Conduit has at least shown, and I hope the opensyc
devs agree, that much more of the work goes into supporting devices
(or dataproviders in conduit lingo) than core sync logic.

Following from that we already have the ability to interact/host
opensync plugins in conduit, and with current changes in SVN, we
should be able to demonstrate syncing conduit with opensync plugins in
the next two weeks (Jc2k - thoughts?)

>
> > Anyway, thats a bit of food for discussion off the top of my head. Is
> > there some way I can participate in the Ubuntu discussion remotely?
>
> There is a one way audio ogg stream from each room (or so I have been
> told) and we could use irc to 'hear' your thoughts back.

OK I will definately do this. It would be best if a video conference
thing was availble, but this will do

When is the meeting?

John Stowers
>
> Best Regards, Martin Owens
>

-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems?  Stop.
Now Search log events and configuration files using AJAX and a browser.
Download your FREE copy of Splunk now >> http://get.splunk.com/
_______________________________________________
Opensync-devel mailing list
Opensync-devel@...
https://lists.sourceforge.net/lists/listinfo/opensync-devel

[John Carr-3]

5.12 AM here. Have just risen and am somewhat groggy so I apologise if
my thoughts are a little.. messy.

On Wed, 2007-10-31 at 17:35 +1300, John Stowers wrote:

> On 10/31/07, Martin Owens <doctormo@...> wrote:
> > > duplication, its at least an effective middle ground (through
> > > libsyncml). I would appreciate your thoughts on this.
> >
> > I will digest your thoughts over sleep and hopefully bring up all the
> > issues at hand;
>
> Cool, thanks. Obviously im not in a position to direct the development
> of all parties, these examples just came to mind first.
>
> My experience with Conduit has at least shown, and I hope the opensyc
> devs agree, that much more of the work goes into supporting devices
> (or dataproviders in conduit lingo) than core sync logic.
>
> Following from that we already have the ability to interact/host
> opensync plugins in conduit, and with current changes in SVN, we
> should be able to demonstrate syncing conduit with opensync plugins in
> the next two weeks (Jc2k - thoughts?)

The stuff i'm landing to enable gconf sync also fixes one of the
problems I had with this work. Then I just have to write a thin shim for
each opensync plugin as i can't figure out how to dynamically probe
opensync in a way that makes sense for the UIs we want to support. I
hope the shims are something I can ditch, but thankfully most of the
logic is encapsulated in the main OpenSync dataprovider - the shims
should be tiny.

Moving forward from there, the refactoring work John S has done has
brought us closer to the point where we could sync with different
engines and our underlying platform is a lot less GNOME dependent -
It's possible to run conduit entirely without GTK as a dbus daemon and
then connect a UI to it over dbus and be able to create the same kind of
UI as well as "new device" wizards.

>
> >
> > > Anyway, thats a bit of food for discussion off the top of my head. Is
> > > there some way I can participate in the Ubuntu discussion remotely?
> >
> > There is a one way audio ogg stream from each room (or so I have been
> > told) and we could use irc to 'hear' your thoughts back.
>
> OK I will definately do this. It would be best if a video conference
> thing was availble, but this will do
>
> When is the meeting?
>
> John Stowers
> >
> > Best Regards, Martin Owens

I think I have more to add, but need to get ready for work. I'll post
again in a bit.

John


-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems?  Stop.
Now Search log events and configuration files using AJAX and a browser.
Download your FREE copy of Splunk now >> http://get.splunk.com/
_______________________________________________
Opensync-devel mailing list
Opensync-devel@...
https://lists.sourceforge.net/lists/listinfo/opensync-devel

[Daniel Gollub]

On Wednesday 31 October 2007, Martin Owens wrote:
> I've pushed the syncing engine and support issue into the UDS as I'm
> here all week; I want to talk seriously about how we can start to
> integrate conduit and opensync in a coherent set of functionality for
> the ubuntu desktop.
>
> I'm sure many of you will be familiar with my emails in the past, I am
> unhappy about the current dispersed and non integrated fashion with
> which this functionality is being developed. but I want to talk about
> it in the context of ubuntu.

No, I'm not - could you point to your mails about those topics?

>
> So if you have any issues you want me to bring up; or your attending
> and would like to take part. please do reply and register on the
> blueprint as required.

Is there any plan to support the development of OpenSync by providing some
people to actually develop on OpenSync Core and the OpenSync core plugins or
even on libsyncml? Currently I only hear that everyone is planning to use
OpenSync, but everyone just wait until OpenSync is "ready". I don't want to
sound rude ... but the development ressource of OpenSync are very limited and
could need some more people which help on hacking on OpenSync and solve
complex tasks. I already spent 110% of my spare time on OpenSync development,
but thats seems to be not enough... and no! OpenSync is not part of my
regular work.

Not quite sure how you involved into Ubuntu, but could you fix Ubuntu Gusty
and upgrade to libsyncml 0.4.2 at least? libsyncml 0.4.1 is broken and year
old - and currently i get overloaded with user request from Ubuntu Gusty
users which can't sync because Gusty got shipped with libnsymcl 0.4.1. If
version update isn't acceptable - could you at least apply:

http://libsyncml.opensync.org/changset/204

best regards,
Daniel

-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems?  Stop.
Now Search log events and configuration files using AJAX and a browser.
Download your FREE copy of Splunk now >> http://get.splunk.com/
_______________________________________________
Opensync-devel mailing list
Opensync-devel@...
https://lists.sourceforge.net/lists/listinfo/opensync-devel

[John Carr-3]

> > I'm sure many of you will be familiar with my emails in the past, I am
> > unhappy about the current dispersed and non integrated fashion with
> > which this functionality is being developed. but I want to talk about
> > it in the context of ubuntu.
>
> No, I'm not - could you point to your mails about those topics?

Hi Daniel. I guess he means the emails we discussed in #opensync. They
are available here as "Hardy: Time for sync project focus...":

https://lists.ubuntu.com/archives/ubuntu-devel-discuss/2007-September/thread.html

> > So if you have any issues you want me to bring up; or your attending
> > and would like to take part. please do reply and register on the
> > blueprint as required.
>
> Is there any plan to support the development of OpenSync by providing some
> people to actually develop on OpenSync Core and the OpenSync core plugins or
> even on libsyncml? Currently I only hear that everyone is planning to use
> OpenSync, but everyone just wait until OpenSync is "ready". I don't want to
> sound rude ... but the development ressource of OpenSync are very limited and
> could need some more people which help on hacking on OpenSync and solve
> complex tasks. I already spent 110% of my spare time on OpenSync development,
> but thats seems to be not enough... and no! OpenSync is not part of my
> regular work.
>
> Not quite sure how you involved into Ubuntu, but could you fix Ubuntu Gusty
> and upgrade to libsyncml 0.4.2 at least? libsyncml 0.4.1 is broken and year
> old - and currently i get overloaded with user request from Ubuntu Gusty
> users which can't sync because Gusty got shipped with libnsymcl 0.4.1. If
> version update isn't acceptable - could you at least apply:
>
> http://libsyncml.opensync.org/changset/204
>
> best regards,
> Daniel

-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems?  Stop.
Now Search log events and configuration files using AJAX and a browser.
Download your FREE copy of Splunk now >> http://get.splunk.com/
_______________________________________________
Opensync-devel mailing list
Opensync-devel@...
https://lists.sourceforge.net/lists/listinfo/opensync-devel

[Daniel Gollub]

On Wednesday 31 October 2007, John Stowers wrote:
> I'm sure John Carr will reply to this mail also. I can only speak for
> my own intentions and as such I have generally been doing the
> following with my development time (in relation to Conduit + Opensync)
>
> 1) Trying to work on areas that opensync does not explicitly target
> (for example, not a complete list obviously)
>     --> GNOME online desktop type foo
>     --> Provide a great GUI
Agree, OpenSync stays desktop and platform independent....

>     --> Focus more on online services
We might conflict here - since we would provide our own set of OpenSync
plugins. Currently we limited our self to PIM data .. since OpenSync is
awesome for syncing content-related data in little size - like PIM data ;)

Example: google-calendar plugin (Could need more love - we need support for
the latest GCal API features, like batch processing and the calendar
ressource API)

>     --> Focus on a attractive file sync use case (not rsync level, but
> workable for hundreds of files at a time)

Hmm, this might conflict with csync ... I'm in contact with the author of
csync. He evaluated OpenSync before i started with csync implementation.
Actually it is his diploma thesis, so the quality should be quite high. It's
like OpenSync+rsync - it doesn't touch the content like OpenSync could to
converter between different formats, it supports bidirectional
synchronization, rsync only supports uni-directional which isn't acceptable
for a default-desktop-user.

> 2) I am also trying to actively make our Conduit core more
> integratable with opensync plugins
>     --> Support both hashes and mtimes
>     --> Be smart with object comparison and rebuilding relationships
>     --> Remove all gnome/gtk dependencies in core
>     --> Reduce memory consumption
>     --> Etc

This sound like completely hijacking the OpenSync synchronization logic :)
Or maybe i missed something...

I know about the idea of J2ck to have a solution to provide different
synchronization logic - not quite sure about that yet if this should really
done in this way. Since we could waste a lot of time with double effort ...
and for sure currently conduit can compare with the synchronization logic of
OpenSync - actually I'm not aware of any powerful synchronization logic like
the one OpenSync has - even the propertaries ones.

>
> I think John Carr has lists similar to above. Although Conduit has not
> made its position wrt opensync publically obvious, I think we all
> agree that blatantly racing to implement the same features helps few.
Agree...

> I was perhaps guilty of this when I added evolution support using our
> own python evolution bindings, but in my defense I think its negligent
> that evolution/gnome didnt provide bindings for eds already.
We currently suffering also with the evolution-data-server and it's interface
as well .. but not because of python bindings. We have to run the evo2-sync
plugin in a separated process.

>
> Furthur to that the following is a list of areas that I have avoided
> development work on so far. This is both because of time issues, and
> because I wanted to agree on how to procede together first.

>    --> How can we use HAL to encapsulate the sync capabilities of a
> device. Such as have fields in FDI files that describe a mobilie
> device as having "Notes", "Contact", etc data to sync
That would be indeed very intersting. But i hope you agree that we don't want
to create tons of separated FDI files to list all the device capabilities.
Only because of the  reason it's easy...

We should kindly ask the HAL developers to add support for certain protocols
and probe them if a device got attached. Examples:

- HAL request and provide x-obex/capabilities information from OBEX interfaces
(not only list obex interfaces...). With those capabilities we could not only
provide  sync capabilities for SyncML and IrMC ... also provide information
about OBEX FTP.

- Palm HotSync interface - in HAL are currently ugly FDI files to point to the
correct usb serial interface for HotSync. Since pilot-link 0.12.x it support
transport via libusb and there is no need to "guess" the usb serial
interface. The libusb transport code of pilot-link handles this in a more
sane way... lots of pilot-link frontends like gnome-pilot and kpilot afaik
suffer on this. gnome-pilot for example listen for HAL and if the FDI
property of this ugly FDI file got set it tries to start a sync. Even if the
palm device doesn't is in "hotsync mode". Newer palm models even appear on
USB even if their not syncing. But the disappear and appear again with in one
second if the cradle got pushed.

- Not quite sure about all the Windows Mobile devices.. i guess J2ck is more
familiar with that...

Regarding Bluetooth on Linux the BlueZ developers did a really nice job with
the BlueZ D-Bus API. I missed the last BlueZ Developer meeting but there are
still plans to get BlueZ integrated in HAL. Anyway so far I'm quite satisfied
with the current BlueZ API. You can get everything what you need with the
from the SDP records. It's even possible to do some periodic device
discovery.

>    --> I have thought long and hard about making the command line
> tools syncml-obex-client and syncml-obex-server into dbus daemons .

I fully disagree ;)
There is something called HAL...
It's just about x-obex/capabilties - you should have a look at the OBEX spec.

> The FDIs in this case could also encode the quirks and specific syncml
> config attributes for the phone, if connected over USB (see the
> opensuse wiki for a list of these).

I heard of quirks and i really don't like quirks. Especially i don't like
quirks if it's possible to avoid quirks if the device protocoll fits all your
needs ...

> At this point I would (1)
> communicate with the daemon from conduit over DBus. I would also (2)
> rewrite the opensync syncml plugin(s) to communicate with these
> daemons over dbus. Obviously I have not yet considered how much work
> (2) is, and even if its possible. While this still has some code
> duplication, its at least an effective middle ground (through
> libsyncml). I would appreciate your thoughts on this.
Sorry, this is really not needed. You should have a look on the OBEX Spec.
The commandline tool obexftp has a simple reference implementation how to
request the x-obex/capabilties. "obexftp -X"

If  the HAL developers don't fully disagree with my idea of implementation
there is no need to have yet another system daemon.

But i see your point in have same process/daemon running which takes care  all
the time about triggering a sync or reporting there is a new and unknown
device which could be synced.

I already had an idea about a (D-Bus) session daemon (which is desktop
independent), which could handle mobile devices related stuff. Like
triggering a sync if your local PIM environment added a new bunch of new
contact entries... so periodic sync like every 5 Minutes is not needed. I
would suggested to only do syncs if the device appear and is after 5 Minutes
still available. I called this "MobileStation" - anyway thats quite offtopic
now ;)

best regards,
Daniel

-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems?  Stop.
Now Search log events and configuration files using AJAX and a browser.
Download your FREE copy of Splunk now >> http://get.splunk.com/
_______________________________________________
Opensync-devel mailing list
Opensync-devel@...
https://lists.sourceforge.net/lists/listinfo/opensync-devel

[John Carr-3]

Hi Martin

It has been a while :)

On Tue, 2007-10-30 at 23:45 -0400, Martin Owens wrote:
> Hi everyone,
>
> I've pushed the syncing engine and support issue into the UDS as I'm
> here all week; I want to talk seriously about how we can start to
> integrate conduit and opensync in a coherent set of functionality for
> the ubuntu desktop.

I think the primary goal of your spec was PIM<->device sync. So these
are some issues to think about. Note that i'm ignoring bluetooth for
now, i'm also ignoring the work Conduit is doing on making GNOME apps
more sync friendly, media sync use cases and playing nice with "Online
Desktop".

This stuff is sync platform neutral and is needed before the user even
thinks about how the heck to make their data sync.

Windows Mobile support.
-----------------------
I'm involved in the SynCE project that targets these devices so am able
to help here somewhat. Here are some thoughts.

We need help packaging the components in a "desktop integrated" way -
for example, to make Windows Mobile support "just rock" then plugging in
a device needs to start 3 programs - odccm, sync-engine and DTPT.py.

I'm sure dbus activation is possible for sync-engine, but that doesnt
handle making sure it goes away when you disconnect. I don't want it
hanging around needlessly.

The other two need to start in response to me plugging in a device and
go away when i unplug the device.

For WM5 and WM6, we will need to patch the kernel. Most of the patches
are upstream and there are some more that can go upstream. This is
proving difficult though. Samsung processors are a big problem and while
I have a patch that solves it in some cases, but its the spawn of satan
and needs love from a more experienced kernel hacker. Note that network
manager will disconnect you from WIFI when you want to sync your phone.
Annoying.

Shipping this driver will allow WM6 people to access the internet using
there device as a modem. It couldnt really be any more plug and play
than it is right now - network manager plays OK in this case.

Among the current active developers there is hardly any WM2003
knowledge. I have a 2003 device and hope to change that. This will
require some way of automatically bringing up a PPP interface in
addition to some or all of the above apps.

SyncML support.
---------------
I've only investigated this in the context of a P900. Again, we need
some way to automatically establish the ppp connection when a device is
plugged in. IP needs to be set and NetworkManager needs to not
interfere.

At that point I think the opensync plugin has everything it needs.

> I'm sure many of you will be familiar with my emails in the past, I am
> unhappy about the current dispersed and non integrated fashion with
> which this functionality is being developed. but I want to talk about
> it in the context of ubuntu.

It's not that dispersed :) I'm a SynCE dev (and it only targets
opensync) and a Conduit dev. I chat with dgollub quite a lot on
#opensync and often try to help out users there (though generally they
are SynCE users who got a bit lost).

As for integrated, thats how I see conduit. The integrator. But I see
Daniel has already replied so I will post my thoughts on that there :)

> So if you have any issues you want me to bring up; or your attending
> and would like to take part. please do reply and register on the
> blueprint as required.

Have you got a URL to hand? I'm obviously going to have a look when i
can (busy @ work right now) but for others tracking discussions it would
be handy.

John


-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems?  Stop.
Now Search log events and configuration files using AJAX and a browser.
Download your FREE copy of Splunk now >> http://get.splunk.com/
_______________________________________________
Opensync-devel mailing list
Opensync-devel@...
https://lists.sourceforge.net/lists/listinfo/opensync-devel

[Pawel Kot-4]

Hi,

On 10/31/07, Daniel Gollub <dgollub@...> wrote:
> On Wednesday 31 October 2007, John Stowers wrote:
> > --> How can we use HAL to encapsulate the sync capabilities of a
> > device. Such as have fields in FDI files that describe a mobilie
> > device as having "Notes", "Contact", etc data to sync
> That would be indeed very intersting. But i hope you agree that we don't want
> to create tons of separated FDI files to list all the device capabilities.
> Only because of the  reason it's easy...

I'd like to put few words in here. Similiar information is extremly
important for all apps of phone manager category (regardless if it is
based on gnokii, gammu or some other libgsm). In phone managers we
would need much more detailed information to know which capabilities
are available to the user. I am already thinking at this topic, so if
there are already some other thoughts made, I'd be happy to discuss
(and implement!) them.

take care,
pkot
--
Pawel Kot

-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems?  Stop.
Now Search log events and configuration files using AJAX and a browser.
Download your FREE copy of Splunk now >> http://get.splunk.com/
_______________________________________________
Opensync-devel mailing list
Opensync-devel@...
https://lists.sourceforge.net/lists/listinfo/opensync-devel

[Michael Banck-3]

Hi,

On Tue, Oct 30, 2007 at 11:45:46PM -0400, Martin Owens wrote:
> I've pushed the syncing engine and support issue into the UDS as I'm
> here all week;

Who else from Ubuntu, Canonical or others is there who is interested in
this and will (along with you, I assume) invest time to
implement/integrate this?  Is Matthew Garrett still working on
bluetooth<->HAL integration, and will this be integrated with your
efforts?

> I am unhappy about the current dispersed and non integrated fashion
> with which this functionality is being developed. but I want to talk
> about it in the context of ubuntu.

I had a look at the UDS schedule some days ago, but couldn't find
anything about syncing.

> So if you have any issues you want me to bring up; or your attending
> and would like to take part. please do reply and register on the
> blueprint as required.

As others pointed out, it might help to point people at specific
resources; I assume most people in here never heard of "blueprint"
(besides, telling people to register on a proprietary service as a
requirement to take place in a discussion sounds slightly rude to me)
for example.

So where is the spec, where is the wiki, when is the discussion and
where can we follow?


thanks,

Michael

-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems?  Stop.
Now Search log events and configuration files using AJAX and a browser.
Download your FREE copy of Splunk now >> http://get.splunk.com/
_______________________________________________
Opensync-devel mailing list
Opensync-devel@...
https://lists.sourceforge.net/lists/listinfo/opensync-devel

[John Carr-3]

Hello again

The problem I see is that there are two IMO great sync implementations.
OpenSync has some really nice features for the PIM sync case
(capabilities) and because of the care that has been taken with design
and scope its low level and portable. Conduit has some really nice
features for media sync, like dynamic conversion to the correct video or
audio format, and nice (though could be nicer!) HAL stuff. And it
exports interfaces over dbus, so "You have connected X" type wizards are
really straightforward to deliver.

Right now opensync especially, but conduit as well, are established
enough that they are not going to abandon the work they've done, but
also don't want to fight and duplicate work. I think we can agree that
opensync is lowlevel. And that conduit is quite.. immoral.. with its
desires to take over your desktop :)

> >     --> Focus more on online services
> We might conflict here - since we would provide our own set of OpenSync
> plugins. Currently we limited our self to PIM data .. since OpenSync is
> awesome for syncing content-related data in little size - like PIM data ;)
>
> Example: google-calendar plugin (Could need more love - we need support for
> the latest GCal API features, like batch processing and the calendar
> ressource API)

A big chunk of our dataproviders currently target getting photos from
F-spot, EOG and Cheese online to Flickr, SmugMug, Box.net and others. We
also do file sync to box.net and hopefully S3.

Our goal is that we become a blessed dependency of GNOME and get f-spot
to use us for exports instead of its own plugins. We are also
implementing f-spot code to let us put data back in. We also want to get
to the point where tomboy switches to conduit instead of its own sync
engine. We implemented the dbus interface for this. I think we already
more than match their supported sync targets.

I think the fact that we are putting code back into those projects (that
a pure opensync project could benefit from too) is a good example of us
being integrated and "good" players in this game.

> >     --> Focus on a attractive file sync use case (not rsync level, but
> > workable for hundreds of files at a time)
>
> Hmm, this might conflict with csync ... I'm in contact with the author of
> csync. He evaluated OpenSync before i started with csync implementation.
> Actually it is his diploma thesis, so the quality should be quite high. It's
> like OpenSync+rsync - it doesn't touch the content like OpenSync could to
> converter between different formats, it supports bidirectional
> synchronization, rsync only supports uni-directional which isn't acceptable
> for a default-desktop-user.

I'm not sure how the implementations stack up as csync is quite low
level and the last time i saw it was tied to samba(?). The work John S
has done here is at a GnomeVFS level so can sync to quite a few
different places, and to sites like S3 and Box.net. Writing a KioSlave
DP would still allow syncing to box.net and S3, obviously.

This could also pokes my AutoSync code in all the right places so will
sync files as they change (with an optional delay to stop spamming).

>
> > 2) I am also trying to actively make our Conduit core more
> > integratable with opensync plugins
> >     --> Support both hashes and mtimes
> >     --> Be smart with object comparison and rebuilding relationships
> >     --> Remove all gnome/gtk dependencies in core
> >     --> Reduce memory consumption
> >     --> Etc
>
> This sound like completely hijacking the OpenSync synchronization logic :)
> Or maybe i missed something...

Integrating with opensync plugins is phase 1, and very mean :-) The hash
refactoring allows us to use the opensync dataproviders as though they
were native to conduit. Only in theory, i've only tested it with the
Evo2 plugin! Unfortunately i can't escape needing to shim the opensync
plugins yet :(

The object comparison/rebuilding relationships stuff does/will take a
different approach to opensync. In this case, python will allow rapid
prototyping of something you might want to backport into opensync :) But
John S is still working on the finer points in his head.

> I know about the idea of J2ck to have a solution to provide different
> synchronization logic - not quite sure about that yet if this should really
> done in this way. Since we could waste a lot of time with double effort ...
> and for sure currently conduit can compare with the synchronization logic of
> OpenSync - actually I'm not aware of any powerful synchronization logic like
> the one OpenSync has - even the propertaries ones.

My thoughts are that the code of conduit is reasonably well split up and
that if in the same way Conduit can use OpenSync dataproviders it should
work in reverse. I can proxy calls by opensync onto the conduit
dataproviders so opensync can use our plugins. This can then be
controlled via conduits dbus interface.

Putting conduit's dbus in charge is important as it allows GUI's to
respond to HAL, avahi and NetworkManager changes without re-inventing
any wheels. It also means that opensync, conduit, csync, and other(?)
partnerships can be managed from one interface. And a considerable
amount of logic can be shared between Kubuntu/Ubuntu GUIs (I presume a
GTK gui would be out of the question Martin? What about Xubuntu?).
Finally, mulitple clients can be active at once. For example, an
application can manage syncing internally.. This ties in to work for
online desktop where Conduit is used to build an app connected to a
database with an online backend that it keeps in sync with. There are
much better uses cases, but i'm already bordering on a stupid length
e-mail.

The main motivation to keep conduit sync logic around is that its a lot
more agile than opensync right now. We are not sure how easily some
things would be to implement in opensync, or if they would be accepted.
Arguably we should push to opensync, and i'd agree. But right now its a
time where we can't afford to lose those features / face massive
regressions. Allow me to explain:

The GNOME Online Desktop people are implementing sync left right and
centre. They are very much for tomboy having its own sync and recently
brought us gconf-sync-daemon. If we can't do tomboy sync and gconf sync
AND have AutoSync that matches theirs then we don't have any hope of
even holding a conversation about removing yet more duplication from the
free desktop.

There are numerous other sync projects popping up. Bright side: At least
opensync and conduit are doing something to work together, eh? We need
to focus on getting sync out of the box in all distros (and Conduit has
contacts on Foresight and at least one other distro to that end as well
as trying to build platform glue in to the GNOME platform). Otherwise
we'll meet the downside. It will be too late and GNOME will be full of
mini-sync engines and you'll have to install x different sync clients
and involve services like scheduleworld to make everything work.

> >
> > Furthur to that the following is a list of areas that I have avoided
> > development work on so far. This is both because of time issues, and
> > because I wanted to agree on how to procede together first.
>
> >    --> How can we use HAL to encapsulate the sync capabilities of a
> > device. Such as have fields in FDI files that describe a mobilie
> > device as having "Notes", "Contact", etc data to sync
> That would be indeed very intersting. But i hope you agree that we don't want
> to create tons of separated FDI files to list all the device capabilities.
> Only because of the  reason it's easy...
>
> We should kindly ask the HAL developers to add support for certain protocols
> and probe them if a device got attached. Examples:

<snip>

I think it would be crazy to ask HAL to speak anything to do with
Windows Mobile. Anyway, this is something we can do ourselves and i had
planned for Windows Mobile. I believe HAL supports callout binaries that
can populate HAL with more info. So in the WM case, I will be populating
all the the info that can be scraped by libsynce into the HAL tree.

I think it's more appropriate to have a bunch of FDI files that say
which plugin is needed to "speak" to a given device. This is what I have
been experimenting with in the SynCE case and in Conduit.

> > At this point I would (1)
> > communicate with the daemon from conduit over DBus. I would also (2)
> > rewrite the opensync syncml plugin(s) to communicate with these
> > daemons over dbus. Obviously I have not yet considered how much work
> > (2) is, and even if its possible. While this still has some code
> > duplication, its at least an effective middle ground (through
> > libsyncml). I would appreciate your thoughts on this.
> Sorry, this is really not needed. You should have a look on the OBEX Spec.
> The commandline tool obexftp has a simple reference implementation how to
> request the x-obex/capabilties. "obexftp -X"
>
> If  the HAL developers don't fully disagree with my idea of implementation
> there is no need to have yet another system daemon.
> But i see your point in have same process/daemon running which takes care  all
> the time about triggering a sync or reporting there is a new and unknown
> device which could be synced.

I don't like the idea of another daemon, and if HAL can be populated
with HAL callouts then I don't think there is a need for this. I would
rather have python-libsyncml than another daemon.

> I already had an idea about a (D-Bus) session daemon (which is desktop
> independent), which could handle mobile devices related stuff. Like
> triggering a sync if your local PIM environment added a new bunch of new
> contact entries... so periodic sync like every 5 Minutes is not needed. I
> would suggested to only do syncs if the device appear and is after 5 Minutes
> still available. I called this "MobileStation" - anyway thats quite offtopic
> now ;)

I'd encourage you to work with Conduit to realise your version of
MobileStation. If we split our source code how I see it your
MobileStation is one of the pieces that is left, but with a much more
interesting scope. It is used for desktop-to-desktop sync, responding to
internet availability and integrating with GNOME Online Desktop rather
than just mobile devices.

Perhaps this is something for Sync Fest? There are other things I would
like to discuss there, including a convergence of our plugin API.
Basically, the current API doesn't really cater for non-sync cases. But
I have a few cases where I want to uses conduit dataproviders directly.

Moving forward from UDS, I'm really interested in trying to get
interested parties together at Linux Conf AU as both myself and John S
will be there. I'm also pretty available within the confines of the UK.

John


-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems?  Stop.
Now Search log events and configuration files using AJAX and a browser.
Download your FREE copy of Splunk now >> http://get.splunk.com/
_______________________________________________
Opensync-devel mailing list
Opensync-devel@...
https://lists.sourceforge.net/lists/listinfo/opensync-devel

[John Carr-3]

On Wed, 2007-10-31 at 15:03 +0100, Michael Banck wrote:

> Hi,
>
> On Tue, Oct 30, 2007 at 11:45:46PM -0400, Martin Owens wrote:
> > I've pushed the syncing engine and support issue into the UDS as I'm
> > here all week;
>
> Who else from Ubuntu, Canonical or others is there who is interested in
> this and will (along with you, I assume) invest time to
> implement/integrate this?  Is Matthew Garrett still working on
> bluetooth<->HAL integration, and will this be integrated with your
> efforts?

You can expect a lot of effort from me on this as both a SynCE guy
(windows mobile support), and as a core dev of something that would,
IMHO, provide a lot of in terms of desktop integration.

>
> > I am unhappy about the current dispersed and non integrated fashion
> > with which this functionality is being developed. but I want to talk
> > about it in the context of ubuntu.
>
> I had a look at the UDS schedule some days ago, but couldn't find
> anything about syncing.

John S pointed me to this photo of a whiteboard. But I too failed to
find anything about it online prior to this :(

http://arstechnica.com/journals/linux.ars/2007/10/27/shuttleworth-at-fosscamp-collaboration-brings-brilliant-flashes-of-innovation

>
> > So if you have any issues you want me to bring up; or your attending
> > and would like to take part. please do reply and register on the
> > blueprint as required.
>
> As others pointed out, it might help to point people at specific
> resources; I assume most people in here never heard of "blueprint"
> (besides, telling people to register on a proprietary service as a
> requirement to take place in a discussion sounds slightly rude to me)
> for example.
>
> So where is the spec, where is the wiki, when is the discussion and
> where can we follow?

These URLs might be handy. Some initial discussion about the plans.
https://lists.ubuntu.com/archives/ubuntu-devel/2007-September/024194.html

The main spec is at
https://wiki.ubuntu.com/PimSyncPlan

But there are lots of related specs. No joke, its like everyone who
can't code and is on launchpad made a sync related blueprint.
 

>
>
> thanks,
>
> Michael
>
> -------------------------------------------------------------------------
> This SF.net email is sponsored by: Splunk Inc.
> Still grepping through log files to find problems?  Stop.
> Now Search log events and configuration files using AJAX and a browser.
> Download your FREE copy of Splunk now >> http://get.splunk.com/
> _______________________________________________
> Opensync-devel mailing list
> Opensync-devel@...
> https://lists.sourceforge.net/lists/listinfo/opensync-devel

-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems?  Stop.
Now Search log events and configuration files using AJAX and a browser.
Download your FREE copy of Splunk now >> http://get.splunk.com/
_______________________________________________
Opensync-devel mailing list
Opensync-devel@...
https://lists.sourceforge.net/lists/listinfo/opensync-devel

[DoctorMO]

> > Who else from Ubuntu, Canonical or others is there who is interested in
> > this and will (along with you, I assume) invest time to
> > implement/integrate this?  Is Matthew Garrett still working on
> > bluetooth<->HAL integration, and will this be integrated with your
> > efforts?

This is the blueprint I'm pushing:

https://blueprints.launchpad.net/ubuntu/+spec/syncintegration

I'm one of those dogs making wiki pages for sync:

https://wiki.ubuntu.com/PimSyncPlan

I think I see some great things here, John you've been thinking about
these problems a great deal I can tell and I'm honestly very impressed
by your solutions and compromises.

There are a few things we need to do, all hardware support should run
via HAL, that will require methods to get unique id's and information
from fdi files for callbacks and other information; it's important to
get opensync using HAL as much as it is to get Conduit using the same
information services. in this way both systems can utilise the same
hardware syncing plugins (eventually)

Amoungst other things that has been put in this thread, I'm going to
talk about tying conduit into a light python based service/gui which
sits on the users tray and notifies them when hardware is plugged in
that is a new phone/pda or and existing one that you've synced before;
making it easy to initiate any of the syncs without loading conduit
gui and using the large gui for the configuration aspects. I think
John and I have already talked about this easy to use top layer.

I'll be talking with various people about it and will post times as
soon as I know.

Best Regards, Martin Owens

-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems?  Stop.
Now Search log events and configuration files using AJAX and a browser.
Download your FREE copy of Splunk now >> http://get.splunk.com/
_______________________________________________
Opensync-devel mailing list
Opensync-devel@...
https://lists.sourceforge.net/lists/listinfo/opensync-devel

[John Stowers]

>
> > --> I have thought long and hard about making the command line
> > tools syncml-obex-client and syncml-obex-server into dbus daemons .
>
> I fully disagree ;)
> There is something called HAL...
> It's just about x-obex/capabilties - you should have a look at the OBEX spec.

see below

>
> > The FDIs in this case could also encode the quirks and specific syncml
> > config attributes for the phone, if connected over USB (see the
> > opensuse wiki for a list of these).
>
> I heard of quirks and i really don't like quirks. Especially i don't like
> quirks if it's possible to avoid quirks if the device protocoll fits all your
> needs ...
>
> > At this point I would (1)
> > communicate with the daemon from conduit over DBus. I would also (2)
> > rewrite the opensync syncml plugin(s) to communicate with these
> > daemons over dbus. Obviously I have not yet considered how much work
> > (2) is, and even if its possible. While this still has some code
> > duplication, its at least an effective middle ground (through
> > libsyncml). I would appreciate your thoughts on this.
> Sorry, this is really not needed. You should have a look on the OBEX Spec.
> The commandline tool obexftp has a simple reference implementation how to
> request the x-obex/capabilties. "obexftp -X"
>
> If  the HAL developers don't fully disagree with my idea of implementation
> there is no need to have yet another system daemon.

See below

>
> But i see your point in have same process/daemon running which takes care  all
> the time about triggering a sync or reporting there is a new and unknown
> device which could be synced.
>
> I already had an idea about a (D-Bus) session daemon (which is desktop
> independent), which could handle mobile devices related stuff. Like
> triggering a sync if your local PIM environment added a new bunch of new
> contact entries... so periodic sync like every 5 Minutes is not needed. I
> would suggested to only do syncs if the device appear and is after 5 Minutes
> still available. I called this "MobileStation" - anyway thats quite offtopic
> now ;)

I actually think we are in some sort of agreement.

I did not suggest that the syncml-obex-client should run all the time,
being yet another system daemon. My point was that putting all of the
policy regarding device sync into a daemon is (IMHO) a good idea. The
daemon+callout+HAL could manage almost everything. We could even
autostart said daemon if/when bluez detects/pairs with a nearby mobile
phone

As John suggested a hal callout that queries the obex capabilities of
the device could then start the appropriate daemon in response to the
device cabilbities sounds interesting.

Regards,

John

-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems?  Stop.
Now Search log events and configuration files using AJAX and a browser.
Download your FREE copy of Splunk now >> http://get.splunk.com/
_______________________________________________
Opensync-devel mailing list
Opensync-devel@...
https://lists.sourceforge.net/lists/listinfo/opensync-devel

[John Stowers]

> I don't like the idea of another daemon, and if HAL can be populated
> with HAL callouts then I don't think there is a need for this. I would
> rather have python-libsyncml than another daemon.

The last time I looked at doing this it seemed very complicated - due
to the main loop stuff in libsyncml. Its not an insurmountable
technical problem but IIRC each gtk app only allows one main loop per
process, so the bootstrap import code for libsyncml would need to
create a new main context and attach this to the relevant syncml data
structures

In addition the thought of binding a large C library to python without
using gobject and the semi-automated binding generators makes my skin
crawl :-)

I would be tempted to write a gobject binding for the libs main
objects first. Then bind that - however if that is the case then Its
an example of a bunch of code that is not really going to be shared
between the two projects, which is something I would like to avoid.

John

-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems?  Stop.
Now Search log events and configuration files using AJAX and a browser.
Download your FREE copy of Splunk now >> http://get.splunk.com/
_______________________________________________
Opensync-devel mailing list
Opensync-devel@...
https://lists.sourceforge.net/lists/listinfo/opensync-devel

[John Carr-3]

I think for now its more sane (and least effort) for us just to use the
opensync plugin... If I can get my P900 all set up then theres no reason
why this wouldnt be viable. And write a HAL callout in C or Vala to poke
and probe for capabilities.

On Thu, 2007-11-01 at 08:44 +1300, John Stowers wrote:

> > I don't like the idea of another daemon, and if HAL can be populated
> > with HAL callouts then I don't think there is a need for this. I would
> > rather have python-libsyncml than another daemon.
>
> The last time I looked at doing this it seemed very complicated - due
> to the main loop stuff in libsyncml. Its not an insurmountable
> technical problem but IIRC each gtk app only allows one main loop per
> process, so the bootstrap import code for libsyncml would need to
> create a new main context and attach this to the relevant syncml data
> structures
>
> In addition the thought of binding a large C library to python without
> using gobject and the semi-automated binding generators makes my skin
> crawl :-)
>
> I would be tempted to write a gobject binding for the libs main
> objects first. Then bind that - however if that is the case then Its
> an example of a bunch of code that is not really going to be shared
> between the two projects, which is something I would like to avoid.
>
> John

-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems?  Stop.
Now Search log events and configuration files using AJAX and a browser.
Download your FREE copy of Splunk now >> http://get.splunk.com/
_______________________________________________
Opensync-devel mailing list
Opensync-devel@...
https://lists.sourceforge.net/lists/listinfo/opensync-devel

[John Stowers]

On 11/1/07, John Carr <john.carr@...> wrote:
> I think for now its more sane (and least effort) for us just to use the
> opensync plugin... If I can get my P900 all set up then theres no reason
> why this wouldnt be viable. And write a HAL callout in C or Vala to poke
> and probe for capabilities.
>

I think as a first pass, and as john mentioned, we are very close to
hosting opensync plugins in conduit.

You may also be interested in my posts in this thread
(http://mail.gnome.org/archives/desktop-devel-list/2007-October/msg00238.html)
where we discuss conduit, online-desktop, and synchronization

John

-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems?  Stop.
Now Search log events and configuration files using AJAX and a browser.
Download your FREE copy of Splunk now >> http://get.splunk.com/
_______________________________________________
Opensync-devel mailing list
Opensync-devel@...
https://lists.sourceforge.net/lists/listinfo/opensync-devel

[Michal Čihař]

Hi

On Wed, 31 Oct 2007 12:37:22 +0100
"Pawel Kot" <pawel.kot@...> wrote:

> I'd like to put few words in here. Similiar information is extremly
> important for all apps of phone manager category (regardless if it is
> based on gnokii, gammu or some other libgsm). In phone managers we
> would need much more detailed information to know which capabilities
> are available to the user. I am already thinking at this topic, so if
> there are already some other thoughts made, I'd be happy to discuss
> (and implement!) them.

Well there is lot of information which would be interesting to know.
Starting on what fields and how much of them does phone support ending
with available options to connect to it. First would be quite useful
for all applications, while the latter for low level configuration of
device (if we can not configure it automatically, user just selects
phone from list and he will get list of possibilities how to connect to
it). I thought many times about starting such database, but I never got
to make it real.

--
        Michal Čihař | http://cihar.com | http://blog.cihar.com


-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems?  Stop.
Now Search log events and configuration files using AJAX and a browser.
Download your FREE copy of Splunk now >> http://get.splunk.com/
_______________________________________________
Opensync-devel mailing list
Opensync-devel@...
https://lists.sourceforge.net/lists/listinfo/opensync-devel

[DoctorMO]

> Well there is lot of information which would be interesting to know.
> Starting on what fields and how much of them does phone support ending
> with available options to connect to it. First would be quite useful
> for all applications, while the latter for low level configuration of
> device (if we can not configure it automatically, user just selects
> phone from list and he will get list of possibilities how to connect to
> it). I thought many times about starting such database, but I never got
> to make it real.

Hardware databases are being developed by other projects, including
but not limited to dohickey, ubuntu-hardware-tests and it looks like
these will be merged with more info being fed back into HAL as
required.

So a quick overview:

Conduit GUI for configuration
New python gui for sync applet icon using dbus to conduit services
OpenSync wedged somehow into HAL for all hardware endpoints
All information about phones to be put into HAL after callouts from
HAL this includes unique ID's and the sync engine required to operate
Conduit and opensync both changed to take advantage or awareness of
HAL if compiled to require it.
Callouts should return the user which the phone has been claimed to
belong and allow later on to extend this to any security on mobile
devices for syncing. (or none if it's brand new phone)
The default pim end points for ubuntu is evolution (begrudgingly) and
kubuntu is kde-pim for one click sync enable.

Let me know if I've missed any high level thoughts.

Best Regards, Martin Owens

-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems?  Stop.
Now Search log events and configuration files using AJAX and a browser.
Download your FREE copy of Splunk now >> http://get.splunk.com/
_______________________________________________
Opensync-devel mailing list
Opensync-devel@...
https://lists.sourceforge.net/lists/listinfo/opensync-devel