GDBus/GVariant plans for next GLib release

[David Zeuthen]

Hey,

At the GNOME summit this weekend, Ryan Lortie, Matthias Clasen, Will
Thompson and myself had a hallway conversation about how to get D-Bus
functionality into the GLib stack. This mail is an attempt at
summarizing what we talked about.

(Unfortunately the scheduled talk on the summit for GLib/D-Bus was on
Sunday where neither Matthias or I was available. Not ideal, but since
we reached consensus we asked Ryan to run that session and solicit
feedback.)

There was an amazing amount of consensus; here's a brief list

 - We want some level of usable D-Bus binding in the GLib stack for
   the next release. We also agreed to avoid doing any C specific
   type/object mapping as part of the first push.

 - Everyone would like to use GVariant as the message encapsulation
   data type (as opposed to e.g. DBusMessage or some wrapper around
   DBusMessage).

 - We want convenience/utility for the "hard" parts of D-Bus, e.g.
    - mainloop integration
    - async name owning
    - async name watching
    - async listening to signals
    - exporting objects

 - We want to use the GIO async patterns, e.g. GCancellable,
   GAsyncReadyCallback, GAsyncInitable. E.g. everything should
   be async and we should deter people from using sync calls
   by suffixing the _sync to the method name.

 - Ideally make insane things impossible to do

   - e.g. only allow proxy creation for unique bus names - not
     well-known names. Instead people will discover they should
     be using e.g. g_bus_watch_name().

 - Strong consensus about using libdbus-1 for stability/portability
   reasons. And also, so a single process can nicely interoperate with
   D-Bus bindings for other languages (e.g. shared object tree)

 - Hide the fact that we are using libdbus-1

 - Make the code available in libgio (perhaps with the libdbus-1 using
   implementation in e.g. GVfs through GIOExtensionPoint - maybe not
   worth the effort)

 - Ensure the provided code is thread-safe / can be used from threads
   - e.g. deliver replies/signals/method-invocation in the thread where
     the method was called from, signal was subcribed to etc.
   - this may^Wwill include fixing libdbus-1 bugs

Since the GDBus stuff I've been working on already does this most of
this, thanks to Havoc's laundry list

 http://lists.freedesktop.org/archives/dbus/2009-February/010988.html

in particular, we agreed that I should a) make it use GVariant; b) rip
out the C object mapping. The past few days, I've been working on this
and the work is now nearing completion.

See

 http://people.freedesktop.org/~david/gdbus-20091014/
 http://cgit.freedesktop.org/~david/gdbus-standalone/

for the current API and code. The best way to see how the code works is
to read the API docs and look at the examples

http://cgit.freedesktop.org/~david/gdbus-standalone/plain/gdbus/example-own-name.c
http://cgit.freedesktop.org/~david/gdbus-standalone/plain/gdbus/example-watch-name.c
http://cgit.freedesktop.org/~david/gdbus-standalone/plain/gdbus/example-watch-proxy.c
http://cgit.freedesktop.org/~david/gdbus-standalone/plain/gdbus/example-server.c

and also the test cases

 http://cgit.freedesktop.org/~david/gdbus-standalone/tree/gdbus/tests

This API isn't yet complete - the main outstanding items include

 - D-Bus peer connections

   - The obvious thing here is to do a GDBusServer (wrapping DBusServer)
     class and add a constructor to GDBusConnection that takes a D-Bus
     address.

   - It might be nicer to just pass a GIOStream and use the new
     socket functionality in GIO. This requires some (perhaps trivial)
     libdbus-1 changes. We can always add this and we probably
     want the former *anyway*.

 - Misc convenience API

   - It might be nice to have methods on GDBusProxy to invoke
     methods. Might be nice to have also have varargs methods on
     GDBusProxy that use the GTypeSerializer functionality of
     GVariant.

   - Various obvious stuff (e.g. g_bus_watch_name_on_connection())

There's also a couple of bugs in libdbus-1, mostly related to threading,
that needs fixing. I will file bugs and get them fixed.

Some notes about GDBus in general

 - The API and code is minimal on purpose and has an extensive
   test-suite (12K LOC, three new classes, 3.5K LOC tests).

   I believe, even with the current code, that nice C type mappings
   (like those generated by EggDBus, e.g. see
   http://people.freedesktop.org/~david/eggdbus-20091014/eggdbus-eggdbusbus.html
   http://people.freedesktop.org/~david/eggdbus-20091014/tests-testfrob.html
   for example (yes, this is generated code with comments)) are
   possible to do.

   The API does not expose a GDBusMessage type since that would cause
   unwanted API that would assist in using the API as a whole in a
   wrong way (e.g. exposing on-wire details like message headers
   can be nasty). Instead, the API is based on invoking/handling methods
   and receiving/emitting signals.

   While it might be possible that people use D-Bus in ways this API
   doesn't cover, I think it is the right decision to cut the cake
   this way.

 - Some people have mentioned the possibility to port e.g. dbus-glib
   to run on top of this code. I am opposed to that idea for at least
   two reasons

   - first of all, that would mean exposing libdbus-1 and adding other
     API that might be unwanted. While this could be done in a non-invasive
     manner I don't think it's worth the effort.

   - similarly, we didn't port gnome-vfs2 to sit on top of GIO. I might
     be comparing apples and oranges here... but don't underestimate how
     quickly people ported apps to GIO.

   That said, if we can add/tweak API (within reason) to make the
   transition easier for people I'm all for it. But it should not be
   our main goal.

 - I decided to represent wire-errors (e.g. DBusError) using GError
   since that type is widely used both in C and language bindings. There
   are facilities for recovering the D-Bus error name. We also support
   transparent passing of any GError (a feature used in e.g. GVfs).

 - The libdbus-1 on Win32 / OS X story. Gah.

   So either we a) restrict this to UNIX (put headers in gio-unix-2.0)
   or; b) we just tell people they need to use e.g. WinDBus on Win32,
   D-Bus from the Fink ports on OS X or whatever.

   Anyway, I handwavely suggest the latter and also just getting
   mainline libdbus-1 to work on Win32 and OS X. Havoc says it's a week
   worth of work or something. It might be more, it might be less, I
   don't know.

   (Right now I'm tempted just to do the libdbus-1/Win32 myself...)

Some notes on GVariant

 - GVariant is very pleasant to use

 - GVariant needs to be LGPL2 (I believe Ryan has fixed this already)

 - GVariant doesn't use en_US spelling. I think this is a deal-breaker,
   it needs to read e.g. serialize instead of serialise. Anyway, I'm
   da_DK myself but...

 - GVariant is a superset of a subset of the D-Bus protocol (!)

   1. GVariant is a superset because it handles e.g. 'maybe' types

   2. GVariant doesn't yet handle the new 'h' type (e.g. unix fd)

   Now, 1. has been discussed extensively on this list. I don't
   have any position on it, really. I guess it would be nice to
   either get the GVariant type extensions into the D-Bus protocol
   or disable them before merging GVariant into GLib. I don't know
   how that would affect other GVariant users.

   We probably need to make GVariant support the 'h' type. I guess
   it shouldn't be too much work but the devil is in the details.
   Ryan?

 - GVariant is not protocol compatible with the D-Bus protocol. This
   is not a deal-breaker, in my view, but it means there's a
   encoding/decoding step involved.

   Ryan assures me that the current D-Bus protocol encoder/decoder (I
   lifted the one from DConf, did some fixes - see gdbusconversion.c
   in the GDBus tree) can be optimized - do some binary thing or
   whatever. Personally I don't care too much - if you have a lot
   of throughput, you probably shouldn't use D-Bus (use e.g. sockets
   instead) - but YMMV.

 - GVariant is very pleasant to use
   - Good job Ryan!

Gosh, that was a long mail. Sorry about that. To add insult to injury,
I'd like to ask if people can reply to separate parts of the mail using
separate mails rewriting the Subject line as needed. Thanks!

Thanks,
David


_______________________________________________
gtk-devel-list mailing list
gtk-devel-list@...
http://mail.gnome.org/mailman/listinfo/gtk-devel-list

[Simon McVittie-6]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

On Wed, 14 Oct 2009 at 21:34:26 -0400, David Zeuthen wrote:
>  - Ideally make insane things impossible to do
>
>    - e.g. only allow proxy creation for unique bus names - not
>      well-known names. Instead people will discover they should
>      be using e.g. g_bus_watch_name().

I don't think using well-known names is insane; you just have to be a little
careful. Forbidding them would break automatic service-activation, which is
not only implemented but also explicitly mentioned in the D-Bus specification,
so clearly Havoc didn't think it was insane either...

In practice if you're using the well-known name on a proxy, then you want to
watch the name owner *as well* - the proxy could do this automatically?

The fundamental decision to make is whether your D-Bus API's assumptions would
break if you called a method on one instance of a service, the instance exited,
a new instance started, and you got a reply from the new instance. Most
complex APIs do break (they are stateful, using that word quite loosely), in
which case service-activation doesn't work and using the well-known name is
indeed insane. As a result, I do agree that the *default* should be to use the
unique name; this is the "safe option", and it's the default in dbus-python
too. (If you make a proxy for a well-known name with follow_name_owner_changes
disabled, as is the default, then dbus-python will try to start it for you.)

However, some APIs either work by coincidence (e.g. because they're so simple),
or are specifically designed to work like that (the new instance will closely
resemble the old one, and either doesn't need any of the old one's state, or
loads that state from disk on startup).

A prerequisite for having calls by well-known name work correctly is having
either a well-known object path as an entry point, or persistent object paths
that can be saved across service runs.

Most APIs that follow the "one service with one object implementing one
interface" pattern can be used with a well-known name, I suspect; for instance,
the Galago/freedesktop.org notification daemon API is mostly OK for this use.

I say "mostly" because in corner cases, like replacing your notification daemon
implementation with one that doesn't support markup, the API requires
capability discovery to be done again. However, if the notification daemon spec
was changed so that messages were *always* in GMarkup-style markup (and
text-only daemons were required to ignore/strip out the markup), then I think
it would be safe to use by well-known name: you could do capability
re-discovery opportunistically, in order to start or stop using advanced
optional features, but the message you sent to the old implementation would
still be perfectly valid for the new one due to the graceful fallback in n-d's
"hints" mechanism.

(I'm not suggesting changing the n-d spec; rather, I'm using a hypothetical
changed version of its spec as an example of a well-known-name-safe API that
would be quite easy to design.)

In Telepathy[1], you can safely use the well-known name for:

* AccountManager (well-known singleton)
* Account (persistent objects in the AccountManager with saved object-paths -
  you do need to handle the corner case where your account was deleted while
  the AccountManager wasn't running, but our domain-specific libraries do that)
* ChannelDispatcher (well-known singleton)
* ConnectionManager (well-known singleton per installed CM)
* Client (UIs, loggers etc. invoked by the ChannelDispatcher - well-known
  singleton per installed Client)

but you can't sensibly use it for:

* Connection (transient object in the CM, representing a connection to the
  server)
* Channel (transient object in the CM, representing a channel of communication)
* ChannelDispatchOperation (transient object in the ChannelDispatcher)
* ChannelRequest (transient object in the ChannelDispatcher)

In telepathy-qt4 we make this very explicit by having an abstract DBusProxy
class (loosely based on telepathy-glib), which concrete subclasses
StatefulDBusProxy and StatelessDBusProxy. Stateful and stateless aren't quite
the right words, but they're the best I could come up with at the time...

    Simon

[1] I think making it halfway through an email about D-Bus before I mention
Telepathy might be a first for me :-)
-----BEGIN PGP SIGNATURE-----

iQIVAwUBStcWH03o/ypjx8yQAQh37RAAhjJ0EupOIDDYy2LMPnVMMLJtj1P6tmAC
CIzhwL+gHWxGsFAAXgBIIk4lvt0af7SH1WuexIYb0ZgtCz5t8PGG+1b3FJeu7MNJ
F9POtjLuLrU8DaZRc4AifogA5xS32TEK0whOx9RSXAduZu3QQHWw1t9TIdU5HNgh
1Ky99PAPkfRneHhKhtOmDTXl4hG9/0v4oOMNH/rjf+jqWdUUP+/Svt4GWjtfTwFq
tuK9XM+MRaj3zVjYmby9v7RK/TlvYJv4oLYg999KYAjC2QtBaUnzIMDeIVFLf/2o
ay4Pvsmo9Dm9p2+NYc3J3Wi2OpO82tV3nDnMOmBn2RWrWqmlvgG5I/Z2VaqFXuWr
Arx7sYTJj7rmE9nBemYMLoiSB7DNN5DPHg8IPq5OuZ6QoyhjWlRXALj6gIwHN0Fj
HYbTG3OMfIK6WYoQf5YVOmS3/HjKULQt/0H0wdIXis2WF661HjOidqMIx4a94SDN
7KP3eB3kOuybkuNHCbpVIJ3mXp9SRKKrCxI7gJENAwSpbKiUlF0YdX83PU5WU881
zhMBombvVSd3ZgXRVXzMoMhmmDKLixUEMJJ64kIp7X8/JPNphkGukGc1fPWR0H2o
/Nbhy+s3k6YhUo0fSPtELfStfJCq06V+CPXegEUWBb1MWJlk90TwdM2ettqU7bCP
B7iLSrD1204=
=J0ZN
-----END PGP SIGNATURE-----
_______________________________________________
gtk-devel-list mailing list
gtk-devel-list@...
http://mail.gnome.org/mailman/listinfo/gtk-devel-list

[Ryan Lortie]

Hi everyone,

On Wed, 2009-10-14 at 21:34 -0400, David Zeuthen wrote:
> There was an amazing amount of consensus; here's a brief list

I am in complete agreement with any point in the list that I am not
directly addressing.  Other comments below:


>  - Ideally make insane things impossible to do
>
>    - e.g. only allow proxy creation for unique bus names - not
>      well-known names. Instead people will discover they should
>      be using e.g. g_bus_watch_name().

There was some small amount of disagreement on this topic: particularly,
there was some discussion that it might make sense to have proxy objects
bound to common names in the case where the service is stateless (ie:
can leave the bus and be re-activated without anyone noticing).

>  - Make the code available in libgio (perhaps with the libdbus-1 using
>    implementation in e.g. GVfs through GIOExtensionPoint - maybe not
>    worth the effort)

This would have the advantage of forcing a good design upon us -- the
convenience API would be forced to have very low coupling with
libdbus-1.  It would also prevent future hackers from accidentally
introducing more 'surface area' between these components.

There are a few approaches we could take to this end.  All of them
assume that the user-facing API lives in libgio and that we use a GIO
extension point.

  1) We could include the libdbus code directly in libgio.so and
     internally register the extension point implementation.

  2) We could include the code for libdbus in gio/ but have it
     build a separate .so file that gets installed into the GIO
     modules directory.

  3) We could put it in GVFS.

>  - Ensure the provided code is thread-safe / can be used from threads
>    - e.g. deliver replies/signals/method-invocation in the thread where
>      the method was called from, signal was subcribed to etc.
>    - this may^Wwill include fixing libdbus-1 bugs

Using the per-thread 'default mainloop' stuff of course.


>    The API does not expose a GDBusMessage type since that would cause
>    unwanted API that would assist in using the API as a whole in a
>    wrong way (e.g. exposing on-wire details like message headers
>    can be nasty). Instead, the API is based on invoking/handling methods
>    and receiving/emitting signals.

I think we might want to have some sort of GDBusMessage class.  Here are
a couple of reasons why:

 - I think it is valuable to be able to register very low-level filter
   functions (or signal watches, or async reply handlers) that deal
   directly with the message as it came off the bus (including the body
   and the full set of headers).

 - This is useful for people who want to do other types of object
   binding on top of GDBus.

 - Maybe you don't want automatic GError conversion

 - Easy way to send a reply to a specific message

 - No need to pass many arguments to functions or to have the types of
   arguments varied by the sort of function (incoming method call vs.
   signal vs. async result, etc...)

This could be approximated with a GHashTable or GVariant dictionary (of
headers) and a GVariant (for the body).  Our GDBusMessage class, in
fact, could be little more than a wrapper around the two.


>  - I decided to represent wire-errors (e.g. DBusError) using GError
>    since that type is widely used both in C and language bindings. There
>    are facilities for recovering the D-Bus error name. We also support
>    transparent passing of any GError (a feature used in e.g. GVfs).

I agree that this functionality should be present in the low-level
bindings (for example, you almost definitely want to have it on simple
g_bus_make_a_call() interfaces) but it should also be possible to
shortcut it.

> Some notes on GVariant
>
>  - GVariant is very pleasant to use
thanks :)

>  - GVariant needs to be LGPL2 (I believe Ryan has fixed this already)
Not already, but of course I will.

>  - GVariant doesn't use en_US spelling. I think this is a deal-breaker,
>    it needs to read e.g. serialize instead of serialise. Anyway, I'm
>    da_DK myself but...
I'm not sure I understand the reasoning, but if it really is a
deal-breaker, I guess I have to do what I have to do :p

>  - GVariant is a superset of a subset of the D-Bus protocol (!)
>
>    1. GVariant is a superset because it handles e.g. 'maybe' types
"Maybe" some day we can get this support into DBus :)

Actually, GVariant is a superset of DBus in several other (more minor)
ways:

  - no nesting depth restriction
  - no message size restriction
  - no restriction that dictionary entries must appear inside arrays
  - the unit type "()" is a valid type

>    2. GVariant doesn't yet handle the new 'h' type (e.g. unix fd)

Holy crap!  This is insane!  I had no idea that this was going on
upstream -- I might have had something to say about it.

It seems kind of unfortunate that the DBus type system used to consist
only of things that could be serialised and now it consists of a special
case that can't be.  That's really difficult to deal with from the
standpoint of GVariant (which, really, is a serialisation framework,
after all).

Now that it does exist upstream though, I guess that means that GVariant
really ought to support it.  This is going to require some thinking....

>    Now, 1. has been discussed extensively on this list. I don't
>    have any position on it, really. I guess it would be nice to
>    either get the GVariant type extensions into the D-Bus protocol
>    or disable them before merging GVariant into GLib. I don't know
>    how that would affect other GVariant users.

Incidentally, the merging of the 'h' type into DBus shows that we
needn't be afraid of having a feature in the type system that is not
necessarily supported for all uses (ie: you won't be able to send 'h'
messages to a bus that you're not using a unix socket for).  I believe
we can have the same restriction for GDBus: have a 'this message is
unsupported' error if you try to do silly things like sending a message
that's too large.

>    We probably need to make GVariant support the 'h' type. I guess
>    it shouldn't be too much work but the devil is in the details.
>    Ryan?

I'm thinking about this, but it's going to be very strange.  I'm
confident that I can do it within the confines of the existing API so
this might end up being something that we only add later on (the feature
in DBus, after all, is *very* new).

>  - GVariant is very pleasant to use
>    - Good job Ryan!

Thanks again :)


> I'd like to ask if people can reply to separate parts of the mail using
> separate mails rewriting the Subject line as needed. Thanks!

You should have put that request at the top.  Now that I'm done writing,
I really don't feel like splitting it all up :)


Thanks for the summary, David.  Feels good to get the ball rolling.

Cheers

_______________________________________________
gtk-devel-list mailing list
gtk-devel-list@...
http://mail.gnome.org/mailman/listinfo/gtk-devel-list

[David Zeuthen]

On Thu, 2009-10-15 at 10:54 -0400, Ryan Lortie wrote:

> >  - Make the code available in libgio (perhaps with the libdbus-1 using
> >    implementation in e.g. GVfs through GIOExtensionPoint - maybe not
> >    worth the effort)
>
> This would have the advantage of forcing a good design upon us -- the
> convenience API would be forced to have very low coupling with
> libdbus-1.  It would also prevent future hackers from accidentally
> introducing more 'surface area' between these components.
>
> There are a few approaches we could take to this end.  All of them
> assume that the user-facing API lives in libgio and that we use a GIO
> extension point.
>
>   1) We could include the libdbus code directly in libgio.so and
>      internally register the extension point implementation.
>
>   2) We could include the code for libdbus in gio/ but have it
>      build a separate .so file that gets installed into the GIO
>      modules directory.
>
>   3) We could put it in GVFS.

Yeah, hmm, while this is theoretically nice and everything, I'm not sure
it's worth the effort to do just yet. I think it's better to just ensure
that libdbus-1 is available (and working) on all platforms where we care
about GLib (e.g. Unix-like, Win32 and OS X).

(Also, one side-effect is that all methods on GDBusConnection would need
to use vfuncs with the incurred cost of doing so.)

(Also, reimplemenations would need to do a lot of things that libdbus-1
does and those things are not really specced out yet - things like
looking up the bus address in the X server (or similar on Win32, OS X),
autospawning etc. comes to mind)

Anyway, we should definitely leave space in GDBusConnectionClass (and
GDBusMethodInvocationClass) for this so we can add an extension point
later.

    David


_______________________________________________
gtk-devel-list mailing list
gtk-devel-list@...
http://mail.gnome.org/mailman/listinfo/gtk-devel-list

[Tor Lillqvist]

>  - The libdbus-1 on Win32 / OS X story. Gah.
>
>   So either we a) restrict this to UNIX (put headers in gio-unix-2.0)
>   or; b) we just tell people they need to use e.g. WinDBus on Win32,

There is a relatively new effort to clear up the dbus situation on
Windows situation, "dbus4win", see
http://lists.freedesktop.org/archives/dbus/2009-April/011207.html ,
but last time when I asked about that I was told that the people doing
it consider it done, but haven't done any releases (source or binary),
or upstreaming efforts.

When I looked at it it seemed to work, but unfortunately the upstream
dbus code has in the meantime diverged quite a bit from when the
dbus4win repo was cloned. No idea how hard it will be to merge back.
(One minor problem in dbus4win is that they use a different build
system (cmake) and their code hardcodes some pathname assumption that
differ from what a "normal" "./configure --prefix=/whatever && make
install" does.)

--tml
_______________________________________________
gtk-devel-list mailing list
gtk-devel-list@...
http://mail.gnome.org/mailman/listinfo/gtk-devel-list

[David Zeuthen]

On Thu, 2009-10-15 at 10:54 -0400, Ryan Lortie wrote:
> >    The API does not expose a GDBusMessage type since that would cause
> >    unwanted API that would assist in using the API as a whole in a
> >    wrong way (e.g. exposing on-wire details like message headers
> >    can be nasty). Instead, the API is based on invoking/handling methods
> >    and receiving/emitting signals.
>
> I think we might want to have some sort of GDBusMessage class.  Here are
> a couple of reasons why:

It's not that message-based APIs is better than signal/method-based APIs
or the other way around - it's just deciding what level of abstraction
we want our APIs to have. My view is that signal/method-based APIs will
cover 99.9% of all cases and that providing a message based API will
make things, say, 10% harder to use for everyone. I don't think that's a
good trade-off.

(I just pulled 95% of these statistics out of my ass)

>
>  - I think it is valuable to be able to register very low-level filter
>    functions (or signal watches, or async reply handlers) that deal
>    directly with the message as it came off the bus (including the body
>    and the full set of headers).

If people want this very low-level functionality, they can always use
libdbus-1 or speak directly to the socket - do you have any specific
examples where the proposed method/signal-based API wouldn't work?

>  - This is useful for people who want to do other types of object
>    binding on top of GDBus.

Any specific examples?

>  - Maybe you don't want automatic GError conversion

With the way things work now, you are guaranteed to _always_ be able to
recover the full D-Bus error in its pristine form from the GError (using
g_dbus_error_get_dbus_error_name() and g_dbus_error_strip_error())

>  - Easy way to send a reply to a specific message
>
>  - No need to pass many arguments to functions or to have the types of
>    arguments varied by the sort of function (incoming method call vs.
>    signal vs. async result, etc...)

Any specific examples for these two points?

> This could be approximated with a GHashTable or GVariant dictionary (of
> headers) and a GVariant (for the body).  Our GDBusMessage class, in
> fact, could be little more than a wrapper around the two.

Keep in mind that we can always add the GDBusMessage class later if it
turns out we really want it. I'm mostly against doing this because it
means exposing a lot of API and some overhead as well.

Thanks,
David


_______________________________________________
gtk-devel-list mailing list
gtk-devel-list@...
http://mail.gnome.org/mailman/listinfo/gtk-devel-list

[David Zeuthen]

On Thu, 2009-10-15 at 10:54 -0400, Ryan Lortie wrote:

> >    2. GVariant doesn't yet handle the new 'h' type (e.g. unix fd)
>
> Holy crap!  This is insane!  I had no idea that this was going on
> upstream -- I might have had something to say about it.
>
> It seems kind of unfortunate that the DBus type system used to consist
> only of things that could be serialised and now it consists of a special
> case that can't be.  That's really difficult to deal with from the
> standpoint of GVariant (which, really, is a serialisation framework,
> after all).
>
> Now that it does exist upstream though, I guess that means that GVariant
> really ought to support it.  This is going to require some thinking....

Yeah, I think we need to support this from the get-go.

Anyway, at the end of the day, UNIX fds are just integers so if you
require that users (such as GDBus itself and apps using GDBus) keep the
fds alive until the method/signal has been sent/received you should be
able to treat them as such. Anyway, it's probably more complicated than
this.

    David


_______________________________________________
gtk-devel-list mailing list
gtk-devel-list@...
http://mail.gnome.org/mailman/listinfo/gtk-devel-list

[David Zeuthen]

On Thu, 2009-10-15 at 13:31 +0100, Simon McVittie wrote:

> On Wed, 14 Oct 2009 at 21:34:26 -0400, David Zeuthen wrote:
> >  - Ideally make insane things impossible to do
> >
> >    - e.g. only allow proxy creation for unique bus names - not
> >      well-known names. Instead people will discover they should
> >      be using e.g. g_bus_watch_name().
>
> I don't think using well-known names is insane; you just have to be a little
> careful. Forbidding them would break automatic service-activation, which is
> not only implemented but also explicitly mentioned in the D-Bus specification,
> so clearly Havoc didn't think it was insane either...

Yeah, of course. But no-one prevents you from using GDBusConnection to
invoke methods on well-known names. It's certainly not ideal because you
typically want to use a generated method on a GDBusProxy subclass.

Anyway, you raise good points about activation here. First of all, I
think we need to have GBusNameWatcherFlags with a member called
G_BUS_NAME_WATCHER_FLAGS_START_SERVICE.

> In practice if you're using the well-known name on a proxy, then you want to
> watch the name owner *as well* - the proxy could do this automatically?

It's very tempting to allow construction of a GDBusProxy (or subclass)
with a well-known name. Let's examine the implications. We'd need

 - a G_DBUS_PROXY_FLAGS_START_SERVICE flag so you can specify that the
   service with the given well-known name should be started if it
   doesn't exist.

 - a GDBusProxy:g-dbus-proxy-bus-name property with the name you passed
   to the constructor. The GDBusProxy:g-dbus-proxy-unique-bus-name
   property would then contain the unique name.

   GDBusProxy:g-dbus-proxy-unique-bus-name would be NULL if exactly when
   (e.g. if, and only if) there is no name owner.

 - a GDBusProxy::g-dbus-proxy-vanished signal when the name owner
   vanishes (and also notify::g-dbus-proxy-unique-bus-name)

 - a GDBusProxy::g-dbus-proxy-appeared signal when the name owner
   appears (and also notify::g-dbus-proxy-unique-bus-name)

 - a new G_DBUS_ERROR_NO_NAME_OWNER_FOR_PROXY error to return when you
   attempt to perform an operation while
   GDBusProxy:g-dbus-proxy-unique-bus-name is NULL

 - a g_dbus_proxy_reconnect() that can be used only when
   GDBusProxy:g-dbus-proxy-unique-bus-name is NULL. It would just
   call StartService()

 - we'd still be using unique names inside GDBusProxy for connecting
   to signals, invoking methods and listening to property changes. So
   the race conditions would be mostly gone.

 - construction of a GDBusProxy with a well-known name would never fail

 - construction of a GDBusProxy with a unique name would still be able
   to fail

So all this is doable... and I guess.. possible to implement in a safe
way. It kinda dilutes the g_bus_watch_name() and g_bus_watch_proxy()
APIs and might cause people to use the wrong APIs. But I think this can
be solved by throwing docs at the problem. Thoughts?

     David


_______________________________________________
gtk-devel-list mailing list
gtk-devel-list@...
http://mail.gnome.org/mailman/listinfo/gtk-devel-list

[Ryan Lortie]

On Thu, 2009-10-15 at 12:38 -0400, David Zeuthen wrote:
> Yeah, I think we need to support this from the get-go.
>
> Anyway, at the end of the day, UNIX fds are just integers so if you
> require that users (such as GDBus itself and apps using GDBus) keep the
> fds alive until the method/signal has been sent/received you should be
> able to treat them as such. Anyway, it's probably more complicated than
> this.

I talked to Lennart on IRC and the 'h' type, as serialised, isn't a file
descriptor so much as it is an index into a table of file descriptors
that are stored separately (in a separate part of the DBusMessage).

Keeping this in mind, we could support this by doing the same thing with
GDBus -- support 'file descriptor array offset' as a special type of
integer in GVariant (much like 'o'bject path is a special type of
string) and have a GDBus call for send_with_file_descriptors() or such.

Lennart mentioned that he thinks that it's a bad idea that the
dbus_message_marshal stuff was ever added to libdbus, and that the
fd-passing stuff in libdbus will break if you try to use those
functions.  If we ever want to make a more efficient GVariant-to-DBus
conversion process we'll have to turn it off for the case where fd's are
present -- that's another reason for having a separate call, maybe.

I don't think we would want to have any more 'deeply integrated' support
than that, for a lot of reasons: Lennart mentioned that this only really
works on Linux (and of course, only works if you connect to the bus
using Unix sockets).  There's also the concern of what it would mean to
serialise a GVariant that contains an fd (to dconf, to the registry, to
text format, to XML, or to anything else).

Cheers

_______________________________________________
gtk-devel-list mailing list
gtk-devel-list@...
http://mail.gnome.org/mailman/listinfo/gtk-devel-list

[Ryan Lortie]

On Thu, 2009-10-15 at 12:24 -0400, David Zeuthen wrote:
> Yeah, hmm, while this is theoretically nice and everything, I'm not sure
> it's worth the effort to do just yet. I think it's better to just ensure
> that libdbus-1 is available (and working) on all platforms where we care
> about GLib (e.g. Unix-like, Win32 and OS X).

I'd actually advocate approach #1.  It's extremely low-overhead.  After
initialisation, in fact, there is no overhead at all (compared to if you
had implemented the interface to the bus as an abstract class and
implemented it in the normal way).

> (Also, one side-effect is that all methods on GDBusConnection would need
> to use vfuncs with the incurred cost of doing so.)

vfuncs on abstract classes are extremely cheap (pointer dereference +
call).  Compared to the fact that we are sending something over DBus,
they practically don't exist.

> (Also, reimplemenations would need to do a lot of things that libdbus-1
> does and those things are not really specced out yet - things like
> looking up the bus address in the X server (or similar on Win32, OS X),
> autospawning etc. comes to mind)

Yes.  Buyer beware, certainly.

> Anyway, we should definitely leave space in GDBusConnectionClass (and
> GDBusMethodInvocationClass) for this so we can add an extension point
> later.

I'm slightly confused what you mean by this.  I was thinking that
DBusConnectionClass (for example) would itself be abstract with vtable
entries for all the stuff you might want to do.  The only reason to
leave space there would be the ordinary one -- the inevitability that
you will want to add some new functions some day.

On second reading, I guess you mean that you want to make the calls
non-virtual and leave space to allow for them to be made virtual in the
future.  This makes sense, except I hope you will consider that the cost
of making them virtual now is practically zero.

Cheers

_______________________________________________
gtk-devel-list mailing list
gtk-devel-list@...
http://mail.gnome.org/mailman/listinfo/gtk-devel-list

[Philip Van Hoof]

On Thu, 2009-10-15 at 14:02 -0400, Ryan Lortie wrote:

> On Thu, 2009-10-15 at 12:38 -0400, David Zeuthen wrote:
> > Yeah, I think we need to support this from the get-go.
> >
> > Anyway, at the end of the day, UNIX fds are just integers so if you
> > require that users (such as GDBus itself and apps using GDBus) keep the
> > fds alive until the method/signal has been sent/received you should be
> > able to treat them as such. Anyway, it's probably more complicated than
> > this.
>
> I talked to Lennart on IRC and the 'h' type, as serialised, isn't a file
> descriptor so much as it is an index into a table of file descriptors
> that are stored separately (in a separate part of the DBusMessage).

The fd-over-dbus is just the result of dup2(), right? Well actually
either F_DUPFD or F_DUPFD_CLOEXEC if I read Lennart's patches correctly.

What is the problem with using integer for that 'h' type?

Well, other than the fact that when you'd serialize the GVariant onto
disk, and deserialize it, then wouldn't the integer have any meaning
anymore. of course.

I guess that's inherent to this "fd over DBus" technique.

> Keeping this in mind, we could support this by doing the same thing with
> GDBus -- support 'file descriptor array offset' as a special type of
> integer in GVariant (much like 'o'bject path is a special type of
> string) and have a GDBus call for send_with_file_descriptors() or such.

Yah - something like that.

[cut]


--
Philip Van Hoof, freelance software developer
home: me at pvanhoof dot be
gnome: pvanhoof at gnome dot org
http://pvanhoof.be/blog
http://codeminded.be

_______________________________________________
gtk-devel-list mailing list
gtk-devel-list@...
http://mail.gnome.org/mailman/listinfo/gtk-devel-list

[David Zeuthen]

On Thu, 2009-10-15 at 14:02 -0400, Ryan Lortie wrote:

> On Thu, 2009-10-15 at 12:38 -0400, David Zeuthen wrote:
> > Yeah, I think we need to support this from the get-go.
> >
> > Anyway, at the end of the day, UNIX fds are just integers so if you
> > require that users (such as GDBus itself and apps using GDBus) keep the
> > fds alive until the method/signal has been sent/received you should be
> > able to treat them as such. Anyway, it's probably more complicated than
> > this.
>
> I talked to Lennart on IRC and the 'h' type, as serialised, isn't a file
> descriptor so much as it is an index into a table of file descriptors
> that are stored separately (in a separate part of the DBusMessage).
>
> Keeping this in mind, we could support this by doing the same thing with
> GDBus -- support 'file descriptor array offset' as a special type of
> integer in GVariant (much like 'o'bject path is a special type of
> string) and have a GDBus call for send_with_file_descriptors() or such.
>
> Lennart mentioned that he thinks that it's a bad idea that the
> dbus_message_marshal stuff was ever added to libdbus, and that the
> fd-passing stuff in libdbus will break if you try to use those
> functions.  If we ever want to make a more efficient GVariant-to-DBus
> conversion process we'll have to turn it off for the case where fd's are
> present -- that's another reason for having a separate call, maybe.
>
> I don't think we would want to have any more 'deeply integrated' support
> than that, for a lot of reasons: Lennart mentioned that this only really
> works on Linux (and of course, only works if you connect to the bus
> using Unix sockets).  There's also the concern of what it would mean to
> serialise a GVariant that contains an fd (to dconf, to the registry, to
> text format, to XML, or to anything else).

Yeah, the 'h' type really is for _handles_ - see the discussion on the
dbus list when this was introduced (well, initially Lenny wanted 'f' -
but some D-Bus implementations use that for gfloat and we didn't want to
screw them).

So only in Unix (mind you, not only Linux but any Unix that can pass fds
- including Solaris and OS X) 'h' used for sending file descriptors but
in Win32 it could probably be used for a HANDLE (e.g. the Win32 concept
of file descriptors) - I actually think you can pass HANDLE to other
Win32 processes.

So how about something like 1. and 2. below? We'd put

 g_dbus_connection_get_handle_for_unix_fd()
 g_dbus_connection_set_handle_for_unix_fd()

in the gio-unix-2.0 headers. Presumably we could add

 g_dbus_connection_get_handle_for_win32_handle()
 g_dbus_connection_set_handle_for_win32_handle()

and put that in gio-win32-2.0 if such a thing was to surface.

In the future we could also have

 g_dbus_connection_get_handle_for_iostream()
 g_dbus_connection_set_handle_for_iostream()

that takes a GIOStream (this might not be a good idea, I don't know).

For this to work, however, I'd need something like

 g_object_set_data_full()
 g_object_get_data()

on the GVariant type since we'd want to clean up stuff when the passed
GVariant is destroyed. It might be nice to have that on GVariant
*anyway*, see

http://bugzilla.gnome.org/show_bug.cgi?id=581106

for why associations are useful in general.

Thanks,
David

---

1. user code for sending variants:

 GVariant *parameters;

 parameters = g_variant_new ("(ih"),
               some_int,
               g_dbus_connection_get_handle_for_unix_fd (connection,
                                                         variant,
                                                         some_fd));
 g_dbus_proxy_invoke_method (proxy,
                             "SomeMethodTakingAnIntAndUnixFd",
                             parameters,
                             -1, NULL, callback, user_data);

2. user code for receiving a variant

 gint some_int;
 gint handle;
 gint fd;

 handle = g_variant_get (result_parameters, "(ih)", &some_int, &handle);
 fd = g_dbus_connection_get_unix_fd_for_handle (connection, handle);
 g_variant_unref (result_parameters);

 /* use fd */

 close (fd);


_______________________________________________
gtk-devel-list mailing list
gtk-devel-list@...
http://mail.gnome.org/mailman/listinfo/gtk-devel-list

[Ryan Lortie]

On Tue, 2009-10-20 at 11:17 -0400, David Zeuthen wrote:
> So how about something like 1. and 2. below? We'd put
>
>  g_dbus_connection_get_handle_for_unix_fd()
>  g_dbus_connection_set_handle_for_unix_fd()

I was actually thinking that we could introduce a GDBusFDSet:

gint g_dbus_fd_set_add_fd (gint fd) {
  list_of_fds[fd_count] = dup (fd);
  return fd_count++;
}

gint g_dbus_fd_set_get_fd (gint index) {
  return dup (list_of_fds[index]);
}

g_dbus_fd_set_finalize () {
  foreach (fd in list_of_fds) close (fd);
}

You'd add fds to that and get ints back (starting at 0 per the set).
Then when sending the dbus message you'd give the GVariant and the
DBusFDSet to the appropriate function.

  var fd = open("file");
  var fdset = new FDSet ();
  var message = new GVariant ("(sh)",
                              "hello world",
                              fdset.add_fd (fd));

  g_dbus_proxy_invoke_with_fds (proxy, "Foo", message, fdset, ...);

When receiving messages, you get a DBusFDSet handed to you in the event
that there were file descriptors (or just %NULL in case not).

The fds held in the GDBusFDSet are closed when the set is finalized.
There is no doubt about who owns which fd in this case -- and in the
case you elect to ignore fds that were sent to you then everything is
still cleaned up properly.

We could also have _steal() and _give() API variants.  _steal would take
the fd out of the set (so the set no longer owns it, no longer closes
it) and _give would let you transfer ownership of an fd into the set --
we save a few syscalls that way in a few cases, I guess.

btw: I don't know if passing HANDLE on Windows is supported (and I would
guess not).  Passing a G[Input|Output|IO]Stream is certainly not
supported (in a portable way, anyway).

Cheers

_______________________________________________
gtk-devel-list mailing list
gtk-devel-list@...
http://mail.gnome.org/mailman/listinfo/gtk-devel-list

[Tor Lillqvist]

> btw: I don't know if passing HANDLE on Windows is supported (and I would
> guess not).

Sure it is. DuplicateHandle() can be used for this. I.e. a process A
that has the required access right (PROCESS_DUP_HANDLE) to two
processes U and V can take a HANDLE value valid in source process U
and duplicate it into a HANDLE value valid in the target process V.
Then it just needs to pass that new HANDLE value (which is just a
number, does not have to be kept secret or anything) using whatever
normal inter-process messaging to process V and process V can use it.

If I understand the documentation correctly,  A must be either U or V,
and in most cases presumably A == U, i.e. a process wants to give
another process a duplicate of one of its own HANDLEs.

Note that the duplicated handle is indeed just a number so no special
mechanisms are needed to pass it to the target process. In process A,
the duplicated HANDLE is not valid. It might have the same numeric
value as some HANDLE (that refers to a totally unrelated object) in
process A, but that is just a coincidence then.

I hate to say this, but IMHO this a more obvious API than the fd
passing over socket thing, which I have always found inelegant,
non-obvious, and in a way quite non-Unixish.

But then, ha, having said that, I now see this gem in the documentation:

"You should not use DuplicateHandle to duplicate handles to the
following objects: [,,,] Sockets. No error is returned, but the
duplicate handle may not be recognized by Winsock at the target
process. Also, using DuplicateHandle interferes with internal
reference counting on the underlying object. To duplicate a socket
handle, use the WSADuplicateSocket function."

OK, so for sockets one needs to call WSADuplicateSocket() instead of
DuplicateHandle(), and WSADuplicateSocket() takes just a target
process parameter. Oh well, that corresponds to the most common case A
== U anyway.

(Windows named pipes also have functionality that I think could be
used in a vague sense to achieve the same end means that fd passing is
often used for, i.e. give another process access to something (an
opened file) it wouldn't have access to (be able to open) otherwise. I
am talking about the "impersonation" thing, but actually now that I
think of it, this functionality is presumably usually used not to give
the named pipe server access to something it wouldn't have access to
otherwise, but on the contrary, to make a named pipe server restrict
its rights to those of the client. And anyway, as Thiago has pointed
out on the dbus list, named pipes are a pain to use.)

--tml
_______________________________________________
gtk-devel-list mailing list
gtk-devel-list@...
http://mail.gnome.org/mailman/listinfo/gtk-devel-list

[Alexander Larsson]

On Wed, 2009-10-14 at 21:34 -0400, David Zeuthen wrote:

>  http://cgit.freedesktop.org/~david/gdbus-standalone/

I just read through this for basic review, and I must say that I like
it. Very nice stuff. Some comments:

In nautilus I'd like to have an object path which is basically a pointer
to another object. Say I have these objects:

/nautilus/window/1/tabs/1/view
/nautilus/window/1/tabs/2/view

Then i want something like:

/nautilus/window/1/active_view

That always points to the view object in the active tab.

Now, i could constantly be re-creating this object all the time, but
thats not really very nice. What i instead want to do is basically
register a subtree at /nautilus/window/1/active_view which dynamically
looks up which tab is active and then re-routes to the right tab.

To implement this cleanly I would need to be able to lookup an object at
a given path (returning the ifaces, vtables and user data) and to
enumerate the children of a given path. All this would be in-client
registered objects/subtrees only, and is enough to implement
GDBusSubtreeVTable for the "pointer subtree".

I don't see any call to dbus_threads_init() or
dbus_threads_init_default(). Surely this should be done if gthread is
enabled. Ideally in g_thread_init_glib(), but thats a bit hard for
something in libgio...

I'd like it to be possible to specify a non-default main context when
creating a connection with g_dbus_connection_new() or
g_dbus_connection_bus_get_private().

g_dbus_connection_send_dbus_1_message_with_reply() has a race condition
if called on a thread other than the one dispatching the reply. The
pending call may be completed before dbus_pending_call_set_notify() is
reached. See _g_dbus_connection_call_async in gvfs/common/gdbusutils.c
for a possible workaround.

The way e.g. properties are implemented is very static. There is no way
to expose an object with a dynamic set of properties (apart from
removing and re-adding the object with new introspection data each time
dynamic properties/methods can't be implemented? (lazy, i.e. not remove
and re-add each time they change). I guess this is sort of weird wrt how
dbus usually works, but might be useful when exporting an object from a
more dynamic language like javascript or python.

Maybe we should not claim to implement org.freedesktop.DBus.Properties
if set/get_property are NULL?

I'm not sure if I missed it, because i didn't try it out, but how do
registered object with a a child that is specified by registering a
subtree get enumerated when introspecting the registered object?

The foo_bar_error_quark example in gdbuserror.c clearly should just call
a standard helper that does all this, just given the pointer to an array
of data.

org.gtk.GDBus.UnmappedGError.Quark0xHEXENCODED_QUARK_NAME_.Code_ERROR_CODE
Why use a hex encoding for the quark name? Most gerror quark names are
quite readable, and it would be nice to preserve this in the encoding
used.

There are no bindings for dbus_connection_flush, which might be nice to
e.g. ensure that the reply sent from a message handler is immediately
sent since we may block or whatnot later. Maybe we want this?

gdbusnameowning.c:request_name_cb() - Isn't it racy to not subscribe to
NameLost/NameAquired until we're processing the request name reply?
We could have gotten a NameLost message inbetween the RequestName reply
gotten and request_name_cb being handled by the mainloop, as the
request_name_cb is called from another idle.

g_dbus_server_new() wants a way to specify the GMainContext to use.

gdbusprivate.c has a bunch of code with a copyright notice from dbus
that is AFL 2.1 or GPL2+. I.e. not LGPL. This seems like a problem for
glib integration.

Are you aware of:
#define G_PARAM_STATIC_STRINGS (G_PARAM_STATIC_NAME |
G_PARAM_STATIC_NICK | G_PARAM_STATIC_BLURB)
It saves some typing.



_______________________________________________
gtk-devel-list mailing list
gtk-devel-list@...
http://mail.gnome.org/mailman/listinfo/gtk-devel-list

[David Zeuthen]

Hey Ryan,

Sorry for the lag,

On Sat, 2009-10-31 at 17:27 -0400, Ryan Lortie wrote:

> On Tue, 2009-10-20 at 11:17 -0400, David Zeuthen wrote:
> > So how about something like 1. and 2. below? We'd put
> >
> >  g_dbus_connection_get_handle_for_unix_fd()
> >  g_dbus_connection_set_handle_for_unix_fd()
>
> I was actually thinking that we could introduce a GDBusFDSet:
>
> gint g_dbus_fd_set_add_fd (gint fd) {
>   list_of_fds[fd_count] = dup (fd);
>   return fd_count++;
> }
>
> gint g_dbus_fd_set_get_fd (gint index) {
>   return dup (list_of_fds[index]);
> }
>
> g_dbus_fd_set_finalize () {
>   foreach (fd in list_of_fds) close (fd);
> }
>
> You'd add fds to that and get ints back (starting at 0 per the set).
> Then when sending the dbus message you'd give the GVariant and the
> DBusFDSet to the appropriate function.
>
>   var fd = open("file");
>   var fdset = new FDSet ();
>   var message = new GVariant ("(sh)",
>                               "hello world",
>                               fdset.add_fd (fd));
>
>   g_dbus_proxy_invoke_with_fds (proxy, "Foo", message, fdset, ...);
>
> When receiving messages, you get a DBusFDSet handed to you in the event
> that there were file descriptors (or just %NULL in case not).
>
> The fds held in the GDBusFDSet are closed when the set is finalized.
> There is no doubt about who owns which fd in this case -- and in the
> case you elect to ignore fds that were sent to you then everything is
> still cleaned up properly.

Hmm, I don't like this approach. It means you'd have to pass this
GDBusFDSet object around in code you use to build/parse the GVariant. In
particular, your proposed GTypeSerializer would need support for it.
That's problematic because that one lives in libgobject, not libgio
(where GDBusFDSet would live).

I still think the approach I described works better.

For example, suppose I have a C type mapping that maps a{sh} to a
GHashTable from strings to gint. I'd like to write code like this

 my_hash = g_hash_table_new (g_str_hash, g_str_equal);
 g_hash_table_insert (my_hash,
                      "fd-for-client",
                      GINT_TO_POINTER (fd_for_client));

 /* need to keep fd_for_client alive until we hand off the hash */

 <other code>

 foo_bar_invoke_process_client (proxy, my_hash, <async stuff>);
 g_hash_table_unref (my_hash);

 /* it's now safe to close the fd, the dbus stuff has already dupped
  * it... note that we could have just used a GDestroyNotify for the
  * values in my_hash to do this.
  */
 close (fd_for_client);

where the latter function represents a generated function for the
ProcessClient() method on the foo.Bar interface that takes a single
a{sh} parameter represented by a GHashTable.

On the receiving end it would look like this. Suppose the GimmeFds
function returns a a{sh}:

 my_hash = foo_bar_invoke_gimme_fds_sync (proxy, cancelable, &error);

 fd = GPOINTER_TO_INT (g_hash_table_lookup (my_hash, "stream1"));
 if (fd > 0)
   {
     fd_for_stream1 = dup (fd);
   }

 /* the value destroy notifier on my_hash closes all fds */
 g_hash_table_unref (my_hash);

 <now use fd_for_stream1 and close it when done>

Thoughts?

Thanks,
David

_______________________________________________
gtk-devel-list mailing list
gtk-devel-list@...
http://mail.gnome.org/mailman/listinfo/gtk-devel-list

[David Zeuthen]

Hey Alex,

On Fri, 2009-11-06 at 21:05 +0100, Alexander Larsson wrote:
> On Wed, 2009-10-14 at 21:34 -0400, David Zeuthen wrote:
>
> >  http://cgit.freedesktop.org/~david/gdbus-standalone/
>
> I just read through this for basic review, and I must say that I like
> it. Very nice stuff. Some comments:

Thanks for reading through it!

> In nautilus I'd like to have an object path which is basically a pointer
> to another object. Say I have these objects:
>
> /nautilus/window/1/tabs/1/view
> /nautilus/window/1/tabs/2/view
>
> Then i want something like:
>
> /nautilus/window/1/active_view
>
> That always points to the view object in the active tab.
>
> Now, i could constantly be re-creating this object all the time, but
> thats not really very nice. What i instead want to do is basically
> register a subtree at /nautilus/window/1/active_view which dynamically
> looks up which tab is active and then re-routes to the right tab.
>
> To implement this cleanly I would need to be able to lookup an object at
> a given path (returning the ifaces, vtables and user data) and to
> enumerate the children of a given path. All this would be in-client
> registered objects/subtrees only, and is enough to implement
> GDBusSubtreeVTable for the "pointer subtree".

Yeah, I guess that's a reasonable thing to add. How about?

 gboolean
 g_dbus_connection_lookup_object (GDBusConnection *connection,
                        const gchar           *object_path,
                        const gchar           *interface_name,
                        GDBusInterfaceInfo   **out_introspection_data,
                        GDBusInterfaceVTable **out_vtable,
                        gpointer              *out_user_data,
                        guint                 *out_id);

 gboolean
 g_dbus_connection_lookup_subtree (GDBusConnection *connection,
                        const gchar           *object_path,
                        GDBusInterfaceVTable **out_vtable,
                        GDBusSubtreeFlags     *out_flags,
                        gpointer              *out_user_data,
                        guint                 *out_id);

> I don't see any call to dbus_threads_init() or
> dbus_threads_init_default(). Surely this should be done if gthread is
> enabled. Ideally in g_thread_init_glib(), but thats a bit hard for
> something in libgio...

Yeah, oops, good point - I guess we should do this in gdbusconnection.c

> I'd like it to be possible to specify a non-default main context when
> creating a connection with g_dbus_connection_new() or
> g_dbus_connection_bus_get_private().

Hmm, yeah, I was thinking about that. It's for specifying the mainloop
for processing D-Bus messages, right?

I was actually thinking that it might be nicer to always have a
dedicated thread for handling D-Bus messages. This would be shared by
all connections and it would just pop messages to the mainloops for
method invocations and signals.

This is feasible even if g_thread_init() has not be called, we'd just
use libpthread directly (which libdbus-1 pulls in already).

> g_dbus_connection_send_dbus_1_message_with_reply() has a race condition
> if called on a thread other than the one dispatching the reply. The
> pending call may be completed before dbus_pending_call_set_notify() is
> reached. See _g_dbus_connection_call_async in gvfs/common/gdbusutils.c
> for a possible workaround.

Will do.

> The way e.g. properties are implemented is very static. There is no way
> to expose an object with a dynamic set of properties (apart from
> removing and re-adding the object with new introspection data each time
> dynamic properties/methods can't be implemented? (lazy, i.e. not remove
> and re-add each time they change). I guess this is sort of weird wrt how
> dbus usually works, but might be useful when exporting an object from a
> more dynamic language like javascript or python.

Hmm, yeah, but remember that D-Bus properties (like GObject properties)
is part of the actual type - e.g. they are declared in introspection
data. So it would be very weird to have dynamic properties, it would be
similar to having dynamic properties on GObject.

If you want dynamic properties it's much better to just define your own
interfaces/methods for this (like e.g. HAL does).

> Maybe we should not claim to implement org.freedesktop.DBus.Properties
> if set/get_property are NULL?

Yeah, probably not.

> I'm not sure if I missed it, because i didn't try it out, but how do
> registered object with a a child that is specified by registering a
> subtree get enumerated when introspecting the registered object?

I'm not sure I understand this question - can you clarify?

> The foo_bar_error_quark example in gdbuserror.c clearly should just call
> a standard helper that does all this, just given the pointer to an array
> of data.

Yeah, I guess we can add some convenience for that.

> org.gtk.GDBus.UnmappedGError.Quark0xHEXENCODED_QUARK_NAME_.Code_ERROR_CODE
> Why use a hex encoding for the quark name? Most gerror quark names are
> quite readable, and it would be nice to preserve this in the encoding
> used.

Yeah, we can choose another encoding that hex. The problem is that D-Bus
is very strict about what characters it accepts for names - in
particular, '-' is not accepted. So we'd need some kind of way to escape
things. That's doable though.

> There are no bindings for dbus_connection_flush, which might be nice to
> e.g. ensure that the reply sent from a message handler is immediately
> sent since we may block or whatnot later. Maybe we want this?

Yup, good point.

> gdbusnameowning.c:request_name_cb() - Isn't it racy to not subscribe to
> NameLost/NameAquired until we're processing the request name reply?
> We could have gotten a NameLost message inbetween the RequestName reply
> gotten and request_name_cb being handled by the mainloop, as the
> request_name_cb is called from another idle.

Hmm, yeah. I need to look at that.

> g_dbus_server_new() wants a way to specify the GMainContext to use.
>
> gdbusprivate.c has a bunch of code with a copyright notice from dbus
> that is AFL 2.1 or GPL2+. I.e. not LGPL. This seems like a problem for
> glib integration.

Maybe we can use the code from GVfs for this?

> Are you aware of:
> #define G_PARAM_STATIC_STRINGS (G_PARAM_STATIC_NAME |
> G_PARAM_STATIC_NICK | G_PARAM_STATIC_BLURB)
> It saves some typing.

Good point.

Btw, what do you think about

http://mail.gnome.org/archives/gtk-devel-list/2009-October/msg00075.html

I was going to do this as well.

Thanks,
David


_______________________________________________
gtk-devel-list mailing list
gtk-devel-list@...
http://mail.gnome.org/mailman/listinfo/gtk-devel-list

[Simon McVittie-6]

On Mon, 09 Nov 2009 at 08:52:08 -0500, David Zeuthen wrote:
> Yeah, we can choose another encoding that hex. The problem is that D-Bus
> is very strict about what characters it accepts for names - in
> particular, '-' is not accepted. So we'd need some kind of way to escape
> things.

You're very welcome to copy telepathy-glib's tp_escape_as_identifier()
(HTTP-style escaping using _ instead of %, fitting within all the restricted
character sets of D-Bus, and also the character set of C identifiers) to any
project that needs it. I think so far I've implemented it at least three times
in two languages...

It's a reversible encoding in order to guarantee that it preserves uniqueness,
but so far I've never needed to write a function to decode it (we usually use
it to encode account names, etc. into our object paths, either for uniqueness,
ease of debugging or both).

It's pretty good for strings that are "nearly identifiers", and pessimal for
arbitrary natural-language text (every escaped byte produces three bytes of
output, so it'd triple the size of any non-ASCII text).

    Simon
_______________________________________________
gtk-devel-list mailing list
gtk-devel-list@...
http://mail.gnome.org/mailman/listinfo/gtk-devel-list

[Ryan Lortie]

On Mon, 2009-11-09 at 08:21 -0500, David Zeuthen wrote:
> Sorry for the lag,

No problems :)


> Hmm, I don't like this approach. It means you'd have to pass this
> GDBusFDSet object around in code you use to build/parse the GVariant. In
> particular, your proposed GTypeSerializer would need support for it.
> That's problematic because that one lives in libgobject, not libgio
> (where GDBusFDSet would live).

I disagree.  I'd actually like to keep support for file descriptors out
of the type serialiser entirely.  It's just a little bit "too magic" for
my tastes.

A file descriptor is not normally considered to be a serialisable
object.

> I still think the approach I described works better.
>
> For example, suppose I have a C type mapping that maps a{sh} to a
> GHashTable from strings to gint. I'd like to write code like this
>
>  my_hash = g_hash_table_new (g_str_hash, g_str_equal);
>  g_hash_table_insert (my_hash,
>                       "fd-for-client",
>                       GINT_TO_POINTER (fd_for_client));
>
>  /* need to keep fd_for_client alive until we hand off the hash */

This, for example, is *way* too magic.  I can't ever think of a case
where I store fds in hash tables, much less want to send a hash table
full of fds over DBus.

>    {
>      fd_for_stream1 = dup (fd);
>    }
>
>  /* the value destroy notifier on my_hash closes all fds */
>  g_hash_table_unref (my_hash);

I agree with the general idea of file descriptor ownership you expose
here.  Does that imply that if you send a single file descriptor over
DBus to a handler function that takes an 'int' then you'll have to
close() the fd in the same way as you're responsible to free the hash
table?

> Thoughts?

In general, I think that sending file descriptors over DBus is done
sufficiently rarely that it is not worth it to have support for it *this
deeply* integrated.  In particular, I think that it's definitely not
appropriate to support it in the GTypeSerialiser code.

I also can never imagine a (non-hypothetical) case of wanting to send an
entire hash table of file descriptors over DBus.  Even if I could
imagine one, it would be highly specialised, and I wouldn't mind writing
the code to do it myself -- and it wouldn't be hard to do.

One reason that I particularly like the GDBusFdSet approach is that, on
receive, it allows the user to give a NULL pointer for the out
parameter.  In that case, the file descriptors received from the bus
will be automatically closed and the user needn't worry about doing
anything at all (ie: no fd leaks).

I appreciate that doing the GDBusFdSet thing results in some
uglification of the GDBus API (ie: addition of a new type and some new
functions call variants in some places) but I think this is the
reasonable thing to do.  fd-passing is *very* DBus-specific.  Please
remember that GVariant and GTypeSerialiser are general purpose datatypes
with many potential uses (eg: dconf, GSettings, mmap files, etc) and
that for these other uses, serialising a file descriptor makes no sense
at all.

Cheers

_______________________________________________
gtk-devel-list mailing list
gtk-devel-list@...
http://mail.gnome.org/mailman/listinfo/gtk-devel-list

[Alexander Larsson]

On Mon, 2009-11-09 at 08:52 -0500, David Zeuthen wrote:

> Hey Alex,
>
> On Fri, 2009-11-06 at 21:05 +0100, Alexander Larsson wrote:
> > On Wed, 2009-10-14 at 21:34 -0400, David Zeuthen wrote:
> >
> > >  http://cgit.freedesktop.org/~david/gdbus-standalone/
> >
> > I just read through this for basic review, and I must say that I like
> > it. Very nice stuff. Some comments:
>
> Thanks for reading through it!
>
> > In nautilus I'd like to have an object path which is basically a pointer
> > to another object. Say I have these objects:
> >
> > /nautilus/window/1/tabs/1/view
> > /nautilus/window/1/tabs/2/view
> >
> > Then i want something like:
> >
> > /nautilus/window/1/active_view
> >
> > That always points to the view object in the active tab.
> >
> > Now, i could constantly be re-creating this object all the time, but
> > thats not really very nice. What i instead want to do is basically
> > register a subtree at /nautilus/window/1/active_view which dynamically
> > looks up which tab is active and then re-routes to the right tab.
> >
> > To implement this cleanly I would need to be able to lookup an object at
> > a given path (returning the ifaces, vtables and user data) and to
> > enumerate the children of a given path. All this would be in-client
> > registered objects/subtrees only, and is enough to implement
> > GDBusSubtreeVTable for the "pointer subtree".
>
> Yeah, I guess that's a reasonable thing to add. How about?
>
>  gboolean
>  g_dbus_connection_lookup_object (GDBusConnection *connection,
>                         const gchar           *object_path,
>                         const gchar           *interface_name,
>                         GDBusInterfaceInfo   **out_introspection_data,
>                         GDBusInterfaceVTable **out_vtable,
>                         gpointer              *out_user_data,
>                         guint                 *out_id);
>
>  gboolean
>  g_dbus_connection_lookup_subtree (GDBusConnection *connection,
>                         const gchar           *object_path,
>                         GDBusInterfaceVTable **out_vtable,
>                         GDBusSubtreeFlags     *out_flags,
>                         gpointer              *out_user_data,
>                         guint                 *out_id);

This is not enough to implement GDBusSubtreeIntrospectFunc. I.E. I can't
get the list of interfaces supported at a specific object path.

> > I'd like it to be possible to specify a non-default main context when
> > creating a connection with g_dbus_connection_new() or
> > g_dbus_connection_bus_get_private().
>
> Hmm, yeah, I was thinking about that. It's for specifying the mainloop
> for processing D-Bus messages, right?

Yeah.

> I was actually thinking that it might be nicer to always have a
> dedicated thread for handling D-Bus messages. This would be shared by
> all connections and it would just pop messages to the mainloops for
> method invocations and signals.
>
> This is feasible even if g_thread_init() has not be called, we'd just
> use libpthread directly (which libdbus-1 pulls in already).

Would it be possible to lazily postpone this until e.g. g_threads_init()
was called, or when another thread uses dbus? It seems a bit of overhead
to add a new thread to every simple single-threaded gtk+ process on the
planet.

> > I'm not sure if I missed it, because i didn't try it out, but how do
> > registered object with a a child that is specified by registering a
> > subtree get enumerated when introspecting the registered object?
>
> I'm not sure I understand this question - can you clarify?

In the implementation of enumerate for a path, say /foo/bar, where you
have registered a subtree at /foo/bar/gazonk, how does the
implementation of enumeration of /foo/bar detect that there is a child
called "gazonk" that should be mentioned in the introspect data.


> > org.gtk.GDBus.UnmappedGError.Quark0xHEXENCODED_QUARK_NAME_.Code_ERROR_CODE
> > Why use a hex encoding for the quark name? Most gerror quark names are
> > quite readable, and it would be nice to preserve this in the encoding
> > used.
>
> Yeah, we can choose another encoding that hex. The problem is that D-Bus
> is very strict about what characters it accepts for names - in
> particular, '-' is not accepted. So we'd need some kind of way to escape
> things. That's doable though.

Well, you could hex-escape anything not allowed by d-bus.
You can see what i did in gvfs at
gvfs/common/gdbusutil.c:_dbus_message_new_from_gerror().

> > g_dbus_server_new() wants a way to specify the GMainContext to use.
> >
> > gdbusprivate.c has a bunch of code with a copyright notice from dbus
> > that is AFL 2.1 or GPL2+. I.e. not LGPL. This seems like a problem for
> > glib integration.
>
> Maybe we can use the code from GVfs for this?

Thats got the main loop integration, which i've written from scratch.
However there is no server support.

>
> Btw, what do you think about
>
> http://mail.gnome.org/archives/gtk-devel-list/2009-October/msg00075.html
>
> I was going to do this as well.

Looks good to me.


_______________________________________________
gtk-devel-list mailing list
gtk-devel-list@...
http://mail.gnome.org/mailman/listinfo/gtk-devel-list