Fixing the Gobject Introspection mess

[Josselin Mouette]

Hi,

with an increasing number of packages providing introspection data for
Gobject, each one doing things its own way, it’s starting to be a big
mess. I’d like to fix this mess before we have a hundred different
packages, all behaving differently.

Which is why I’m proposing a Gobject-introspection mini-policy. And of
course, to implement it in existing packages.


1. Package layout

GObject-introspection packages provide introspection data
in /usr/share/gir-1.0/Foo-X.Y.gir, and the
optional /usr/lib/girepository-1.0/Foo-X.Y.typelib.

The packages should be architecture-dependent.


2. Naming scheme

The package should be named gir1.0-foo-X.Y. For example, the package
containing WebKit-1.0.gir will be named gir1.0-webkit-1.0.

Giant repositories of dozens of unrelated introspection data should be
avoided. (Under this rationale, gobject-introspection-repository will be
split.) However, related libraries that are known to evolve together can
live in the same package (example: Gst*-0.10).

If, alternatively, the introspection data belongs in the same source
package as the library it references, it can be put in the same binary
package. In this case, it must feature a Provides: field corresponding
to the name of the introspection data. For example, libfoo2.0-2
containing libfoo-2.0.so.2 and Foo-2.0.gir must provide gir1.0-foo-2.0.

Introspection packages belong in section libs for the moment. If there
are enough packages to justify it, a new section will be requested to
the FTP masters.


3. Dependencies of introspection packages

Introspection packages must depend on the libraries they reference, with
a sufficient version for the symbols they reference.

For that effect, I propose to introduce a new helper, dh_gir, in the
gobject-introspection package (which is already a build-dependency for
introspection packages). It would wrap dpkg-shlibdeps (possibly using
its internals until the needed interfaces are exported) so that the
dependencies are the same as those of similar ELF binaries.

Introspection packages must depend on other introspection packages that
are referenced through <include> statements. The helper should generate
such dependencies as well.


4. Dependencies on introspection packages

Currently, there are only Seed (JavaScript) scripts to use these
introspection packages. In the future, there might also be Python or
other interpreted languages.

Generating dependencies automatically for interpreted languages is not
reliable. Therefore, these packages must depend by hand on the
appropriate gir1.0-* packages. The interpreters themselves don’t need to
depend on packages they don’t use directly.



Thoughts anyone? I’ll start working on dh_gir as described if there is
consensus on the proposed policy. Once implemented in a few packages,
the policy would be put in a more formal way and added to
gobject-introspection.

Cheers,
--
 .''`.      Josselin Mouette
: :' :
`. `'   “I recommend you to learn English in hope that you in
  `-     future understand things”  -- Jörg Schilling

[Julien Cristau-6]

On Fri, Sep 25, 2009 at 13:32:54 +0200, Josselin Mouette wrote:

> 2. Naming scheme
>
> The package should be named gir1.0-foo-X.Y. For example, the package
> containing WebKit-1.0.gir will be named gir1.0-webkit-1.0.
>
> Giant repositories of dozens of unrelated introspection data should be
> avoided. (Under this rationale, gobject-introspection-repository will be
> split.) However, related libraries that are known to evolve together can
> live in the same package (example: Gst*-0.10).
>
> If, alternatively, the introspection data belongs in the same source
> package as the library it references, it can be put in the same binary
> package. In this case, it must feature a Provides: field corresponding
> to the name of the introspection data. For example, libfoo2.0-2
> containing libfoo-2.0.so.2 and Foo-2.0.gir must provide gir1.0-foo-2.0.
>

Doesn't this break co-installability of libfoo2.0-X and libfoo2.0-Y, if
both install Foo-2.0.gir?

Cheers,
Julien


--
To UNSUBSCRIBE, email to debian-gtk-gnome-REQUEST@...
with a subject of "unsubscribe". Trouble? Contact listmaster@...

[Josselin Mouette]

Le vendredi 25 septembre 2009 à 13:44 +0200, Julien Cristau a écrit :

> On Fri, Sep 25, 2009 at 13:32:54 +0200, Josselin Mouette wrote:
>
> > If, alternatively, the introspection data belongs in the same source
> > package as the library it references, it can be put in the same binary
> > package. In this case, it must feature a Provides: field corresponding
> > to the name of the introspection data. For example, libfoo2.0-2
> > containing libfoo-2.0.so.2 and Foo-2.0.gir must provide gir1.0-foo-2.0.
> >
> Doesn't this break co-installability of libfoo2.0-X and libfoo2.0-Y, if
> both install Foo-2.0.gir?

Good point. Maybe we can recommend this solution only for libraries
where this is not expected to happen.

Or, given the (large) size of introspection stuff, to mandate having it
always split.

--
 .''`.      Josselin Mouette
: :' :
`. `'   “I recommend you to learn English in hope that you in
  `-     future understand things”  -- Jörg Schilling

[Sebastian Dröge-4]

Am Freitag, den 25.09.2009, 13:32 +0200 schrieb Josselin Mouette:

>
> [...]
>
> 1. Package layout
>
> GObject-introspection packages provide introspection data
> in /usr/share/gir-1.0/Foo-X.Y.gir, and the
> optional /usr/lib/girepository-1.0/Foo-X.Y.typelib.
>
> The packages should be architecture-dependent.

Only the typelib file actually is meant to be architecture-dependent
(right, the GIR file could be arch-dep too if the headers and API parts
of the sources (GObject properties/signals) are different from arch to
arch. But who does that?!).

Also the GIR file is only meant for build time things so it could simply
be placed in the -dev packages (as I've done so far).

The typelib file is another issue though...

> 2. Naming scheme
>
> The package should be named gir1.0-foo-X.Y. For example, the package
> containing WebKit-1.0.gir will be named gir1.0-webkit-1.0.
>
> Giant repositories of dozens of unrelated introspection data should be
> avoided. (Under this rationale, gobject-introspection-repository will be
> split.) However, related libraries that are known to evolve together can
> live in the same package (example: Gst*-0.10).
>
> If, alternatively, the introspection data belongs in the same source
> package as the library it references, it can be put in the same binary
> package. In this case, it must feature a Provides: field corresponding
> to the name of the introspection data. For example, libfoo2.0-2
> containing libfoo-2.0.so.2 and Foo-2.0.gir must provide gir1.0-foo-2.0.

Putting the typelib files in the shared library packages will create
conflicts for soname changes. Not good (I know I placed them there in
some packages). Creating a new package just for the typelib file is not
good either ;)

> [...]

Everything else sounds good to me.

[Marco d'Itri]

On Sep 25, Julien Cristau <jcristau@...> wrote:

> Doesn't this break co-installability of libfoo2.0-X and libfoo2.0-Y, if
> both install Foo-2.0.gir?
And what about multiarch?

--
ciao,
Marco

[Josselin Mouette]

Le samedi 26 septembre 2009 à 07:56 +0200, Sebastian Dröge a écrit :

> Am Freitag, den 25.09.2009, 13:32 +0200 schrieb Josselin Mouette:
> > 1. Package layout
> >
> > GObject-introspection packages provide introspection data
> > in /usr/share/gir-1.0/Foo-X.Y.gir, and the
> > optional /usr/lib/girepository-1.0/Foo-X.Y.typelib.
> >
> > The packages should be architecture-dependent.
>
> Only the typelib file actually is meant to be architecture-dependent
> (right, the GIR file could be arch-dep too if the headers and API parts
> of the sources (GObject properties/signals) are different from arch to
> arch. But who does that?!).

This has already happened, and this will happen again. For the same
reason we make headers architecture-dependent, I think we should make
GIR files the same.

> Also the GIR file is only meant for build time things so it could simply
> be placed in the -dev packages (as I've done so far).

There is no “build time” for interpreters. What do you mean exactly?

> Putting the typelib files in the shared library packages will create
> conflicts for soname changes. Not good (I know I placed them there in
> some packages). Creating a new package just for the typelib file is not
> good either ;)

Agreed. We should probably mandate to always split the introspection
files in a separate package.

Cheers,
--
 .''`.      Josselin Mouette
: :' :
`. `'   “I recommend you to learn English in hope that you in
  `-     future understand things”  -- Jörg Schilling

[Sebastian Dröge-4]

Am Samstag, den 26.09.2009, 11:54 +0200 schrieb Josselin Mouette:

> Le samedi 26 septembre 2009 à 07:56 +0200, Sebastian Dröge a écrit :
> > Am Freitag, den 25.09.2009, 13:32 +0200 schrieb Josselin Mouette:
> > > 1. Package layout
> > >
> > > GObject-introspection packages provide introspection data
> > > in /usr/share/gir-1.0/Foo-X.Y.gir, and the
> > > optional /usr/lib/girepository-1.0/Foo-X.Y.typelib.
> > >
> > > The packages should be architecture-dependent.
> >
> > Only the typelib file actually is meant to be architecture-dependent
> > (right, the GIR file could be arch-dep too if the headers and API parts
> > of the sources (GObject properties/signals) are different from arch to
> > arch. But who does that?!).
>
> This has already happened, and this will happen again. For the same
> reason we make headers architecture-dependent, I think we should make
> GIR files the same.

Ok, with my proposal to put them into the -dev packages that is no
problem then ;)

> > Also the GIR file is only meant for build time things so it could simply
> > be placed in the -dev packages (as I've done so far).
>
> There is no “build time” for interpreters. What do you mean exactly?

Interpreters should use the typelib file and not the GIR file. The GIR
file is meant for generating bindings (like the current Python/C++/C#
bindings) while the typelib file is meant to be used at runtime to
generated bindings on the fly.

> > Putting the typelib files in the shared library packages will create
> > conflicts for soname changes. Not good (I know I placed them there in
> > some packages). Creating a new package just for the typelib file is not
> > good either ;)
>
> Agreed. We should probably mandate to always split the introspection
> files in a separate package.

So you would always put the GIR and typelib files into a single separate
package? Or only the typelib file? IMHO only the latter would make
sense, the GIR file can/should be in the -dev package, and if only
because of space savings.

If you meant the former, shall we also create separate packages for Vala
bindings? They're very similar to the GIR files but are also only needed
at build time ;)

[Josselin Mouette]

Le samedi 26 septembre 2009 à 13:12 +0200, Sebastian Dröge a écrit :
> > There is no “build time” for interpreters. What do you mean exactly?
>
> Interpreters should use the typelib file and not the GIR file. The GIR
> file is meant for generating bindings (like the current Python/C++/C#
> bindings) while the typelib file is meant to be used at runtime to
> generated bindings on the fly.

OK, then I agree the .gir file should go in a separate development
package.

But that will make dependency computation more difficult, since it will
have to be done from the .typelib file. Time to dig deeper into the
libgirepository API…

> So you would always put the GIR and typelib files into a single separate
> package? Or only the typelib file? IMHO only the latter would make
> sense, the GIR file can/should be in the -dev package, and if only
> because of space savings.

Yes, agreed.

--
 .''`.      Josselin Mouette
: :' :
`. `'   “I recommend you to learn English in hope that you in
  `-     future understand things”  -- Jörg Schilling

[Josselin Mouette]

After an insightful discussion with Sebastian, and taking into account
other suggestions from the list, here is a new proposal.



1. Directory layout

GObject-introspection data is generally provided in two formats:
      * XML format in /usr/share/gir-1.0/Foo-X.Y.gir
      * binary format in /usr/lib/girepository-1.0/Foo-X.Y.typelib


2. Binary introspection packages

The binary typelib file must be put in an architecture-dependent package
named after its namespace. The name should be gir1.0-NAMESPACE-VERSION.
For example, the package containing WebKit-1.0.typelib will be named
gir1.0-webkit-1.0.

Giant repositories of unrelated introspection data should be avoided.
(Under this rationale, gobject-introspection-repository will be split.)
However, related libraries that are known to evolve together can live in
the same package (example: Gst*-0.10).

Introspection packages belong in section libs for the moment. If there
are enough packages to justify it, a new section will be requested to
the FTP masters.


3. XML introspection data

The XML format introspection must be shipped in another
architecture-dependent* package of the same source.

              * This is so that it is guaranteed to be accessible at
                build time by the tool that will compute the
                dependencies for the .typelib files.

If the source package also contains the library itself, this data should
be in the development binary package. If the introspection data is split
from the library source (e.g. for gobject-instrospection-repository), a
separate package containing this XML data can be created, in section
libdevel.

The package containing the XML data doesn’t need to depend on the
introspection package. It can contain XML data for several unrelated
libraries, since in the end it doesn’t depend on them.


4. Dependencies of introspection packages

Introspection packages must depend on the libraries they reference, with
a sufficient version for the symbols they reference.

For that effect, I propose to introduce a new helper, dh_girepository,
in the gobject-introspection package (which is already a
build-dependency for introspection packages). It would wrap
dpkg-shlibdeps so that the dependencies are the same as those of similar
ELF binaries.

Introspection packages must depend on other introspection packages that
are referenced through <include> statements. The helper should generate
such dependencies as well.

Packages containing the XML data don’t need any specific new
dependencies.


5. Dependencies on introspection packages

Currently, there are only Seed (JavaScript) scripts to use these
introspection packages. In the future, there might also be Python or
other interpreted languages.

Generating dependencies automatically for interpreted languages is not
reliable. Therefore, these packages must depend by hand on the
appropriate gir1.0-* packages. The interpreters themselves don’t need to
depend on packages they don’t use directly.


6. Example

Suppose that libfoo-2.0 is an API built on libbar-3.0. The libfoo-2.0
source generates the following files, put in the following packages:
        libfoo-2.0-3 /usr/lib/libfoo-2.0.so.3*
        libfoo-2.0-dev /usr/lib/libfoo-2.0.so (and other usual stuff)
        libfoo-2.0-dev /usr/share/gir-1.0/Foo-2.0.gir
        gir1.0-foo-2.0 /usr/lib/girepository-1.0/Foo-2.0.typelib

libfoo-2.0-dev Depends: libfoo-2.0-3, libbar-3.0-dev
gir1.0-foo-2.0 Depends: ${gir:Depends} which expands to:
libfoo-2.0-3, gir1.0-bar-3.0

If foobar is a package containing the following JS script:
        #! /usr/bin/seed
        Foo = imports.gi.Foo;
        // Stuff that uses the Foo 2.0 API

Then foobar Depends: gir1.0-foo-2.0



I’ve started preliminary work on dh_girepository. The first version will
be a bit hackish and will rely on building an intermediate binary
referencing the appropriate symbols that will be fed to dpkg-shlibdeps.
Doing better will require extending dpkg-shlibdeps so that it accepts
another input format than ELF (it doesn’t have to be the GObject XML
format, it could be anything easy to generate).

--
 .''`.      Josselin Mouette
: :' :
`. `'   “I recommend you to learn English in hope that you in
  `-     future understand things”  -- Jörg Schilling

[Paul Wise-3]

2009/9/26 Josselin Mouette <joss@...>:

> 1. Directory layout
>
> GObject-introspection data is generally provided in two formats:
>      * XML format in /usr/share/gir-1.0/Foo-X.Y.gir
>      * binary format in /usr/lib/girepository-1.0/Foo-X.Y.typelib
...
> 6. Example
>
> Suppose that libfoo-2.0 is an API built on libbar-3.0. The libfoo-2.0
> source generates the following files, put in the following packages:
>        libfoo-2.0-3 /usr/lib/libfoo-2.0.so.3*
>        libfoo-2.0-dev /usr/lib/libfoo-2.0.so (and other usual stuff)
>        libfoo-2.0-dev /usr/share/gir-1.0/Foo-2.0.gir
>        gir1.0-foo-2.0 /usr/lib/girepository-1.0/Foo-2.0.typelib

Would this proposal need to be adjusted for multiarch support?

If so, perhaps just insert an arch triple?

 /usr/lib/<arch>/girepository-1.0/Foo-2.0.typelib

--
bye,
pabs

http://wiki.debian.org/PaulWise


--
To UNSUBSCRIBE, email to debian-gtk-gnome-REQUEST@...
with a subject of "unsubscribe". Trouble? Contact listmaster@...

[Hendrik Sattler-3]

Am Sonntag 27 September 2009 04:59:38 schrieb Paul Wise:

> 2009/9/26 Josselin Mouette <joss@...>:
> > 1. Directory layout
> >
> > GObject-introspection data is generally provided in two formats:
> >      * XML format in /usr/share/gir-1.0/Foo-X.Y.gir
> >      * binary format in /usr/lib/girepository-1.0/Foo-X.Y.typelib
>
> ...
>
> > 6. Example
> >
> > Suppose that libfoo-2.0 is an API built on libbar-3.0. The libfoo-2.0
> > source generates the following files, put in the following packages:
> >        libfoo-2.0-3 /usr/lib/libfoo-2.0.so.3*
> >        libfoo-2.0-dev /usr/lib/libfoo-2.0.so (and other usual stuff)
> >        libfoo-2.0-dev /usr/share/gir-1.0/Foo-2.0.gir
> >        gir1.0-foo-2.0 /usr/lib/girepository-1.0/Foo-2.0.typelib
>
> Would this proposal need to be adjusted for multiarch support?
>
> If so, perhaps just insert an arch triple?
>
>  /usr/lib/<arch>/girepository-1.0/Foo-2.0.typelib

Isn't the lib directory for each arch already different?

I just question the need for a seperate package for _one_ typelib file.

HS


--
To UNSUBSCRIBE, email to debian-gtk-gnome-REQUEST@...
with a subject of "unsubscribe". Trouble? Contact listmaster@...

[Josselin Mouette]

Le dimanche 27 septembre 2009 à 10:59 +0800, Paul Wise a écrit :
> >        gir1.0-foo-2.0 /usr/lib/girepository-1.0/Foo-2.0.typelib
>
> Would this proposal need to be adjusted for multiarch support?

I’m not sure. It will be mostly used by interpreters, which themselves
don’t need support for multiarch with the current state of the proposal.

Cheers,
--
 .''`.      Josselin Mouette
: :' :
`. `'   “I recommend you to learn English in hope that you in
  `-     future understand things”  -- Jörg Schilling