gobject-performance

[Benjamin Otte]

Hi,

I'd like to move the work done on the gobject-performance branch to
master now that 2.22.0 is out. It contains tremendous improvements for
threaded applications and even noticably speeds up non-threaded
performance. The patches on the branch have been developed, reviewed
and tested by at least Alex Larsson, Edward Hevery and me, so it
should be sufficiently reviewed.
I'd have liked to get Tim to review it, but he seems to be quite busy
now and IMO we threw enough eyes and CPU time at the issue to be sure
there is no regressions. (From the tests we added, I'd even say it's
more stable than master).

I'd also like to get it merged while we're far away from the next
release, so it gets enough testing, as these changes are quite deep
down the stack and we want to find remaining issues with them before
we release the next stable.

So can we merge the branch or is there anything that we should take
care of before?

Benjamin
_______________________________________________
gtk-devel-list mailing list
gtk-devel-list@...
http://mail.gnome.org/mailman/listinfo/gtk-devel-list

[A. Walton-2]

On Thu, Sep 24, 2009 at 9:37 AM, Benjamin Otte <otte@...> wrote:

> Hi,
>
> I'd like to move the work done on the gobject-performance branch to
> master now that 2.22.0 is out. It contains tremendous improvements for
> threaded applications and even noticably speeds up non-threaded
> performance. The patches on the branch have been developed, reviewed
> and tested by at least Alex Larsson, Edward Hevery and me, so it
> should be sufficiently reviewed.
> I'd have liked to get Tim to review it, but he seems to be quite busy
> now and IMO we threw enough eyes and CPU time at the issue to be sure
> there is no regressions. (From the tests we added, I'd even say it's
> more stable than master).
>
> I'd also like to get it merged while we're far away from the next
> release, so it gets enough testing, as these changes are quite deep
> down the stack and we want to find remaining issues with them before
> we release the next stable.
>
> So can we merge the branch or is there anything that we should take
> care of before?

Just have a small question on the subject: has anyone performed any
library or application benchmarks to see how much this means in the
real world and not just for the threaded microbenchmarks that you
wrote? The non-threaded speedups are pretty impressive on their own,
but it'd be nice to see what we could expect from e.g. GStreamer with
these changes applied.

Thanks,
-A. Walton.

>
> Benjamin
> _______________________________________________
> gtk-devel-list mailing list
> gtk-devel-list@...
> http://mail.gnome.org/mailman/listinfo/gtk-devel-list
_______________________________________________
gtk-devel-list mailing list
gtk-devel-list@...
http://mail.gnome.org/mailman/listinfo/gtk-devel-list

[Benjamin Otte]

On Fri, Sep 25, 2009 at 1:16 AM, A. Walton <awalton@...> wrote:
> Just have a small question on the subject: has anyone performed any
> library or application benchmarks to see how much this means in the
> real world and not just for the threaded microbenchmarks that you
> wrote? The non-threaded speedups are pretty impressive on their own,
> but it'd be nice to see what we could expect from e.g. GStreamer with
> these changes applied.
>
The motivations for the patches have in all cases been real world
problems, though I never explicitly benchmarked the effects
afterwards, as I was happy just looking at my microbenchmarks.

The work on lockless O(1) interface lookups was motivated by trying to
sort a huge treeview and getting a huge overhead from it. I just
tested my simple liststore script again (attached here), and sorting
the numbers column on that liststore is roughly 25% faster now. So I'd
expect you to see these improvements in multithreaded apps that sort
huge treeviews: file chooser and nautilus with lots of files, probably
evolution in huge dirs.

Speeding up g_type_class_ref() was not motivated by code that's
already out in the wild, but by the desire to make ever smaller
objects into real objects, that is currently not feasible due to this
overhead. Examples here are an mpegps (mpegts?) parser the GStreamer
guys had been working on, that was having 70% overhead just creating
lots of small GstBuffers, Philip Van Hoof's work on Iterable
interfaces or Vala automatically making a GObject out of Vala classes.
So no real code I can point you at, but definitely a huge demand.

The big problem with "show me real-life benchmarks" type questions is
that developers usually work around visible limitations in lower
layers, so you cannot see any immediate results. Instead, developers
can - if they realize the limitations are gone - get rid of their
workarounds or use features that previously just didn't work. And then
people can get rid of APIs like for example g_type_class_peek_static()
(for g_type_class_ref()), ETable (for GtkTreeView) or libgstvideo (for
Cairo).

Benjamin


_______________________________________________
gtk-devel-list mailing list
gtk-devel-list@...
http://mail.gnome.org/mailman/listinfo/gtk-devel-list

[Wim Taymans-2]

On Fri, 2009-09-25 at 10:00 +0200, Benjamin Otte wrote:

> On Fri, Sep 25, 2009 at 1:16 AM, A. Walton <awalton@...> wrote:
> > Just have a small question on the subject: has anyone performed any
> > library or application benchmarks to see how much this means in the
> > real world and not just for the threaded microbenchmarks that you
> > wrote? The non-threaded speedups are pretty impressive on their own,
> > but it'd be nice to see what we could expect from e.g. GStreamer with
> > these changes applied.
> >
> The motivations for the patches have in all cases been real world
> problems, though I never explicitly benchmarked the effects
> afterwards, as I was happy just looking at my microbenchmarks.

Another real world problem is explained (with a quite detailed analysis
of the profiling info) in this bug report:
https://bugzilla.gnome.org/show_bug.cgi?id=585375

I hope we can get some feedback on how these optimisations improve
things.

Wim

_______________________________________________
gtk-devel-list mailing list
gtk-devel-list@...
http://mail.gnome.org/mailman/listinfo/gtk-devel-list

[Alexander Larsson]

On Thu, 2009-09-24 at 15:37 +0200, Benjamin Otte wrote:

> Hi,
>
> I'd like to move the work done on the gobject-performance branch to
> master now that 2.22.0 is out. It contains tremendous improvements for
> threaded applications and even noticably speeds up non-threaded
> performance. The patches on the branch have been developed, reviewed
> and tested by at least Alex Larsson, Edward Hevery and me, so it
> should be sufficiently reviewed.
> I'd have liked to get Tim to review it, but he seems to be quite busy
> now and IMO we threw enough eyes and CPU time at the issue to be sure
> there is no regressions. (From the tests we added, I'd even say it's
> more stable than master).
>
> I'd also like to get it merged while we're far away from the next
> release, so it gets enough testing, as these changes are quite deep
> down the stack and we want to find remaining issues with them before
> we release the next stable.
>
> So can we merge the branch or is there anything that we should take
> care of before?

Man, this situation with such low maintainer activity in gobject is
getting out of hand. We really need a way to get important fixes like
this into gobject in finite amount of time. After all gobject is the
core of the whole gtk and gnome platform, if we can't get fixes into it
then we're in a very bad shape.

I realize that GObject is a very complex and important piece of software
and that timj knows it much better than anyone else. However, we just
cannot totally stop work on it when timj is busy with other things.

Now, some stuff in the gobject-performance branch is pretty trivial, but
some of it is extremely complex (lock free stuff, atomic refcounting
etc), so I don't really like to just "wait a bit and then commit without
timj approval". However, I'd like to have *some* way of getting things
in, while still having a decent chance of finding problems before they
land in glib.

What about introducing some sort of ack rules. If you can get, say,
three detailed reviews with acks from well known developers (core
people, maintainers, etc) then we can commit stuff during to the
unstable branch.

Opinions?


_______________________________________________
gtk-devel-list mailing list
gtk-devel-list@...
http://mail.gnome.org/mailman/listinfo/gtk-devel-list

[Philip Van Hoof]

On Wed, 2009-09-30 at 15:06 +0200, Alexander Larsson wrote:

[CUT]

> On Thu, 2009-09-24 at 15:37 +0200, Benjamin Otte wrote:

> What about introducing some sort of ack rules. If you can get, say,
> three detailed reviews with acks from well known developers (core
> people, maintainers, etc) then we can commit stuff during to the
> unstable branch.

I'm not petitioning to become such a reviewer, as I don't think I'd be a
very good one. But I like this idea very much.

I propose to ask Jürg Billeter to help review gobject patches. With his
work on Vala Jürg seems to have an in-depth knowledge of gobject.

> Opinions?

Thanks a lot for trying to get things moving here, Alex.

Appreciated.


--
Philip Van Hoof, freelance software developer
home: me at pvanhoof dot be
gnome: pvanhoof at gnome dot org
http://pvanhoof.be/blog
http://codeminded.be

_______________________________________________
gtk-devel-list mailing list
gtk-devel-list@...
http://mail.gnome.org/mailman/listinfo/gtk-devel-list

[Stefan Kost]

A. Walton schrieb:

> On Thu, Sep 24, 2009 at 9:37 AM, Benjamin Otte <otte@...> wrote:
>  
>> Hi,
>>
>> I'd like to move the work done on the gobject-performance branch to
>> master now that 2.22.0 is out. It contains tremendous improvements for
>> threaded applications and even noticably speeds up non-threaded
>> performance. The patches on the branch have been developed, reviewed
>> and tested by at least Alex Larsson, Edward Hevery and me, so it
>> should be sufficiently reviewed.
>> I'd have liked to get Tim to review it, but he seems to be quite busy
>> now and IMO we threw enough eyes and CPU time at the issue to be sure
>> there is no regressions. (From the tests we added, I'd even say it's
>> more stable than master).
>>
>> I'd also like to get it merged while we're far away from the next
>> release, so it gets enough testing, as these changes are quite deep
>> down the stack and we want to find remaining issues with them before
>> we release the next stable.
>>
>> So can we merge the branch or is there anything that we should take
>> care of before?
>>    
>
> Just have a small question on the subject: has anyone performed any
> library or application benchmarks to see how much this means in the
> real world and not just for the threaded microbenchmarks that you
> wrote? The non-threaded speedups are pretty impressive on their own,
> but it'd be nice to see what we could expect from e.g. GStreamer with
> these changes applied.
>
> Thanks,
> -A. Walton.
>
>  

I have a quite complex use-case, that is loading and playing bigger
songs in buzztard. Its probably not a common use case, but there you can
measure the difference (values are best out of 3).
GST_DEBUG_NO_COLOR=1 GST_DEBUG="bt-core:3" ~/buzztard/bin/buzztard-cmd
2>&1 --command=play
--input-file=../share/buzztard/songs/buzz/Aehnatron-noPrimiFun.bmw |
grep async
before:
0:00:02.949420960 16963  0x8712538 INFO                 bt-core
song.c:692:bt_song_play: ->PAUSED needs async wait
0:00:06.205656656 16963  0x8712538 INFO                 bt-core
song.c:480:on_song_async_done: async state-change done
after:
0:00:02.760566012 17054  0x9e14538 INFO                 bt-core
song.c:692:bt_song_play: ->PAUSED needs async wait
0:00:06.065655074 17054  0x9e14538 INFO                 bt-core
song.c:480:on_song_async_done: async state-change done

number vary, but its like a 0.15 sec saving. That might not sound very
impressive, but then I see no point in not making the savings if we can.
As people have already invested time and energy to come up with the
changes, also the remaining changes should be merge to get more wider
testing.

Stefan
_______________________________________________
gtk-devel-list mailing list
gtk-devel-list@...
http://mail.gnome.org/mailman/listinfo/gtk-devel-list