New proposal for the jackd/jackdbus mess

[Stéphane Letz]

Hi all,

New much simplified proposal, should be "Fons compatible", hopefully  
"Nedko compatible" (with little work), "Paul one package only  
compatible", others "keep it simple compatible"...

The first "big" conceptual change compared to the current SVN state is  
this new "control IPC" scheme. That is the so called control API can  
be used on client side also. The other conceptual change is that  
"jackd" process is supposed to be an "always running" daemon that  
defines an IPC entry point to be used from "clients". This daemon does  
not "automatically" starts the server (as it does now), but will  when  
requested (either by the "jackd" code directly using C API ) or by the  
request of external control font-end using IPC.


1) Server side:

- libjackserver.so contains:  server code + C control API  + "new" IPC  
control API (server side) +  C Jack API  + IPC Jack API (server side)

- jackd executable is linked with libjackserver.so  (nothing new here)

- backends (ALSA, dummy...) are linked with libjackserver.so  (nothing  
new here)

- a "standalone" client (that wants to embed the server in it's  
process) is linked with libjackserver.so and directly uses the C  
control API  to control/start the server and C Jack API to be a client  
(nothing new here).

2) Client side:

- libjack.so contains :  "new" IPC control API (client side) + IPC  
Jack API (client side)

- clients are linked to libjack.so (nothing new here)

- new control front-end (jackdbus, jackOSC...) are linked to  
libjack.so: they control the server using the IPC control API (client  
side), they can be regular clients using IPC Jack API (client side) to  
deal with connections management and so on...

- a "default" centralized state for the server is always kept in ~/
jackdrc. When a client wants to auto-start, this "default" state is  
used. (this is important to keep in mind)

- libjack may have to start the "jackd" executable using the fork+exec  
way, or the "jackd" process is an "always running + relaunch" process  
(this has to be more defined later on...)

- Qjakctl stays as a regular client, it can still start the "jackd"  
process as usual. It can keep its own way of keeping multiple  
configurations as it does now.

- more sophisticated control front-end (jackdbus, jackOSC...) are now  
regular clients. They can use the IPC control API (client side) for  
more sophisticated control of the server. As regular clients, they  
access the API to control connections... and so on. The important  
thing is that those clients are *obliged* to deal with this "default"  
centralized state. Even if they deal with multiple configs in a new  
format  (XML...) they are supposed to always put a "default" state in  
~/jackdrc for the client "auto-start"  feature to continue working.

- Ardour can still do it's server control mess on its own... ((-:

3) General:

- a single jack2 package is needed. It contains the "jackd" daemon/
server are before.

- "jackdbus" is now conceptually separated from the Jack source code.  
It only uses jack.h + control.h and is linked to libjack.so as any  
regular client. It can be distributed separately as a more  
sophisticated control front-end available, or be available in the  
jack2 package.

- old fashion users can keep their habits

- new "D-Bus aware" guys can explore new fields...

This scheme seems to hopefully solve most of the problems we had, and  
requires only a bit of change for the "jackdbus" front-end to continue  
working, but not much.

Comments?

Stephane








_______________________________________________
Jack-Devel mailing list
Jack-Devel@...
http://lists.jackaudio.org/listinfo.cgi/jack-devel-jackaudio.org

[Rui Nuno Capela]

On Wed, May 20, 2009 10:32, Stéphane Letz wrote:

> Hi all,
>
>
> New much simplified proposal, should be "Fons compatible", hopefully
> "Nedko compatible" (with little work), "Paul one package only
> compatible", others "keep it simple compatible"...
>
> The first "big" conceptual change compared to the current SVN state is
> this new "control IPC" scheme. That is the so called control API can be
> used on client side also. The other conceptual change is that "jackd"
> process is supposed to be an "always running" daemon that defines an IPC
> entry point to be used from "clients". This daemon does not
> "automatically" starts the server (as it does now), but will  when
> requested (either by the "jackd" code directly using C API ) or by the
> request of external control font-end using IPC.
>
>
> 1) Server side:
>
>
> - libjackserver.so contains:  server code + C control API  + "new" IPC
> control API (server side) +  C Jack API  + IPC Jack API (server side)
>
> - jackd executable is linked with libjackserver.so  (nothing new here)
>
>
> - backends (ALSA, dummy...) are linked with libjackserver.so  (nothing
> new here)
>
> - a "standalone" client (that wants to embed the server in it's
> process) is linked with libjackserver.so and directly uses the C control
> API  to control/start the server and C Jack API to be a client
> (nothing new here).
>
>
> 2) Client side:
>
>
> - libjack.so contains :  "new" IPC control API (client side) + IPC
> Jack API (client side)
>
>
> - clients are linked to libjack.so (nothing new here)
>
>
> - new control front-end (jackdbus, jackOSC...) are linked to
> libjack.so: they control the server using the IPC control API (client
> side), they can be regular clients using IPC Jack API (client side) to deal
> with connections management and so on...
>
> - a "default" centralized state for the server is always kept in ~/
> jackdrc. When a client wants to auto-start, this "default" state is used.
> (this is important to keep in mind)
>
>
> - libjack may have to start the "jackd" executable using the fork+exec
> way, or the "jackd" process is an "always running + relaunch" process (this
> has to be more defined later on...)
>
> - Qjakctl stays as a regular client, it can still start the "jackd"
> process as usual. It can keep its own way of keeping multiple
> configurations as it does now.
>
> - more sophisticated control front-end (jackdbus, jackOSC...) are now
> regular clients. They can use the IPC control API (client side) for more
> sophisticated control of the server. As regular clients, they access the
> API to control connections... and so on. The important
> thing is that those clients are *obliged* to deal with this "default"
> centralized state. Even if they deal with multiple configs in a new format
> (XML...) they are supposed to always put a "default" state in
> ~/jackdrc for the client "auto-start"  feature to continue working.
>
>
> - Ardour can still do it's server control mess on its own... ((-:
>
>
> 3) General:
>
>
> - a single jack2 package is needed. It contains the "jackd" daemon/
> server are before.
>
> - "jackdbus" is now conceptually separated from the Jack source code.
> It only uses jack.h + control.h and is linked to libjack.so as any
> regular client. It can be distributed separately as a more sophisticated
> control front-end available, or be available in the jack2 package.
>
> - old fashion users can keep their habits
>
>
> - new "D-Bus aware" guys can explore new fields...
>
>
> This scheme seems to hopefully solve most of the problems we had, and
> requires only a bit of change for the "jackdbus" front-end to continue
> working, but not much.
>
> Comments?
>

no comments.

Stéphane, you're a champion hero!

cheers
--
rncbc aka Rui Nuno Capela
rncbc@...


_______________________________________________
Jack-Devel mailing list
Jack-Devel@...
http://lists.jackaudio.org/listinfo.cgi/jack-devel-jackaudio.org

[Stéphane Letz]

Le 20 mai 09 à 12:20, Krzysztof Foltman a écrit :

> Stéphane Letz wrote:
>
>> This scheme seems to hopefully solve most of the problems we had, and
>> requires only a bit of change for the "jackdbus" front-end to  
>> continue
>> working, but not much.
>
> One obvious problem is that it will be necessary to create yet another
> IPC protocol for control communication between libjack.so and
> libjackserver.so.

Not really, the existing IPC server/client scheme just need to be  
extended with new "calls".

Stephane
_______________________________________________
Jack-Devel mailing list
Jack-Devel@...
http://lists.jackaudio.org/listinfo.cgi/jack-devel-jackaudio.org

[Stéphane Letz]

Le 20 mai 09 à 13:55, Krzysztof Foltman a écrit :

> Stéphane Letz wrote:
>
>> Not really, the existing IPC server/client scheme just need to be
>> extended with new "calls".
>
> Okay.
>
> Then, if you're keeping the "fork and exec" method of starting the
> server, will it in any way be guaranteed than all control clients will
> have the same functionality as the client which initially started  
> jackd?

Why not?
>
> Otherwise, starting a single client will prevent any control  
> application
> started later from being able to do its job.

In what way?

> In non-DBUS JACK the
> application that started jackd had a monopoly on access to its
> stdout/stderr.

Not clear for me.

>
>
> Also, what if the application that did fork and exec crashes?

Same behaviour as now, the server detects crashed client and stops if  
started in "temporary" (-T mode)
or continue to run if started non temporary.

> What if
> the server crashes?

As usual, client can detect that using the "shutdown" callback.

> What if both crash, but other control applications
> are still running?

  "shutdown" callback.

> I think that in any correct solution, the fact that
> an application has started jackd shouldn't give it any more (or less)
> influence on jackd than any other client in the system - otherwise  
> it is
> a race condition. The hypothetical new version of qjackctl should have
> the same feature set no matter if it was started before or after, say,
> Hydrogen.
>
> Can the same IPC protocol you plan to use for the control protocol be
> reused for the out-of-band MIDI SysEx data? (we've been talking about
> that before, and it's really out of scope of control protocol
> discussion, but it might be useful at some point).
>

No idea right now. I think we should focus on the global design.

Stephane
_______________________________________________
Jack-Devel mailing list
Jack-Devel@...
http://lists.jackaudio.org/listinfo.cgi/jack-devel-jackaudio.org

[Fernando Lopez-Lezcano]

On Wed, 2009-05-20 at 11:20 +0100, Krzysztof Foltman wrote:

> Stéphane Letz wrote:
>
> > This scheme seems to hopefully solve most of the problems we had, and
> > requires only a bit of change for the "jackdbus" front-end to continue
> > working, but not much.
>
> One obvious problem is that it will be necessary to create yet another
> IPC protocol for control communication between libjack.so and
> libjackserver.so. Why not use something already proven and with existing
> tools like call monitor, command line interface etc. - that's why D-Bus
> was used in first place.
>
> Don't we already have too much NIH?
>
> Oh well, maybe not ;)

One problem is that it is not necessarily true that dbus will always be
running when you want to run jack. Tying basic stuff like running jack
to an external server (dbus is a server) does not look to me like a good
idea.

-- Fernando


_______________________________________________
Jack-Devel mailing list
Jack-Devel@...
http://lists.jackaudio.org/listinfo.cgi/jack-devel-jackaudio.org

[Paul Davis]

On Mon, May 25, 2009 at 7:27 PM, Fernando Lopez-Lezcano
<nando@...> wrote:

> On Wed, 2009-05-20 at 11:20 +0100, Krzysztof Foltman wrote:
>> Stéphane Letz wrote:
>>
>> > This scheme seems to hopefully solve most of the problems we had, and
>> > requires only a bit of change for the "jackdbus" front-end to continue
>> > working, but not much.
>>
>> One obvious problem is that it will be necessary to create yet another
>> IPC protocol for control communication between libjack.so and
>> libjackserver.so. Why not use something already proven and with existing
>> tools like call monitor, command line interface etc. - that's why D-Bus
>> was used in first place.

Even though for the last couple of years (at least), Stephane has been
doing a lot more work and taking on much more responsibility for JACK
than myself, I still seem to be wearing the hat of JACK's "benign
dictator", so I'm going to cast a couple of things in stone that I
don't think have been made clear enough in the email discussion (not
true on IRC, but this email exchange has also attracted a lot of
written words).

1) JACK is intentionally a multi-platform technology, and still
attempts to use only POSIX-defined system interfaces unless the
required components are not available or so poorly functioning that an
alternative is required.

2) Because of this, JACK itself is not going to use platform-specific
technologies for any kind of public interfaces. JACK on OS X and
Windows uses non-POSIX technologies but not in any way that is visible
to a typical user or to JACK clients. This observation rules out the
use of *any* platform-specific IPC system, whether its D-Bus,
Microsoft's COM or various parts of the Cocoa API on OS X. The fact
that D-Bus can be run on Windows or OS X doesn't change this - no
developer on those platforms would consider it to be a part of those
platforms or consider it viable to require users to install and use
it.

3) The discussions about the control API have managed to gloss over
the specific problems that were outlined at LAC2008 (Koln) in a way
that is probably not helpful. So I will restate my recollection of
them here. This is why we wanted to extend the JACK API (in no
particular order):

     * better desktop integration when desired
     * provide the ability to stop/start/reconfigure the server,
including adding/removing/changing backends and/or devices in use
     * enable to use of "profiles" to define startup parameters for
the server, thus allowing any JACK client to correctly start
           the server without a hack like ~/.jackdrc (but
back-compatibility with ~/.jackdrc was considered important)
     * provide an easier way to load internal clients
     * provide a better way to present messages to the user, probably
but not necessarily via a control application

So, these are the things that need to be considered "cast in stone".
Nothing should prevent anyone from integrating the JACK "control" and
"configuration" APIs with D-Bus, OSC, Freedesktop.org/XDG, Cocoa, COM,
or any other "broader" system. But such integration should not and
will not be a part of the "JACK" implementation(s) "core". It is
possible that we may even distribute some code  that performs this
integration for specific protocols as part of JACK. But it should not
be thought of as a core part of the system, merely a bridge to other
things to make JACK a bit more useful and a bit easier to use, similar
to the way JACK currently uses libsndfile.

I will write separately on some specifics of the proposal that Stephane posted.
_______________________________________________
Jack-Devel mailing list
Jack-Devel@...
http://lists.jackaudio.org/listinfo.cgi/jack-devel-jackaudio.org