netjack redux

[Paul Davis]

Its time to do something about the dreadful situation with NetJACK.
More and more people are asking about using it, and it is more or less
impossible to provide them with any coherent answer about what to do.
Here is my reading of the situation as it stands today:

1) we have 2 implementations called "netjack", both of which are
incompatible with each other
2) netjack1, primarily written by torben, is only part of jack1
3) netjack2, primarily written by nedko, is only part of jack2
4) netjack1 has been aggressively developed over the last 6-10 months
to give far superior network performance than netjack2, both over
local and long haul networks
 4a) the long haul improvements come mostly from using the CELT codec
 4b) the LAN improvements have come in part from many discussions with
Alex Carot, one of the main authors of SoundJack (presented at AES and
elsewhere)
5) netjack2 has a discovery protocol, so that setup on a local network
can be more automatic and "avahi"-like. netjack1 has no such protocol
- master & slave must be explicitly connected to each other.
6) torben has set up his own git branch called jacknone that merges
jack2 with netjack1
7) torben has been hard to reach for a while.

I would welcome any corrections to my portrayal of the situation as I
have described above. Assuming for the moment that it is correct ...

... this situation cannot be allowed to continue. Network audio is
rising rapidly in importance, and we have the technology to do what
people need done. What we don't have is a workable, coherent
explanation of what to do for people who are not familiar with all 7
points above, and we don't have a GUI for any platform to make setup
easy.

I would like it best if the principals in this, nedko and torben,
could resolve this conflicting aspects of the situation on their own
within the month of October. I personally doubt that this will happen
for a number of reasons.

After that, I plan to consult privately with Stephane and come to a
decision about how to resolve it without necessarily involving either
of the two principal authors. This would obviously be unfortunate. The
goal will be to have a single version of netjack that works with jack1
and jack2 installs, across Linux, OS X and Windows.

Although I am wary of making this next comment, I think that future
success relies on honesty here, and so I will offer my own rather
strong opinion: I believe it was a mistake for Stephane to have
allowed netjack2 into Jack2 even though the reasons for netjack2's
development were reasonably clear and defensible. That doesn't mean
that I am married to netjack1 or that I could not be persuaded to
think differently about the situation. It just means that *in
retrospect* it doesn't seem to have been the right decision. Change my
mind if you feel I am wrong, or convince enough others that my opinion
doesn't matter.

Lets move forward on this and make the amazing work Stephane has done
bringing JACK to OS X and Windows even more useful to JACK users

--p
_______________________________________________
Jack-Devel mailing list
Jack-Devel@...
http://lists.jackaudio.org/listinfo.cgi/jack-devel-jackaudio.org

[Fernando Lopez-Lezcano]

On Tue, 2009-10-06 at 21:12 -0400, Paul Davis wrote:
> 5) netjack2 has a discovery protocol, so that setup on a local network
> can be more automatic and "avahi"-like. netjack1 has no such protocol
> - master & slave must be explicitly connected to each other.

Just a comment on this point. I'm currently looking at how to run
netjack2... ('cause we run jack2 here at ccrma).

I don't currently have multicast enabled and I have to find how to set
that up. AFAICT in netjack2 it is only possible to do auto discovery
which is problematic right now (maybe that has changed since the last
time I looked at this?). Or am I missing something? for whoever keeps
track of this, I would really like the option of just typing an explicit
address and that's it (I know, quaint and antiquated :-)

-- Fernando


_______________________________________________
Jack-Devel mailing list
Jack-Devel@...
http://lists.jackaudio.org/listinfo.cgi/jack-devel-jackaudio.org

[Stéphane Letz]

Le 7 oct. 2009 à 03:12, Paul Davis a écrit :

> Its time to do something about the dreadful situation with NetJACK.
> More and more people are asking about using it, and it is more or less
> impossible to provide them with any coherent answer about what to do.
> Here is my reading of the situation as it stands today:
>
> 1) we have 2 implementations called "netjack", both of which are
> incompatible with each other
> 2) netjack1, primarily written by torben, is only part of jack1
> 3) netjack2, primarily written by nedko, is only part of jack2

Nope, netjack2 was the work of Romain Moret who did a 6 months work  
period here at Grame last year. I am supposed to understand most of  
his code.

> 4) netjack1 has been aggressively developed over the last 6-10 months
> to give far superior network performance than netjack2, both over
> local and long haul networks
> 4a) the long haul improvements come mostly from using the CELT codec
> 4b) the LAN improvements have come in part from many discussions with
> Alex Carot, one of the main authors of SoundJack (presented at AES and
> elsewhere)
> 5) netjack2 has a discovery protocol, so that setup on a local network
> can be more automatic and "avahi"-like. netjack1 has no such protocol
> - master & slave must be explicitly connected to each other.
> 6) torben has set up his own git branch called jacknone that merges
> jack2 with netjack1
> 7) torben has been hard to reach for a while.
>
> I would welcome any corrections to my portrayal of the situation as I
> have described above. Assuming for the moment that it is correct ...
>
> ... this situation cannot be allowed to continue. Network audio is
> rising rapidly in importance, and we have the technology to do what
> people need done. What we don't have is a workable, coherent
> explanation of what to do for people who are not familiar with all 7
> points above, and we don't have a GUI for any platform to make setup
> easy.
>
> I would like it best if the principals in this, nedko and torben,
> could resolve this conflicting aspects of the situation on their own
> within the month of October. I personally doubt that this will happen
> for a number of reasons.

Since Romain is not working on that stuff anymore, I am supposed to be  
the guy to discuss with...

>
> After that, I plan to consult privately with Stephane and come to a
> decision about how to resolve it without necessarily involving either
> of the two principal authors. This would obviously be unfortunate. The
> goal will be to have a single version of netjack that works with jack1
> and jack2 installs, across Linux, OS X and Windows.
>
> Although I am wary of making this next comment, I think that future
> success relies on honesty here, and so I will offer my own rather
> strong opinion: I believe it was a mistake for Stephane to have
> allowed netjack2 into Jack2 even though the reasons for netjack2's
> development were reasonably clear and defensible. That doesn't mean
> that I am married to netjack1 or that I could not be persuaded to
> think differently about the situation. It just means that *in
> retrospect* it doesn't seem to have been the right decision. Change my
> mind if you feel I am wrong, or convince enough others that my opinion
> doesn't matter.

Well historical reasons mainly, netjack1 development was completely  
stalled at that time, the code was (still is??) hard to follow and to  
restructure to better fit a multi-platform design. The netjack1  
resurrected and both implementations evolved their own way.

>
> Lets move forward on this and make the amazing work Stephane has done
> bringing JACK to OS X and Windows even more useful to JACK users
>
> --p

Wo how to we start? Fist agree on a set of feature? (discovery  
protocol or not.., protocol definition, multi-platform, LAN/WAN...  
etc..)

Stéphane
_______________________________________________
Jack-Devel mailing list
Jack-Devel@...
http://lists.jackaudio.org/listinfo.cgi/jack-devel-jackaudio.org

[Nedko Arnaudov]

Paul Davis <paul@...> writes:

> Its time to do something about the dreadful situation with NetJACK.
> More and more people are asking about using it, and it is more or less
> impossible to provide them with any coherent answer about what to do.
> Here is my reading of the situation as it stands today:
>
> 1) we have 2 implementations called "netjack", both of which are
> incompatible with each other
> 2) netjack1, primarily written by torben, is only part of jack1
> 3) netjack2, primarily written by nedko, is only part of jack2
> 4) netjack1 has been aggressively developed over the last 6-10 months
> to give far superior network performance than netjack2, both over
> local and long haul networks
>  4a) the long haul improvements come mostly from using the CELT codec
>  4b) the LAN improvements have come in part from many discussions with
> Alex Carot, one of the main authors of SoundJack (presented at AES and
> elsewhere)
> 5) netjack2 has a discovery protocol, so that setup on a local network
> can be more automatic and "avahi"-like. netjack1 has no such protocol
> - master & slave must be explicitly connected to each other.
> 6) torben has set up his own git branch called jacknone that merges
> jack2 with netjack1
> 7) torben has been hard to reach for a while.
>
> I would welcome any corrections to my portrayal of the situation as I
> have described above. Assuming for the moment that it is correct ...
>
> ... this situation cannot be allowed to continue. Network audio is
> rising rapidly in importance, and we have the technology to do what
> people need done. What we don't have is a workable, coherent
> explanation of what to do for people who are not familiar with all 7
> points above, and we don't have a GUI for any platform to make setup
> easy.
>
> I would like it best if the principals in this, nedko and torben,
> could resolve this conflicting aspects of the situation on their own
> within the month of October. I personally doubt that this will happen
> for a number of reasons.
>
> After that, I plan to consult privately with Stephane and come to a
> decision about how to resolve it without necessarily involving either
> of the two principal authors. This would obviously be unfortunate. The
> goal will be to have a single version of netjack that works with jack1
> and jack2 installs, across Linux, OS X and Windows.
>
> Although I am wary of making this next comment, I think that future
> success relies on honesty here, and so I will offer my own rather
> strong opinion: I believe it was a mistake for Stephane to have
> allowed netjack2 into Jack2 even though the reasons for netjack2's
> development were reasonably clear and defensible. That doesn't mean
> that I am married to netjack1 or that I could not be persuaded to
> think differently about the situation. It just means that *in
> retrospect* it doesn't seem to have been the right decision. Change my
> mind if you feel I am wrong, or convince enough others that my opinion
> doesn't matter.
>
> Lets move forward on this and make the amazing work Stephane has done
> bringing JACK to OS X and Windows even more useful to JACK users

First, as Stephane said too, I'm not behind netjack2 development. I
havent even ran any netjack and I don't really know how to set it up. No
idea how you got that impression but it is completely wrong. AFAIK it is
Romain Moret that implemented netjack2.

WRT netjack2 & netjack2, I've expressed my opinion many times on IRC and
now I'll do it in the mailing list: Maintaining two versions of
netjack is even worse than maintaining two versions of jack itself.
All of them don't have too much spin and all of them have long track of
issues (bugs & missing features). Given the limited development power
that JACK stuff receives, fragmentizing the effort because of lack of
communication has bad effects on software quality. Different opinions
are natural of course, and whom wants to develop will probably do it
anyway, especially now that jack sources can be git-branched. IMO, it is
important for whom maintains "the" JACK, to prediodically merge work of
others. Of course what is merged is decision of the maintainer and will
probably be driven by community wishes.

Now some words on techincal aspects. I hope some day ladish will have
GUI for netjack and thus of course I agree that it is needed. If it is
developed externally, I'd like to avoid wheel reinvention.

--
Nedko Arnaudov <GnuPG KeyID: DE1716B0>


_______________________________________________
Jack-Devel mailing list
Jack-Devel@...
http://lists.jackaudio.org/listinfo.cgi/jack-devel-jackaudio.org

[Fons Adriaensen-2]

On Tue, Oct 06, 2009 at 09:36:50PM -0700, Fernando Lopez-Lezcano wrote:

> I don't currently have multicast enabled and I have to find how to set
> that up. AFAICT in netjack2 it is only possible to do auto discovery
> which is problematic right now (maybe that has changed since the last
> time I looked at this?). Or am I missing something? for whoever keeps
> track of this, I would really like the option of just typing an explicit
> address and that's it (I know, quaint and antiquated :-)

Same here. If it ever does anything not explicitly requested
it's no-go.

Ciao,

--
FA

Io lo dico sempre: l'Italia è troppo stretta e lunga.

_______________________________________________
Jack-Devel mailing list
Jack-Devel@...
http://lists.jackaudio.org/listinfo.cgi/jack-devel-jackaudio.org

[Jörn Nettingsmeier-5]

Paul Davis wrote:
<very good points about netjack>
> Lets move forward on this and make the amazing work Stephane has done
> bringing JACK to OS X and Windows even more useful to JACK users

you are probably aware of it, but to make sure:
there is also the jacktrip project (ccrma). while it lacks many features
of netjack1, it might be interesting to look at, because it aims at
being friendly to musicians (as opposed to "jack developers").
it's at jacktrip.sf.net.

maybe a concerted effort to create a user-friendly, coherent netjack
should include the jacktrip community as a whole? looks like the aims
are more or less identical.

as to torben being MIA, i'll be in berlin regularly in the next month -
maybe i get lucky and flush him out of his den :-D
(problem is, he is going to yell at me for not getting that netjack
gigabit test rig together as i promised, ages ago. but i'm prepared to
face the danger.)

best,

jörn


--
Jörn Nettingsmeier

Meister für Veranstaltungstechnik

Audio and event engineer
Ambisonic surround recordings

http://stackingdwarves.net
[phone]

_______________________________________________
Jack-Devel mailing list
Jack-Devel@...
http://lists.jackaudio.org/listinfo.cgi/jack-devel-jackaudio.org

[Kacper Wysocki-2]

2009/10/7 Jörn Nettingsmeier <nettings@...>:

> Paul Davis wrote:
> <very good points about netjack>
>> Lets move forward on this and make the amazing work Stephane has done
>> bringing JACK to OS X and Windows even more useful to JACK users
>
> you are probably aware of it, but to make sure:
> there is also the jacktrip project (ccrma). while it lacks many features
> of netjack1, it might be interesting to look at, because it aims at
> being friendly to musicians (as opposed to "jack developers").
> it's at jacktrip.sf.net.

Hi all,
I had no idea there were two netjack's. I got into jack to get a
firewire audio device working so-so. Two different, incompatible
firewire stacks, let alone two different jackd's make high-end audio
on linux a digital rats nest. And netjack sounds real cool :-)

--
http://kacper.doesntexist.org
http://windows.dontexist.com
Employ no technique to gain supreme enlightment.
- Mar pa Chos kyi blos gros
_______________________________________________
Jack-Devel mailing list
Jack-Devel@...
http://lists.jackaudio.org/listinfo.cgi/jack-devel-jackaudio.org

[torbenh]

On Tue, Oct 06, 2009 at 09:12:01PM -0400, Paul Davis wrote:
> Its time to do something about the dreadful situation with NetJACK.
> More and more people are asking about using it, and it is more or less
> impossible to provide them with any coherent answer about what to do.
> Here is my reading of the situation as it stands today:
>
> 1) we have 2 implementations called "netjack", both of which are
> incompatible with each other
> 2) netjack1, primarily written by torben, is only part of jack1
> 3) netjack2, primarily written by nedko, is only part of jack2
not written by nedko.

> 4) netjack1 has been aggressively developed over the last 6-10 months
> to give far superior network performance than netjack2, both over
> local and long haul networks
>  4a) the long haul improvements come mostly from using the CELT codec

celt is just a small piece in this puzzle.
whats more important is handling of packet loss and reordering.
the tricky stuff is maintaining sync even when packets are lost.

this code is using heuristics which are the result of lots of trial and
error. thats why its hard to follow, and pretty much not straght
forward. in its current state it can be simplified, but it requires some
successful tests, which would convince me that its working fine.

my motivation to making music is zero currently, so such testing
sessions with me would be pretty boring.
however i have used it successfuly to make music, with a friend, who was
using windows.

and windows is really shitty. it works much better with linux<->linux


>  4b) the LAN improvements have come in part from many discussions with
> Alex Carot, one of the main authors of SoundJack (presented at AES and
> elsewhere)
> 5) netjack2 has a discovery protocol, so that setup on a local network
> can be more automatic and "avahi"-like. netjack1 has no such protocol
> - master & slave must be explicitly connected to each other.

i am seeing some posts, which say that auto-discovery is not working for
their setups.

> 6) torben has set up his own git branch called jacknone that merges
> jack2 with netjack1

i am currently merging jack2 trunk into it. seems straight forward.

> 7) torben has been hard to reach for a while.

oh well... i cant promiss anything... but i am still here.

>
> I would welcome any corrections to my portrayal of the situation as I
> have described above. Assuming for the moment that it is correct ...
>
> ... this situation cannot be allowed to continue. Network audio is
> rising rapidly in importance, and we have the technology to do what
> people need done. What we don't have is a workable, coherent
> explanation of what to do for people who are not familiar with all 7
> points above, and we don't have a GUI for any platform to make setup
> easy.

i have a pyglade gui for jack_netsource. its not really cool, but its a
start. http://netjack.sf.net/gtknetsource.py 

the problem is that without dynamic backend switching, the slave needs
to restart his jackd. and there is still the weakness that not all
connection parameters are autodetected.

its all autodetected for the lan case though, but compression settings
for a WAN connect need to be given on both ends.

>
> I would like it best if the principals in this, nedko and torben,
> could resolve this conflicting aspects of the situation on their own
> within the month of October. I personally doubt that this will happen
> for a number of reasons.

if it was between nedko and me, there wouldnt be a problem :P
i must note here, that the port of netjack1 to jack2 had the goal of
sharing as much code between the jack1 and the jack2 version as
possible.

thats why its not a beautiful C++ driver. i tried to wrap the C driver
in a thin C++ layer which is necessary because jack2 wants c++ drivers.

it diverged more than i liked, but thats necessary. however the tricky
parts are still residing in a file which is mostly shared between jack1
and jack2.

this is also the reason why netone is not using the socket abstraction
netjack2 is using, i just added some #ifdef to make it work on windows.
but its still using POSIX sockets.

> After that, I plan to consult privately with Stephane and come to a
> decision about how to resolve it without necessarily involving either
> of the two principal authors. This would obviously be unfortunate. The
> goal will be to have a single version of netjack that works with jack1
> and jack2 installs, across Linux, OS X and Windows.

well.. the current netjack1 protocol does not support changing
connection parameters on the fly.
so this is definitely not the final solution.
i was under the false impression, that i had the energy to come up with
the netjack3 protocol, which would solve this mess.

but i realize this will not be happening.

also with jack2 a workaround is possible, just teardown the connection,
and restart jack_netsource with a different channel size.
the slave jack continues to run, and will accept the new connect.

>
> Although I am wary of making this next comment, I think that future
> success relies on honesty here, and so I will offer my own rather
> strong opinion: I believe it was a mistake for Stephane to have
> allowed netjack2 into Jack2 even though the reasons for netjack2's
> development were reasonably clear and defensible. That doesn't mean
> that I am married to netjack1 or that I could not be persuaded to
> think differently about the situation. It just means that *in
> retrospect* it doesn't seem to have been the right decision. Change my
> mind if you feel I am wrong, or convince enough others that my opinion
> doesn't matter.

the problem is that testing netjack in remote context is not so easy,
and its pretty annoying to look for testers on irc.

and then bore them with "not being creative" in the resulting jam.
the lan case is not important, it just works. i broke it sometimes,
but i think its running fine currently.

>
> Lets move forward on this and make the amazing work Stephane has done
> bringing JACK to OS X and Windows even more useful to JACK users

seconded.
>
> --p
> _______________________________________________
> Jack-Devel mailing list
> Jack-Devel@...
> http://lists.jackaudio.org/listinfo.cgi/jack-devel-jackaudio.org

--
torben Hohn
http://galan.sourceforge.net -- The graphical Audio language
_______________________________________________
Jack-Devel mailing list
Jack-Devel@...
http://lists.jackaudio.org/listinfo.cgi/jack-devel-jackaudio.org

[Jörn Nettingsmeier-5]

On 10/24/2009 10:00 PM, torbenh@... wrote:
> the problem is that testing netjack in remote context is not so easy,
> and its pretty annoying to look for testers on irc.
>
> and then bore them with "not being creative" in the resulting jam.
> the lan case is not important, it just works. i broke it sometimes,
> but i think its running fine currently.

i'll try and be available for netjack testing if needed. as to
creativity, i don't give an f. play a 1khz sine for all i care, as long
as it uncovers packet loss effects and shows concealment artifacts.

since my testing time is kind of erratic atm due to job requirements,
the most useful thing i can do is offer jack developers shell access to
one of my servers on the web, to bounce netjack tests off of. even root
access would be possible in a dedicated virtual machine. if anyone here
is interested, let me know, preferably this week, so that i can set it
up on the weekend. the testing scaffold would need to be tolerant to
local scheduling delays, though, since it's a xen environment. something
like -p 1024 or higher.

best,

jörn





--
Jörn Nettingsmeier

Audio engineer
Meister für Veranstaltungstechnik (Bühne/Studio)

Lortzingstr. 11, D-45128 Essen, Germany
[phone]

http://stackingdwarves.net

_______________________________________________
Jack-Devel mailing list
Jack-Devel@...
http://lists.jackaudio.org/listinfo.cgi/jack-devel-jackaudio.org