Csound Live Front End proposal and plan.

[iain duncan]

Attached is the proposal for the beginning of a csound front end
tailored for multi-user real time live work. Any interested in helping
or following this project, please email me off list. If you are an
experienced developer and would like to follow it and offer suggestions
but do not have time to work on the project, I would welcome your input
as well. I will set up some form of mailing list if there is enough
interest. Any help and suggestions are welcome. If you received this as
private email as well and do not want to get subsequent mail outs,
please let me know. I would also welcome any suggestions on the best way
to manage such a project as I have no experience with anything more than
coding with one buddy!


Thanks
Iain


Notes on the Csound Live Front End
Nov 5/2005, Iain Duncan iainduncan@...

The intention of the front end is to make a framework and templates for front ends designed specifically
around the needs of real time live performance with csound with a special focus toward multi-user
improvisation and real time collaboration. For my personal needs this will act as the interface to a csound
sequencing, synthesis, mixing, and production ensemble. I would like development to use two branches,
one a minimal working example of the framework and architecture that is easy to alter, and the other will
be a fuller featured set up for my own csound instruments designed to test the framework under real time load
and conditions. I am not a very experienced developer, so I welcome any feedback on architecture, coding
practice, project management, etc. I would especially like to know how best to manage collaboration, and
find out what folks would like best, ie mailing list, email group, sourceforge, etc.


Prinicipal components:
- multi-threaded host in C++ that will embed csound in one high priority thread
- graphic display window(s) in a display/gui thread used for displaying contents of csound tables/variables
- gui components for input to both front end settings and csound data
- csound interface object that translates and parses user i/o and handles communication with the csound instance
- midi i handled in a midi thread and parsed by the interface object before translation into csound api calls
- file i/o for frond end and csound settings
- later: network, serial port, and OSC input threads that will work much the same way as the midi input


Design requirements & goals:
- The csound thread should be highest priority and be serviced before all others, input should back up until
 time is available
- The csound interface object will handle all translation, parsing, and queuing of user i/o destined for
  csound, this acts as the hub
- The csound interface object will have an easy to use object oriented programming interface, and be easy for
  users to customize to match the way their csound files work, ie one should only have to make changes to the
  interface object to reflect table numbering, etc
- All other components should be unhinged from the interface and csound thread so that they can easily be turned
  on or off, edited, etc
- As much as possible, the gui interface will follow a Model-View-Controller approach, the display being updated
  from calls to the csound interface at a user defined rate. This will allow the display to accurately display
  csound settings that may be dynamically updated from outside the gui interface, and allow us to prioritize it
  accordingly and user a lower display rate then control rate
- Input through the gui ( mouse and keyboard events ) will be sent as fairly generic messages to the csound
  interface object, the gui should contain as little of the "brains" as possible
- Adding new tabs or windows to the gui should be easy to do using generic message passing to the controller.
  ie, the gui only needs to know that the 64 dials on screen result in something like
  csound_interface->dial_send( input_mode, dial, value );
- Midi automation will not be handled *through* the display. Messages will be handled by the interface object,
  and if they result in changes to data that is currently being displayed, this be reflected on screen when the
  display updates its data
- By using seperate input and display threads that all communicate through one central csound_interface hub,
  it should be possible to have several users all interacting with the hub/csound over network connections,
  multiple midi ports, multiple display windows, etc
- All routing and handling of multi-user data will be handled by the csound interface object so that again,
  any gui designs do not need to know how multi-user input is handled
- I/O to and from the csound interface object ( control/display->interface->csound and vice versa ) should all
  use well designed queue systems. Ultimately this should allow us to pass large amounts of data between the
  host and csound by deferring how much is done in each kpass. I will consider this properly achieved if we can
  load and save large audio files through the host, passing them into csound sample by sample at a controlled
  rate, allowing disk i/o without interferring with csound during real time playback


Tools in use:
So far I am using the GNU C++ compiler, fltk and portmidi. I would like to keep the basic framework to cross
platform tools, and I think it is a good idea to use the same tools being used in csound. ( fltk and portmidi ).
However, I would like the framework to be modular enough that it would be straightforward for one to add a
different input module ( say alsa midi ) or make display windows in another gui toolkit.


Some miscellaneous notes:
Csound_Interface class and controller object:
- executes in seperate thread from csound
- output to csound in form of table writes is done by putting messages on the control->csound queue
- gets incoming messages to deal with off the input->control queue: midi, keypresses, gui events, network events
- does all the input mode stuff
- also acts as the host to csound interface abstraction
- data members include: csound pointer, mutex, exit_now, cs_arc, cs_argv
- methods will include: start, stop, exit, load, send, receive, set, get, note, etc
- all other modules get a pointer to this class to do all their interfacing
- any calls to the csound class must start by locking the mutex, the user should not need to know about this
- for now, it will take all messages off it's queue and input them, later we will figure out how to buffer this
- it's messages are ONLY things that need to be api calls,
- perhaps it will also get display requests off it's queue? it would be good if display threads could
simply ask the controller for data if this does not significantly increase overhead.

Display thread and gui
- fltk display class, gets a pointer to the Csound_Interface class as an argument
- has a keyboard handler override to handle keyboard i/o bypassing widget handling
- in order to avoid needless redrawing, the event from the timer should not necessarily tell fltk to
  redraw, we should check all values and redraw only if there is a change
- the display uses the Csound_Interface class to update any pertinent settings
  updated by the timer function running at whatever we think is a reasonable update rate
- any display tab will thus have it's associated local data structure and update function, or the update method
  will take an argument to represent the current tab and have a switch in there somewhow.

Input threads/object:
- one per i/o type, midi input, keys, fltk display, etc
- each input object will have a pointer to the Csound_Interface class used for all i/o to/from csound
- passing/recieving data to/from the Csound_Interface should be as simple as making method calls to the
Csound_Interface class, any mutex locking and queueing will be handled on the Csound_Interface end:
- ie in a display window   csound_inter->send_m



Plan:
I have so far been testing various components of the framework in isolation. I have now gotten to the point
where I now that we are able to use a display that updates from csound flying 128 knobs around with heavy
csound load and no audio breakup ( though the latency could be better, it is certainly usable ). I intend for
the testing branch to use a very scaled down version of my work with fewer options and tracks.
The first version will not use network connection, and will use only one display window and one midi port.
However, the display window and midi input will be handled seperately, thus the display may follow the midi
input mode or not, as the user sees fit. The hub will accept up to 4 users, and the display can be set to be
one of those users. In order to test the various requirements, I believe I will use the following:

Csound ensemble:
- very simple one bar step sequencer, with 4 tracks, 4 paramaters per track
  - sequencing is handled within my csound files and should be opaque to the host
- 4 csound instruments/mixers playing from the step sequencer
- 2 different gui views, mixer and sequencer, 2 sub views of each
- 2 different gui input modes attached to the above views
- 2 different midi input modes ( ways in which midi input is translated to control the engine )
- ability to receive input data from 2 simultaneous users over midi, thus midi input will be parsed in a
  multi user fashion
- the gui/display will have an associated user number, thus if the display is set to the same user number
  as one of the midi users, it's settings will follow midi input, otherwise it will just display whatever
  it is set to
- example: while user 1 uses sequencer input mode 1 on midi channels 1-4 and user 2 uses sequencer input mode
  2 on midi channels 5-8, display will reflect user 1s actions if it is set to user 1.



_______________________________________________
csoundtekno mailing list
csoundtekno@...

Subscribe, unsubscribe, change mailing list options:
http://plot.bek.no/mailman/listinfo/csoundtekno