Vbf:Ang: [GOK] plan for GNOME 3.0 - please comment

[Mats Lundälv]

Hi all,

I just got aware that the message below was by mistake only sent to Ben and not to the list, as intended. Here it comes a bit delayed ...

Mats

-----Vidarebefordrat av Mats Lundälv/vgregion 2009-10-12 02:31 -----

Till: Ben Konrath <ben.konrath@...>
Från: Mats Lundälv/vgregion
Datum: 2009-10-06 03:31
Ärende: Ang: [GOK] plan for GNOME 3.0 - please comment

Hi Ben and all,

It's great to see the activity and plans building up in this area!

Sorry for not being faster to respond (since my post and your reply in the oatsoft oats-sig list)!

It's obvious that there is a deep consensus about going for the Python platform for this project in the Gnome environment. (I suggested Java as an possible alternative, possibly opening for better cross-platform aspects for this kind of an app.) But that's fine then.

Without having deep insight I'd flag for keeping the timing accuracy issue in mind. For a high class switch input application, very quick response and accurate and reliable timing is very essential. It should in fact ideally be the absolute priority thread of the whole system. Is there a reason for concerns here?

(To get a feeling of state-of-the-art performance in terms of control and timing we should look at AssistiveWare's OSK software for the Mac OS X – SwtchXS – take a look at some of the video clips at http://www.assistiveware.com/switchaccess.php !)

I basically think your analysis and draft plan looks very reasonable – with the following remarks:

It's fine to start with the focus on switch input users – as long as the basic design allows for adding high class direct access input later, not forgetting that there is a great need for advanced configurable functionality in that area too – for head-mouse and eye-gaze users with greatly varying pointing, timing and visual perception preconditions etc.
Very good to have Nomon and Dasher integration in the pipeline!

It's fine to start with a limited set of layout options – as long as the basic design allows for adding options for advanced, wide-ranging and user-friendly layout configurability later (without starting from scratch).

It's fine to start with letter-based input and word prediction – as long as:

a) we design for full multilingual support – that is NOT the English a-z alphabet only – from the start!

b) the basic design allows added good support for both graphic symbols (image formats) and text later on – but ideally from the start (also think Asian multi-layered characters/words – perhaps Ruby Annotation support etc )

It's fine to start with the limited set of switch input/navigation methods you suggest – as long as the design is prepared for smooth later addition of more methods, where 2 switch user-scan (as described in the ACE doc) is one obvious, but also later “directed scan” (joystick/4-5switch), quadrant keyboard and a set of direct selection options may line up.

The switch options should include support for the standard USB Game interface from the start.

I've got a lot on my mind about the principal design issues, but will have to come back to that in due time. Just some initial thought about two major points:

Navigation/selection:
- Pay much attention to designing a highly efficient and reliable “Focus Navigation” protocol.
- Apart from real time accuracy etc., think freely there; e.g. leave the door open for the possibility of (later) allowing more than one focus navigation and selection thread if possible.

Layout:
- One (and in principle more) OSK canvas(es) – ideally with (later) possibility of adjustable transparency and to add background image

- principally design for allowing configurable layout with, and navigation between and within, both single and compound selection components (buttons and button grids).

- the first implementation being an OSK canvas with just one compound selection grid component (or possibly two, where the word prediction is presented in the second one)

- the selection button grid I envisage similar to a table component (in an office application) – possibly built on some existing one – where a regular grid with a certain number of rows and columns is set up – then the width of columns and hight of rows may be adjusted, cells may be joined, grouping options may be added etc.  – with a limited set of built-in focus navigation schemes (that may later be expanded). (I remember a button grid component like that in the good old NextStep development environment – is there something like that floating around in the Python universe ;-)

Cheers,

Mats

-----gok-list-bounces@... skrev: -----

Till: Gerd Kohlberger <lowfi@...>, gok-list@...
Från: Ben Konrath <ben.konrath@...>
Sänt av: gok-list-bounces@...
Datum: 2009-10-02 19:47
Ärende: [GOK] plan for GNOME 3.0 - please comment

Hi Gerd / list,

After some investigation and thought, I would like to propose a plan to
move forward with development on GOK. My research included extensively
using GOK, reading 'Switch access to technology: A comprehensive guide"
published by the ACE Centre [1] and meeting with a number of people
familiar with GOK and on-screen keyboards in general.

From this research, my feeling is that the ideas behind the advanced
features in GOK are good but the UI in GOK fails expose these features
in usable way. Because of these usability issues, I think a
straight-forward port to python is not the best use of our time.
Instead, I would like to start from scratch pulling in the parts of GOK
that we need to make a better user experience.

This new app will focus on creating a usable solution for people whose
primary way of accessing a computer is a switch device. I think it's too
ambitious to promise feature parity with GOK for GNOME 3.0 so I would
like to start by creating an on-screen keyboard that has switch device
access as it's primary use case. This will allow us to work on the user
experience / switch interaction before we move to the UI navigation.
Here are the features I would like to deliver for GNOME 3.0:

* on-screen keyboard (qwerty, alpha numeric sorted alphabetically,
  alpha numeric sorted by frequency)
* word prediction as a library ported from GOK (the goal for this would
  is to eventually to get other people interested in using this too).
* new on-screen display consisting of a docked window taking 1/4 -
  1/3 of the screen height
* fully usable with 1 switch Autoscan (listed as essential priority on
  page 31 of [1])
* fully usable with 1 switch Userscan (listed as high priority on page
  31 of [1])
* UI for setting up switch (mouse button, keyboard button, official
  switch interface etc).

Post GNOME 3.0 I would like to explore:

* adding UI interaction - obviously this needs to be fleshed out more
* add option to pop up only when a text entry
* add option to use nomon (or nomon like interface for text entry)
* add option to use dasher (or dasher like interface for text entry)
* conduct user testing once UI interaction is in place
* add 2 switch access

I would also like to have a new name for this app. This allows us to
make a clean start and get away from the idea that the keyboard part of
the app is the primary thing that we do. One idea for this app name is
caribou - from what I've read, caribou make lots of clicking sounds when
they walk. Any other suggestions?

The only big question on my mind is how should we gracefully exit from
current GOK? Here are some suggestions:
   -> ensure that GOK compiles and runs with GNOME 3.0
   -> don't work on any major or time consuming bugs
   -> #ifdef the cspi code - not sure if this is useful though

What do people think of this plan? Gerd, Are you on board for helping
with this? Is there anything that you'd be interested in doing?

Comments are appreciated.

Cheers, Ben

1.
http://www.ace-centre.org.uk/assets/Product%20Downloads/SwitchScanningMaster_8_472.pdf
_______________________________________________
gok-list mailing list
gok-list@...
http://mail.gnome.org/mailman/listinfo/gok-list

_______________________________________________
gnome-accessibility-list mailing list
gnome-accessibility-list@...
http://mail.gnome.org/mailman/listinfo/gnome-accessibility-list

[Steve Lee-3]

You make some excellent points Mats and I agree about speed being an
important priority. It's amazing how fast some users can scan.

2009/10/12 Mats Lundälv <mats.lundalv@...>:
> - the selection button grid I envisage similar to a table component (in an
> office application) – possibly built on some existing one – where a regular
> grid with a certain number of rows and columns is set up – then the width of
> columns and hight of rows may be adjusted, cells may be joined, grouping
> options may be added etc.  – with a limited set of built-in focus navigation
> schemes (that may later be expanded). (I remember a button grid component
> like that in the good old NextStep development environment – is there
> something like that floating around in the Python universe ;-)

Well Maavis [1] ticks quite a few of your boxes but it's XUL +
javascript, with a Python server for some functionality. It is
deployed as a Firefox extension but could be ported to XUL Runner.

The vision is that Maavis will be a platform for creating new and
innovative styles of interaction by those with web-style technical
skills and yet provides very simple configuration for non technical
facilitators. An intermediary level of skill could easily be added for
using a GUI to configure cell layouts etc. So far we have a simple
touch UI and have just added switch support for research in schools.

I took the design approach that selection sets are created in XUL
(which is similar to HTML) + CSS + javascript. The idea being to use a
familiar declarative web style that gives great flexibility and has
low barrier to entry as far as technical skills are concern. However
this is not suitable for non technical facilitators who use a simple
file copying paradigm to setup resources (plus some text files that
will eventually become a GUI). The platform is rich enough to allow
the development of GUIs for editing selection sets, indeed work has
started in the form of a 'settings' UI . As experience has grown it's
clear that much can be abstracted into standard features available
from the higher levels. This is done by adding new XUL elements and
attributes as well as providing a small API for using from javascript.

The switch access we have added for Maavis@school was designed to be
expandable from the simple linear scanning provided and to allow
spatial modes as well. This is yet to be proven though. Technically it
is a simple event driven state machine with the specifics of
navigation and selection/focus being provided by the layout code.
We've started with 1/2 switch auto/user scan with USB devices as
suggested by Simon Judge.

While Maavis has been developed on Windows XUL is naturally cross
platform and the platform specific areas have been localised. It uses
Outfox which is a local Python server, currently for voip integration
and switch input (via pygame). There is obviously scope for moving
functionality across the client/server divide but I would expect UI
and network services would stay on XUL client side as they are it's
strengths.

I'd be very happy to discuss how Maavis could be used as we are keen
to have engaged users and contributors .

1: https://www.assembla.com/wiki/show/maavis

Steve
_______________________________________________
gnome-accessibility-list mailing list
gnome-accessibility-list@...
http://mail.gnome.org/mailman/listinfo/gnome-accessibility-list