After the previous false starts in using XUL and LSR this phase concentrated on creating working access to Firefox menus and the web using a simple OSK and switch and with minimal platform dependencies on Linux. This was achieved and it is now possible to use Jambu to browse the web with a toe (thanks to Eitan Isaacson for this observation in his review). A UI is provided that allows navigation around selected areas of Firefox (toolbar, shortcut bar, web document) using 'next' and 'action' buttons. This adds a minimal level of indirection between user actions (on the Jambu cells) and control of the target application (Firefox). However this is still fairly frustrating to use so the next phase will concentrate on in-application selection which will further reduce the indirection.
The result is a 'framework' for creating simple SVG described UIs that consist of a number of 'cells' and which can be operated using a switch or the system pointer (mouse). Very little is 'hard coded' into Jambu with the appearance and behaviour being declared in SVG extended with tags to control visibility and respond to events by calling 'primitive' functions. A minimal 'virtual machine' model has been created that consists of cells which have properties that may be manipulated and that effect the display by hiding or showing sections of SVG. This is is sufficient to provide a control set for Firefox and a basic On Screen Keyboard, while being flexible enough for many other purposes. Control of target applications is via accessibility API (ATSPI) or simulated key events.
An early decision was made to create a framework rather than specifically solving the Firefox control as one of the prime objectives of Jambu is to encourage community input and innovation through providing an easy way of authoring UIs using declarative web like documents. This obviously delays development results and requires effort in supporting the framework and this effort turned out to be more expensive that initial experiments indicated. However the resulting framework is usable and flexible with authoring consisting of hand editing the Jambu file and possibly creating new primitive functions in python.
A number of refinements are required to better meet the goals, in particular there is currently a new expression 'language' to be learnt and which should be replaced with Python or Javascript. Graphics editing with inkscape is not yet seamless as is required to allow designers to work on the visuals in an environment that suits them. Performance is reasonable but needs optimisation and is untested with complex SVG. Switch input is semi hard coded and should be abstracted and made part of the file format. More flexible device support is needed as it is currently for USB joysticks only. The warning/error handling needs rationalisation as errors appear on stdout and can be cryptic.
Several SVG problems were encountered which may be librsvg bugs so these need to be investigated and bugs raised or patches submitted.
Technical appraisal
Jambu has a small set of dependencies; Python 2.5, pygtk, librsvg, pyatspi and pygame. It currently works on Linux but the desire is to port it to Windows. While early experiments proved pygtk is usable on Windows, the critical need is for a pyatspi (or rather a pyia2) for Windows. It is worth noting that pyatspi makes accessibility access from ATs extreamly easy (big thanks to Peter Parente, Eitan Isaacson and Willie Walker).
The basic approach used is to leverage SVG extensibility mechanisms to add visibility control and behaviour (like XBL) to a master SVG document. Jambu loads this and displays and operates the UI in a GTK custom widget. A fundamental constraint is that librsvg is not a DOM so it is not possible to update parts of the document and get automatic display refresh. Rather librsvg will only draw entire documents (actually the default python bindings are missing some functions to draw a sub tree). Thus we need to break up the document into individual SVG documents for each cell using state to decide on the exact contents. The chosen solution was to embed expressions that determine if sections of each cell are visible by being evaluated at display time. This has performance impact and also clutters up the SVG with behaviour so is will be changed to the more traditional setting of visibility properties from code (e.g. the display CSS style).
Another useful feature SVG is that nested SVG tags can use a viewBox for co-ordinate mapping. This allows us to use corse grid co-ords for the location of the cells and high resolution for the cells contents. it also means a cells co-ords are it's grid x,y,width,height without needing to add attributes for this. The nested SVGs get removed when rendering. However this solution is not complete as inkscape loses the grid.
The expression format was intended to be a temporary and experimental place holder until a decision was made to go either fully scripted (document driven like XBL or possibly code driven as in libglade) or fully declarative (ala iHTML). This was a mistake as it was much too expensive to develop even using Python's excellent dynamic facilities and it also creates a new language to be learnt by authors. It is now clear that declarative code syntax is really unworkable (debugging for one). There are also concerns that Python sandbox is unsafe and Jambu files should be web deployable (so Javascript may be a solution).
Python has proven such as delight to use for this and has no doubt saved a lot of time and code, even though I've been learning it's deeper features. It was relatively easy to use the dynamic features in conjunction with classes and functions being first class objects with flexible construction. I hope the resultant code is not obscure (though it goes deep). It is a little tricky in places and I would redo the use of bound/unbound instance methods in a more pythonic way if this was not going to be depreciated. Decorators provided a neat way to encapsulate argument checking in the cell primative functions, removing a lot of boiler plate code.
I used a Test Driven approach for developing the core parts and highly recommend it. It divides the problem, gives a focus on usage, instils a rhythm and provides reassurance when refactoring. Plus it tells you when to stop coding, when the temptation can be to add 'just one more' feature. When used in conjunction with a debugger and stepping through code TDD provides a high confidence in quality (Ok so some TDD folks say you don't need a debugger). In any case you need unit tests for code coverage and regression. I miss being able to use these techniques for the UI so will probably make Jambu accessible itself and use a test framework like DogTail to drive it.
A personal note
I have really enjoyed this phase though it has been a steep learning curve (gtk custom widgets, SVG, deep Python) and a little pressured to get it all done in time for the summit. Creating something like this from scratch is in some ways harder than learning someone else's platform (even complex ones like XUL) and can be a little lonely and insecure when you're not sure if you're going down the right track. I won't really know until we get people churning out Jambu files for all sorts of use cases (and then some will want a WYSIWYG editor).
I really appreciate the support of Aaron Leventhal and Frank Hecker. Mozilla's funding of accessibility is making Jambu possible and I look forward to the time when users can add their thanks.
Resources
The Jambu file format is described in more detail and issues are discussed in jambuFileFormat.txt.
Details of obtaining and using Jambu can be found in the README and on the Jambu home page.