Arthur I. Karshmer

Computers Helping People with Special Needs

The Special Thematic Session of ICCHP 2014 entitled Accessible Media promises to yield exciting research and development from around the world in the areas of access to television and audio-video content, eBooks, and social media. The scholarly works in this session report on and discuss a wide range of activities under the umbrella of this important theme, which are all working towards the United Nations specified goal of providing more equal access to the cultural environment for person with disabilities.

Navigation of HTML tables, frames, and XML fragments

In this paper, we provide a progress report on the development of technology to support the non-visual navigation of complex HTML and XML structures.

Towards a Universal Maths Conversion Library

The study of Mathematics and Sciences have always been a difficult problem for blind students especially because of the complexity of Braille mathematical notations. Various projects developed converters allowing people to translate a formula from mainstream notations (like LATEX or MathML) to Braille notations and vice versa. Today a new generation of tools aims at facilitating the understanding of the formulas by blind users, and the communication between sighted and Braille users. The project of Universal Maths Conversion Library is born from the decision of 6 organisations both American and European to join their efforts in that field.

Progress in Automated Computer Recognition of Sign Language

This paper reviews the extensive state of the art in automated recognition of continuous signs, from different languages, based on the data sets used, features computed, technique used, and recognition rates achieved. We find that, in the past, most work has been done in finger-spelled words and isolated sign recognition, however recently, there has been significant progress in the recognition of signs embedded in short continuous sentences. We also find that researchers are starting to address the important problem of extracting and integrating non-manual information that is present in face and head movement. We present results from our own experiments integrating non-manual features.

A domain specific language framework for non-visual browsing of complex HTML structures

We present a general framework for navigating complex structures - specifically, tables, frames, and forms?found in web-pages. Our framework is based on an (automatically or manually created) program written in a domain specific language that captures the semantic structure of the table/frame/form as well as specifies the strategy to be used for navigating it. We describe our general framework and the domain specific language we have designed.

UMA: a system for universal mathematics accessibility

We describe the UMA system, a system developed under a multi-institution collaboration for making mathematics universally accessible. The UMA system includes translators that freely inter-convert mathematical documents transcribed in formats used by unsighted individual (Nemeth, Marburg) to those used by sighted individuals (LaTeX, Math-ML, OpenMath) and vice versa. The UMA system also includes notation-independent tools for aural navigation of mathematics. In this paper, we give an overview of the UMA system and the techniques used for realizing it.

Cognitive analysis of equation reading: Application to the development of the math Genie

Can information about the perceptual and cognitive processes involved in equation reading be applied in the creation of assistive technology for blind equation readers? The present research used four cognitive/perceptual studies to examine several hypotheses about equation reading: people (1) read equations from left to right, one element at a time, (2) back scan when reading equations, (3) substitute the outcome of a parenthetical expression for the initial elements, and (4) scan the entire equation before element by element reading to create a schematic structure. The process tracing study provided evidence for all of the hypotheses, with three experiments supporting the first three hypotheses, but not the fourth. These results have been implemented in assistive software for visually-impaired users, the Math Genie – an auditory browser.

Translating MathML into nemeth braille code

An assistive software application has been created that translates math statements encoded as MathML into Nemeth Braille Code (NBC). This translation is conducted in two phases, the translation of the MathML elements into NBC, then the implementation of rules specific to Nemeth Braille that are irrelevant to MathML. All MathML elements holding semantically relevant information are translated by this program, including the nesting of elements to any level. Some of the syntactical rules inherent to NBC such as the use of the numeric indicator; additional space characters; and some contractions have also been implemented; other rules remain to be incorporated. The NBC can be exported in three ways (1) directly to a refreshable Braille device via a serial connection in real time; (2) saved as a text file then downloaded into a Braille device; and (3) save as a text file then embossed by a third party application. This application allows a person with no Braille experience to enter a math equation into any equation editor that can save that statement in the MathML format, and then convert that statement into Nemeth Braille Code for perusal by the visually impaired

Access to Mathematics by Blind Students

Access to, and doing mathematics, is one of the biggest obstacles for blind students in school and at the university. Our special thematic session (STS) will present new approaches to offering blind students better access to mat, to provide new tools for doing math and offer support tools to math teachers to help them offer better instruction to blind students. In the remainder of this short paper, the basic problems and historical solutions to the problem are discussed as a means of laying the groundwork for our STS.

An experimental sound-based hierarchical menu navigation system for visually handicapped use of graphical user interfaces

The use of modern computers by the visually handicapped has become more difficult over the past few years. In earlier systems the user interface was a simple character based environment. In those systems, simple devices like screen readers, braille output and speech synthesizers were effective. Current systems now run Graphical User Interfaces (GUIs) which have rendered these simple aids almost useless. In the current work we are developing a tonally based mechanism that allows the visually handicapped user to navigate through the same complex hierarchical menu structures used in the GUI. The software can be easily, and cheaply, incorporated in modern user interfaces, making them available for use by the visually handicapped. In the remainder of this paper we present a description of the sound-based interfaces as well as the techniques we have developed to test them.