Yanyu Lu

Moving object recognition under simulated prosthetic vision using background-subtraction-based image processing strategies

Abstract A visual prosthesis that applies electrical stimulation to different parts of the visual pathway has been proposed as a viable approach to restore functional vision. However, the created percept is currently limited due to the low-resolution images elicited from a limited number of stimulating electrodes. Thus, methods to optimize the visual percepts providing useful visual information are being considered. We used two image-processing strategies based on a novel background subtraction technique to optimize the content of dynamic scenes of daily life. Psychophysical results showed that background reduction, or background reduction with foreground enhancement, increased response accuracy compared with methods that directly merged pixels to lower resolution. By adding more gray scale information, a background reduction/foreground enhancement strategy resulted in the best performance and highest recognition accuracy. Further development of image-processing modules for a visual prosthesis based on these results will assist implant recipients to avoid dangerous situations and attain independent mobility in daily life.

Chinese Character Recognition Using Simulated Phosphene Maps

PURPOSE A visual prosthetic device may produce phosphene maps in which individual phosphene characteristics can be altered. This study was an investigation of the ability of normally sighted subjects to recognize Chinese characters (CCs) after altering simulated phosphene maps. METHODS Thirty volunteers with normal or corrected visual acuity of 20/20 were recruited. CC recognition accuracy and response time were investigated while one parameter was changed (distortion, pixel dropout percentage, pixel size variability, or pixel gray level) or different combinations of three parameters were used. Five hundred CCs consisting of 1 to 16 strokes were used for the character sets. RESULTS CC recognition accuracy and response times respectively decreased and increased when distortion, dropout, and pixel size variability increased. Gray levels did not significantly affect the results, except when eight levels were used. To maintain an 80% accuracy rate, there should be a distortion index (k) of no more than 0.2 (irregularity), a pixel dropout of 20%, and a pixel size range of 1 to 16 mm (7-112 min arc). Only a combination of a k=0.1 distortion index, a dropout of 10%, and a pixel size range of 1.33 to 12 mm (9.3-84 min arc) achieved a goal of ≥80% accuracy. CONCLUSIONS Distortion, dropout percentage, and pixel size variability have a significant impact on pixelated CC recognition. Although at present the visual ability of prosthesis users is limited, it should be possible to extend this to CC recognition and reading in the future. The results will help visual prosthesis researchers determine the effects of altering phosphene maps and improve outcomes for patients.

Reading Pixelized Paragraphs of Chinese Characters Using Simulated Prosthetic Vision

PURPOSE Visual prostheses offer a possibility of restoring useful reading ability to the blind. The psychophysics of simulating reading with a prosthesis using pixelized text has attracted attention recently. This study was an examination of the reading accuracy and efficiency of pixelized Chinese paragraphs after different parameters were altered. METHODS Forty native Chinese speakers with normal or corrected visual acuity (20/20) participated in four experiments. Reading accuracy and efficiency were measured after changing the character resolution, character size, pixel dropout percentage, number of gray levels, and luminance. RESULTS A 5° × 5° character appeared to be the optimal size necessary for accurate pixelized reading. Reading accuracy close to 100% could be achieved with 10 × 10 pixels/character and ∼60% with a 6 × 6 pixel resolution. Pixel dropout adversely affected accuracy, and paragraphs with a 50% dropout were unreadable. Luminance had little effect; however, the number of gray levels significantly affected reading performance. Paragraph reading was at least 5% more accurate at each resolution than was the accuracy of Chinese character recognition. CONCLUSIONS Character size and resolution, pixel dropout, and the number of gray levels clearly affected the reading performance of pixelized Chinese paragraphs. Compared with pixelized character recognition, pixelized Chinese paragraph reading achieved higher accuracy; thus, optimal Chinese reading performance may require prostheses with more electrodes (1000) than are required to read paragraphs in the Latin alphabet (500).

Recognition of Objects in Simulated Irregular Phosphene Maps for an Epiretinal Prosthesis

Visual prostheses offer a possibility of restoring vision to the blind. It is necessary to determine minimum requirements for daily visual tasks. To investigate the recognition of common objects in daily life based on the simulated irregular phosphene maps, the effect of four parameters (resolution, distortion, dropout percentage, and gray scale) on object recognition was investigated. The results showed that object recognition accuracy significantly increased with an increase of resolution. Distortion and dropout percentage had significant impact on the object recognition; with the increase of distortion level and dropout percentage, the recognition decreased considerably. The accuracy decreased significantly only at gray level 2, whereas the other three gray levels showed no obvious difference. The two image processing methods (merging pixels to lower the resolution and edge extraction before lowering resolution) showed significant difference on the object recognition when there was a high degree of distortion level or dot dropout.

Optimizing Chinese character displays improves recognition and reading performance of simulated irregular phosphene maps.

PURPOSE A visual prosthesis may elicit an irregular phosphene map relative to a regular electrode array. This study used simulated irregular phosphene maps as a way of optimizing the display methods of Chinese characters (CCs) to improve recognition and reading performance. METHODS TWENTY SUBJECTS WITH NORMAL OR CORRECTED SIGHT PARTICIPATED IN TWO EXPERIMENTS (9 FEMALES, 11 MALES, 2030 YEARS OF AGE). EXPERIMENT 1: two character display methods were proposed: selecting phosphenes covered by character strokes on a simulated phosphene array (projection method) and finding the phosphene closest to the expected location in some range of an irregular phosphene array as a substitute (nearest neighbor search [NNS] method). The recognition accuracy of CCs was investigated using six levels for the coverage ratio of stroke and phosphene area and for search range, respectively, for two methods, for several irregularity levels. Experiment 2: reading accuracy (RA) and reading efficiency (RE) were measured using the regular array correspondence and NNS methods. RESULTS EXPERIMENT 1: projection and NNS methods were significantly affected by coverage ratio or search range. NNS significantly improved CC recognition accuracy to the highest at 81.3 ± 2.7% and 59.1 ± 5.2%, respectively, for different irregularity levels, compared with the projection method. Experiment 2: RA and RE significantly decreased as the distortion level increased; NNS significantly improved RA (from approximately 40% to >80%) and RE (from approximately 13 char/min to >40 char/min) when reading more irregular paragraphs. CONCLUSIONS The performance of CC recognition and paragraph reading when using an irregular phosphene array can be improved through optimizing the display method.

Estimation of Simulated Phosphene Size Based on Tactile Perception

Clinical trials have successfully shown that a visual prosthesis can elicit visual perception (phosphenes) in the visual field. Psychophysical studies based on simulated prosthetic vision offer an effective means to evaluate and refine prosthetic vision. We designed three experiments to examine the effect of phosphene luminance, flicker rate, and eccentricity on the ability to estimate simulated phosphene sizes using tactile perception. Thirty subjects participated in the three experiments. There was a linear increase in reported size as visual stimulus size increased. Judgment was significantly affected by stimulus luminance and eccentricity (P < 0.05) but not by flicker rates. Brighter stimuli were perceived as being larger, and the more eccentric the position, the larger the estimated size. These simulation studies, although idealized, suggested that tactile perception is a potential way to estimate phosphene sizes.

Estimating the position of simulated phosphenes using a tactile guide.

A visual prosthesis provides usable visual information to the patient in the form of phosphenes, that is, punctate photic sensations seen after electrical stimulation. Stimulation via different electrodes results in phosphenes in different positions within the visual field. Simulation studies can provide data on the possible limitations of prosthetic stimulation. We used a head mounted screen to monocularly present constant or flickering light spots of different sizes, or luminance to normally sighted subjects. Subjects were asked to judge the location of the spots using a polar coordinate tactile guide; positioning average error and dispersion were analyzed. With the increase of eccentricity, the positioning average error and dispersion were also increased. The performances under large, stable and high luminance conditions were better than that under small, flickering and low luminance conditions, respectively. Repeated training sessions were shown to significantly improve the positioning performance.

Correction for Chinese character patterns formed by simulated irregular phosphene map

To reduce the unfavorable influence of phosphene array irregularity on the form of Chinese character pattern so as to improve recognition accuracy in visual prostheses, two correction methods were put forward. One method was to generate phosphene closest to the target point in regular arrays using weighted nearest neighbor search. The other was to generate phosphene whose center located in the region covered by dilated characters. Based on a simulation system, Chinese character recognition tests were given to fifteen normally sighted subjects under five degrees of array irregularity (0.2, 0.4, 0.6, 0.8, 1.0) without correction. The recognition accuracy decreased with the increase of irregularity. When the recognition accuracy dropped below 80%, two correction methods were applied and their effects were evaluated. With the increase of array irregularity, both effects on the accuracy of recognition grew considerably. Comparison between the two methods revealed that the former method afforded higher recognition accuracy and the latter only applied to phosphene map with serious irregularity.