Hyun W. Ka

Three Dimentional Computer Vision-Based Alternative Control Method for Assistive Robotic Manipulator

JACO (Kinova Technology, Montreal, QC, Canada) is an assistive robotic manipulator that is gaining popularity for its ability to assist individuals with physical impairments in activities of daily living. To accommodate a wider range of user population especially those with severe physical limitations, alternative control methods need to be developed. In this paper, we presented a vision-based assistive robotic manipulation assistance algorithm (AROMA) for JACO, which uses a low-cost 3D depth sensing camera and an improved inverse kinematic algorithm to enable semi-autonomous or autonomous operation of the JACO. The benchtop tests on a series of grasping tasks showed that the AROMA was able to reliably determine target gripper poses. The success rates for the grasping tasks ranged from 85% to 100% for different objects.

Intelligent single switch wheelchair navigation

We have developed an intelligent single switch scanning interface and wheelchair navigation assistance system, called intelligent single switch wheelchair navigation (ISSWN), to improve driving safety, comfort and efficiency for individuals who rely on single switch scanning as a control method. ISSWN combines a standard powered wheelchair with a laser rangefinder, a single switch scanning interface and a computer. It provides the user with context sensitive and task specific scanning options that reduce driving effort based on an interpretation of sensor data together with user input. Trials performed by 9 able-bodied participants showed that the system significantly improved driving safety and efficiency in a navigation task by significantly reducing the number of switch presses to 43.5% of traditional single switch wheelchair navigation (p < 0.001). All participants made a significant improvement (39.1%; p < 0.001) in completion time after only two trials. Implications for Rehabilitation Intelligent navigation algorithm helps investigators develop smart wheelchair technology that may ultimately provide independent mobility to a segment of the population that currently finds it difficult or impossible to operate a conventional wheelchair. intelligent single switch wheelchair navigation improves the driving safety, comfort, and efficiency for individuals who rely on single switch scanning as a wheelchair control method by significantly reducing the number of switch presses and completion time.

Performance evaluation of 3D vision-based semi-autonomous control method for assistive robotic manipulator

Abstract We developed a 3D vision-based semi-autonomous control interface for assistive robotic manipulators. It was implemented based on one of the most popular commercially available assistive robotic manipulator combined with a low-cost depth-sensing camera mounted on the robot base. To perform a manipulation task with the 3D vision-based semi-autonomous control interface, a user starts operating with a manual control method available to him/her. When detecting objects within a set range, the control interface automatically stops the robot, and provides the user with possible manipulation options through audible text output, based on the detected object characteristics. Then, the system waits until the user states a voice command. Once the user command is given, the control interface drives the robot autonomously until the given command is completed. In the empirical evaluations conducted with human subjects from two different groups, it was shown that the semi-autonomous control can be used as an alternative control method to enable individuals with impaired motor control to more efficiently operate the robot arms by facilitating their fine motion control. The advantage of semi-autonomous control was not so obvious for the simple tasks. But, for the relatively complex real-life tasks, the 3D vision-based semi-autonomous control showed significantly faster performance. Implications for Rehabilitation A 3D vision-based semi-autonomous control interface will improve clinical practice by providing an alternative control method that is less demanding physically as well cognitively. A 3D vision-based semi-autonomous control provides the user with task specific intelligent semiautonomous manipulation assistances. A 3D vision-based semi-autonomous control gives the user the feeling that he or she is still in control at any moment. A 3D vision-based semi-autonomous control is compatible with different types of new and existing manual control methods for ARMs.

Performance Evaluation of a Mobile Touchscreen Interface for Assistive Robotic Manipulators: A Pilot Study

Background: Assistive robotic manipulators (ARMs) have been developed to provide enhanced assistance and independence in performance of daily activities among people with spinal cord injury when a caregiver is not on site. However, the current commercial ARM user interfaces (UIs) may be difficult to learn and control. A touchscreen mobile UI was developed to overcome these challenges. Objective: The object of this study was to evaluate the performance between 2 ARM UIs, touchscreen and the original joystick, using an ARM evaluation tool (ARMET). Methods: This is a pilot study of people with upper extremity impairments (N = 8). Participants were trained on 2 UIs, and then they chose one to use when performing 3 tasks on the ARMET: flipping a toggle switch, pushing down a door handle, and turning a knob. Task completion time, mean velocity, and open interviews were the main outcome measurements. Results: Among 8 novice participants, 7 chose the touchscreen UI and 1 chose the joystick UI. All participants could complete the ARMET tasks independently. Use of the touchscreen UI resulted in enhanced ARMET performance (higher mean moving speed and faster task completion). Conclusions: Mobile ARM UIs demonstrated easier learning experience, less physical effort, and better ARMET performance. The improved performance, the accessibility, and lower physical effort suggested that the touchscreen UI might be an efficient tool for the ARM users.

Circling motion and screen edges as an alternative input method for on-screen target manipulation

Abstract Purpose: To investigate a new alternative interaction method, called circling interface, for manipulating on-screen objects. To specify a target, the user makes a circling motion around the target. To specify a desired pointing command with the circling interface, each edge of the screen is used. The user selects a command before circling the target. Method: To evaluate the circling interface, we conducted an experiment with 16 participants, comparing the performance on pointing tasks with different combinations of selection method (circling interface, physical mouse and dwelling interface) and input device (normal computer mouse, head pointer and joystick mouse emulator). Results: A circling interface is compatible with many types of pointing devices, not requiring physical activation of mouse buttons, and is more efficient than dwell-clicking. Across all common pointing operations, the circling interface had a tendency to produce faster performance with a head-mounted mouse emulator than with a joystick mouse. The performance accuracy of the circling interface outperformed the dwelling interface. Conclusions: It was demonstrated that the circling interface has the potential as another alternative pointing method for selecting and manipulating objects in a graphical user interface. Implications for Rehabilitation A circling interface will improve clinical practice by providing an alternative pointing method that does not require physically activating mouse buttons and is more efficient than dwell-clicking. The Circling interface can also work with AAC devices.

Empirical evaluation of circling interface with head-mounted mouse emulator users

Abstract Purpose: To evaluate the performance of the circling interface, which is an alternative interaction method for selecting and manipulating on-screen objects based on circling the target, rather than pointing and clicking. Method: We conducted empirical evaluations with actual head-mounted mouse emulator users from two different groups: individuals with spinal cord injury (SCI) and individuals with cerebral palsy (CP), comparing each group’s performance and satisfaction level on pointing tasks with the circling interface to performance on the same tasks when using dwell-clicking software. Results: Across all operations, for both subjects with SCI and with CP, the circling interface showed faster performance than the dwell-clicking interface. For the single-click operation, the circling interface showed slower performance than dwell selection, but for both double-click and drag-and-drop operations, the circling interface produced faster performance. Subjects with CP required much longer time to complete the tasks compared to subjects with SCI. If errors caused by circling on an area with no target and unintentional circling caused by jerky movements and an abnormally tiny circle are automatically corrected by the circling interface, their performance accuracy with the circling interface outperformed existing solutions without a steep learning curve. Conclusions: Circling interface can be used in conjunction with existing techniques and this kind of combined approach achieve more effective mouse use for some individuals with pointing problems. It is also expected to be useful for both computer access and augmentative communication software. Implications for Rehabilitation A circling interface will improve clinical practice by providing an alternative pointing method that does not require physically activating mouse buttons and is more efficient than dwell-clicking. Being used in conjunction with existing techniques, some individuals who are head mouse users can achieve more effective mouse use. The Circling interface can also work with AAC devices.