Panadda Marayong

Development of haptic assistance for route assessment tool of NASA nextgen cockpit situation display

The NextGen Cockpit Situation Display (CSD), developed by NASA Amess Flight Deck Display Laboratory, provides advanced flight control functionalities and traffic/weather displays to pilots [1]. Traditionally, the user operates with the CSD using a computer mouse and receives only visual feedback about the controlling actions. In this work, we integrate force feedback in the Route Assessment Tools of the CSD, where the user can manage the flight plan to resolve conflicts in real-time. A spring force, with a variable stiffness coefficient, was used to model the force feedback with its strength varying proportionally to the overall path length. Force display was provided as an indicator of the effort required to deviate from the optimal path to assist the user in decision making. The force feedback models were evaluated on a software testbed created on Microsoft Foundation Class with the Novint Falcon haptic-feedback input device.

Integration framework for NASA NextGen Volumetric Cockpit Situation Display with haptic feedback

In this paper, we present a framework for the integration of force feedback information in a NASA NextGen Volumetric Cockpit Situation Display (CSD). With the current CSD, the user retrieves operational information solely through visual displays and interacts with the CSD tools through using a mouse. The advanced capabilities of the CSD may require complex manipulation of information which may be difficult to perform with input devices found in todays cockpits. Performance with the CSD could benefit from a new user input device and enhanced user feedback modalities that can be operated safely, effectively, and intuitively in a cockpit environment. In this work, we investigate the addition of force feedback in two key CSD tasks: object selection and route manipulation. Different force feedback models were applied to communicate guidance commands, such as collision avoidance and target contact. We also discuss the development of a GUI-based software interface to allow the integration of a haptic device for the CSD. A preliminary user study was conducted on a testbed system using the Novint Falcon force-feedback device. A full experiment, assessing the effectiveness and usability of the feedback model in the CSD, will be performed in the next phase of our research.

Vibrotactile device for rehabilitative training of persons with lower-limb amputation

Individuals with prostheses use sensations experienced through the residual limb during ambulation to perceive the state of the prosthesis. For lower-limb prosthetic users, the loss of proprioception can affect stability and contribute to falls. In this work, we developed a vibrotactile device to enhance residual limb proprioception, which can be used for rehabilitation training of persons with transtibial amputation. The system consists of two vibrating motors, a solenoid, a goniometer, and a control interface. Two types of vibrotactile feedback can be generated: a short vibration via the motors and a single knock via the solenoid. The device is universally designed to fit standard prosthetic components and is inserted in line between the pylon and the socket adapter allowing the vibration to propagate from the unit to the residual limb. The design and the results of the initial functionality test of the vibrotactile unit are reported.

Effects of Type and Strength of Force Feedback on Movement Time in a Target Selection Task

Future cockpits will likely include new onboard technologies, such as cockpit displays of traffic information, to help support future flight deck roles and responsibilities. These new technologies may benefit from multimodal feedback to aid pilot information processing. The current study investigated the effects of multiple levels of force feedback on operator performance in an aviation task. Participants were presented with two different types of force feedback (gravitational and spring force feedback) for a discrete targeting task, with multiple levels of gain examined for each force feedback type. Approach time and time in target were recorded. Results suggested that the two highest levels of gravitational force significantly reduced approach times relative to the lowest level of gravitational force. Spring force level only affected time in target.

Gait Training in Chronic Stroke Using Walk-Even Feedback Device: A Pilot Study

Asymmetrical gait and a reduction in weight bearing on the affected side are a common finding in chronic stroke survivors. The purpose of this pilot study was to determine the effectiveness of a shoe insole device that we developed, called Walk-Even, in correcting asymmetric gait in chronic stroke survivors. Six individuals with chronic (>6 months) stroke underwent 8 weeks of intervention with 2 sessions/week, each consisting of 20 minutes of gait training and 20 minutes of lower-extremity strength training. The 2 control participants underwent conventional gait training, while 4 participants underwent gait training using the Walk-Even. Following intervention, all the participants improved on most of the gait measures: peak pressure of the foot, time of transfer of weight from heel-to-forefoot, center of pressure (COP) trajectory, COP velocity, asymmetry ratio of stance, mean-force-heel, mean-force-metatarsals, Timed “Up and Go,” and Activities-specific Balance Scale. The improvement was more pronounced in the 4 participants that underwent training with Walk-Even compared to the control participants. This pilot study suggests that a combination of strength and gait training with real-time feedback may reduce temporal asymmetry and enhance weight-bearing on the affected side in chronic stroke survivors. A large randomized controlled study is needed to confirm its efficacy.

Effects of Type and Strength of Force Feedback on the Path of Movement in a Target Selection Task

New flight deck technologies being developed under the proposed NextGen National Airspace System will require precise and efficient input from flight crews. The benefits of force feedback for these types of inputs in terms of a reduction in overall movement times have been shown in the past; however, an important component of input efficiency is the path taken by the cursor. The present study investigates the effects of multiple levels of two types of force feedback (gravitational and spring forces) on the path of movement for a target selection task. Mean square error from an ideal straight line path and cursor speeds in terms of the distance from the target were measured. Results suggest that increasing the gravitational force has an effect on path error at short distances and produces higher cursor speeds as the target is approached.

Force-Feedback Integration with NASA’s Next Generation Air Transportation System Cockpit Situation Display

NASA’s Next Generation Air Transportation System Volumetric Cockpit Situation Display is an advanced software tool for real-time flight management from the cockpit. In its current development, users interact with visual elements on the display to retrieve operational information and manipulate objects using a standard computer mouse. In the present study, force feedback is integrated in the Cockpit Situation Display framework and its effectiveness on the performance of two tasks is investigated: object selection and route manipulation. For the object-selection task, an attractive force feedback was used to guide the user to the target. For the route-manipulation task, guidance was generated as variable force feedback, corresponding to the changes to the flight path, relative to obstacles. The effectiveness of the proposed force-feedback models was evaluated in user studies using a Novint Falcon haptic device. It is found that force feedback improves the movement time in the object-selection tasks for diag...

Real-time biofeedback device for gait rehabilitation of post-stroke patients

AbstractIn this work, we develop a device, called ‘Walk-Even’, that can provide real-time feedback to correct gait asymmetry commonly exhibited in post-stroke survivors and persons with certain neurological disorders. The device computes gait parameters, including gait time, swing time, and stance time of each leg, to detect gait asymmetry and provide corresponding real-time biofeedback by means of auditory and electrotactile stimulation to actively correct the user’s gait. The system consists of customized force-sensor-embedded insoles adjustable to fit any shoe size, electrotactile and auditory feedback circuits, microcontroller, and wireless XBee transceivers. The device also offers data saving capability. To validate its accuracy and reliability, we compared the gait measurements from our device with a commercial gait and balance assessment device, Zeno Walkway. The results show good correlation and agreement in a validity study with six healthy subjects and reliability study with seventeen healthy subjects. In addition, preliminary testing on six post-stroke patients after an 8-week training shows that the Walk-Even device helps to improve gait symmetry, foot pressure and forefoot loading of the affected side. Thus, initial testing indicates that the device is accurate in measuring the gait parameters and effective in improving gait symmetry using real-time feedback. The device is portable and low cost and has the potential for use in a non-clinical setting for patients that can walk independently without assistance. A more extensive testing with stroke patients is still ongoing.

A high-speed color-based object detection algorithm for quayside crane operator assistance system

Improvement to user interface technology for port crane operators can lead to safer and more ergonomie environments for cargo transport. An accurate and responsive container-handling guidance system can increase productivity and reduces costs. In this work, a vision-based assistive system for quayside crane operator is developed for collision warning. The system applies a new object edge detection algorithm, called Edge Approaching, to achieve faster detection rate in real-time using a stand-alone embedded system that can be easily integrated to an existing crane interface. Experiments are conducted on a scaled testbed to validate the concept. The proposed algorithms significantly increase the detection rate from as compared to the conventional Canny edge detection and Hough transform method, while maintaining a high accuracy rate of 99%.

Effectiveness of Force Feedback on Movement Time and Path for Target Selection in the Presence of Distractors

Implementation of new, effective onboard computer technologies into commercial cockpits will alter the current role and actions taken by pilots. These new technologies will require precise and efficient input methods due to the unstable nature of a cockpit environment. The benefits of including haptic force feedback for input devices have been shown in previous research. The present study investigated the effects of force feedback distractors on movement time, movement path, and workload when force-enabled distractors were present. Results demonstrated that in the presence of distractors, resistive spring force levels most strongly influenced all performance measures. Attractive gravitational force levels had no impact on movement times and minimal impact on path of movement.