Imad H. Elhajj

Haptic information in Internet-based teleoperation

Many tasks can be done easily by humans turn out to be very difficult to accomplish with a teleoperated robot. The main reason for this is the lack of tactile sensing, which cannot be replaced by visual feedback alone. Once haptic devices are developed, their potential in many fields is obvious. Especially, in teleoperation systems, where haptic feedback can increase the efficiency and even render some tasks feasible. This paper studies Internet-based teleoperation systems that include haptic feedback, concentrating on the control of such systems and their performance. The potential of this technology and its advantages are explored. In addition, key issues, such as stability, synchronization, and transparency are analyzed and studied. Specifically, an event-based planning and control of Internet-based teleoperation systems is presented with experimental results of several implemented system scenarios in micro- and macro-scales.

Supermedia-enhanced Internet-based telerobotics

This paper introduces new planning and control methods for supermedia-enhanced real-time telerobotic operations via the Internet. Supermedia is the collection of video, audio, haptic information, temperature, and other sensory feedback. However, when the communication medium used, such as the Internet, introduces random communication time delay, several challenges and difficulties arise. Most importantly, random communication delay causes instability, loss of transparency, and desynchronization in real-time closed-loop telerobotic systems. Due to the complexity and diversity of such systems, the first challenge is to develop a general and efficient modeling and analysis tool. This paper proposes the use of Petri net modeling to capture the concurrency and complexity of Internet-based teleoperation. Combined with the event-based planning and control method, it also provides an efficient analysis and design tool to study the stability, transparency, and synchronization of such systems. In addition, the concepts of event transparency and event synchronization are introduced and analyzed. This modeling and control method has been applied to the design of several supermedia-enhanced Internet-based telerobotic systems, including the bilateral control of mobile robots and mobile manipulators. These systems have been experimentally implemented in three sites test bed consisting of robotic laboratories in the USA, Hong Kong, and Japan. The experimental results have verified the theoretical development and further demonstrated the stability, event transparency, and event synchronization of the systems.

Real-time control of Internet based teleoperation with force reflection

The use of the Internet is no longer limited to the transmission of data. In the past few years many successful attempts have been made to use the Internet as a command transmission media; through which control can be sent to remote systems and feedback can be obtained. But with this media come several limitations: delay, lost packets and disconnection. All of these limitations may cause instability in the system especially if the system loop is closed. All the previous work addressing these problems assumed several conditions; for example, time delay is constant or has an upper bound, control is not in real-time. A new real-time control approach is presented that deals with these limitations without any assumptions made regarding delay. The approach is based on event-based control, which was implemented on a mobile robot over the Internet. The commands sent to the robot are velocity and the feedback is force based on the environment. It will be shown that this approach results in a stable system. In addition, a new force feedback generation method is used.

Supermedia in Internet-Based Telerobotic Operations

In the past decade robotics and the Internet, fed by the great advances in computing and networking, matured into giant interdisciplinary scientific fields. Therefore, it is not surprising that many are trying to merge these two technologies and develop Internet-based robotic teleoperation. More interestingly, Internet-based bilateral teleoperation, where supermedia is fed back to the operator in order to increase efficiency and achieve telepresence. Supermedia is the collection of multimedia (video, audio, ...), haptic and other sensory information. This paper studies supermedia enhanced teleoperation via the Internet, concentrating on the real-time control of such systems and their performance. The potential of this technology and its advantages will be explored. In addition, key issues, such as stability, synchronization and transparency, will be analyzed and studied. Specifically, event-based planning and control of Internet-based teleoperation systems is presented with experimental results of several implemented system scenarios.

A 2-D PVDF force sensing system for micro-manipulation and micro-assembly

Despite the enormous research efforts in creating new applications with MEMS, the research efforts at the backend such as packaging and assembly are relatively limited. We present our ongoing development of a polyvinylidene fluoride (PVDF) multi-direction micro-force sensing system that can be potentially used for force-reflective manipulation of micro-mechanical devices or micro-organisms over remote distances. Thus far, we have successfully demonstrated 1D and 2D sensing systems that are able to sense force information when a micro-manipulation probe-tip is used to lift a micro mass supported by 2 /spl mu/m/spl times/30 /spl mu/m/spl times/200 /spl mu/m polysilicon beams. Hence, we have shown that force detection in the 50 /spl mu/N range is possible with PVDF sensors integrated with commercial micro-manipulation probe-tips.

Synchronization and control of supermedia transmission via the Internet

The growth of the Internet has been accompanied by an increase in its application. Many new Internet based applications have been introduced during the past decade. One of the most interesting of these is teleoperation, where the Internet is used as a bridge between operators and machines. The operator sends commands and receives visual and haptic feedback. Collectively, we refer to all those streams as supermedia. Teleoperation over the Internet comes with several problems: delay, lost packets and disconnection. All of these limitations may cause instability and desynchronization in teleoperation systems especially if those systems include haptic feedback. Most of the previous work in Internet based teleoperation rests on many limiting assumptions: for example, time delay is constant or has an upper bound, control is not in real-time. This paper presents a new real-time haptic feedback system that deals with these limitations and difficulties without any assumptions made regarding the time delay. The approach is based on event based control, which has been implemented on a mobile robot and a mobile manipulator over the Internet.

Cooperative teleoperation of a multirobot system with force reflection via Internet

With the rapid development of information technology, the Internet has evolved from a simple data-sharing media to an amazing information world where people can enjoy various services. Recently, the use of the Internet has been expanded to the field of automation, i.e., using the Internet as a tool to control equipment located at remote sites. This work presents a system that enables multiple operators at different sites to cooperatively control multiple robots with real-time force reflecting via the Internet. To overcome instability and reliability problems caused by random time delay of the Internet communication, we adopt an event as the reference for the controller design. To improve real-time efficiency and reduce the complexity of the controller, a distributed approach is proposed for the control of remote robots, so that the time delay in one control loop does not affect performance of the others. A vision-based method is developed to monitor and render interactions between the robots. The usefulness and effectiveness of the developed method and system have been verified by teleoperation experiments on a two-robots cooperative system among Hong Kong, mainland China, and the U.S.

Modeling and control of Internet based cooperative teleoperation

Robotic operations carried out via the Internet face several challenges and difficulties. These range from human-computer interfacing and human-robot interaction to overcoming random time delay and task synchronization. These limitations are intensified when multi-operators at multisites are collaboratively teleoperating multirobots to achieve a certain task. In this paper, a new modeling and control method for Internet-based cooperative teleoperation is developed. Combining Petri net model and event-based planning and control theory, the new method provides an efficient way to model the concurrence and complexity of the Internet-based cooperative teleoperation. It also provides an efficient analysis tool to study the stability, transparency and synchronization of the system. Furthermore, the new modeling and control method enables us to design an Internet-based cooperative telerobotic system that is reliable, safe and intelligent. This new method has been experimentally implemented in a three site test bed consisting of robotic laboratories in the USA, Hong Kong and Japan. The experimental results have verified the theoretical development and further demonstrated the advantages of the new modeling and control method.

Design and Analysis of Internet-Based Tele-Coordinated Multi-Robot Systems

The coordination of multi-robots is required in many scenarios for efficiency and task completion. Combined with teleoperation capabilities, coordinating robots provide a powerful tool. Add to this the Internet and now it is possible for multi-experts at multi-remote sites to control multi-robots in a coordinated fashion. For this to be feasible there are several hurdles to be crossed including Internet type delays, uncertainties in the environment and uncertainties in the object manipulated. In addition, there is a need to measure and control the quality of tele-coordination. To this end, the measure of force sensed by each robot is suggested and justified as a coordination index. It was proven that if n robots are event-transparent and event-synchronous then they can be teleoperated under random delay conditions to coordinate to any index value, which is feasible under no delay conditions. The design procedure that ensures a system can satisfy a small coordination index was presented and analyzed.In addition, the design and analysis of event-synchronous systems using Petri Nets is detailed. The Petri Net design methodology is presented for both event-synchronous single operator single robot teleoperation systems and event-synchronous multi-operator multi-robot teleoperation systems.The theory developed was tested by bilaterally tele-coordinating two mobile manipulators via the Internet. The experimental results confirmed the theoretical results presented.

Wireless Sensor for Continuous Real-Time Oil Spill Thickness and Location Measurement

Marine pollution by oil spills is a devastating environmental hazard, requiring a low-cost efficient system for continuous and real-time thickness measurement and localization of oil. Knowing that none of the previous detection methods has managed to fully meet these requirements, it is necessary to devise a new technique for guiding and speeding up the clean-up process of oil spills. This paper presents a sensor device that is capable of sensing, processing, and transmitting information about an oil spill (location and thickness). This paper discusses two new methodologies of detection based on the difference in the absorbance spectral signatures and electric conductivity properties of oil and water. This paper also discusses the mechanical design of the device, the hardware implementation of its components, and the integration and evaluation of the whole system. The experimental results confirm the effectiveness of the proposed method under different lighting, salinity, temperature, and sea conditions.