Shigeoki Hirai

Automatic microassembly system assisted by vision servoing and virtual reality

We proposed an automated micromanipulation workcell for visually servoed teleoperated microassembly assisted by virtual reality techniques. It is composed of two micromanipulators equipped with microtools operating under a light microscope. Visual servoing techniques are applied for efficient and reliable position/force feedback during microassembly tasks. First, a pushing-based micromanipulation strategy for the microobject to follow a planned trajectory is proposed under vision based-position control. Then, we present the cooperation control strategy of the microhandling operation under vision-based force control integrating a sensor fusion framework approach. A guiding-system based on virtual microworld exactly reconstructed from the CAD-CAM databases of the real environment being considered is presented for the imprecisely calibrated microworld. Finally, a planned scenario is executed and experimental results of microassembly tasks performed on millimeter-sized components are provided.

Language-aided robotic teleoperation system (LARTS) for advanced teleoperation

The language-aided robotic teleoperation system incorporates two sets of teieoperational languages with a master-slave manipulator. One offers flexible means, called software jigs, to specify motion constraints superimposed on operational motion, simplifying task motion. The other offers an easy method of teaching elementary tasks which frequently appear in teleoperation. Re-execution of the thus-taught program is effectively utilized.

Application of intelligent monitoring for super long distance teleoperation

Time delay and limited communication capacity are the primary constraints in super-long distance telerobotic systems such as space telerobotic systems. Intelligent monitoring is efficient for this problem to provide a function which selects important scenes to help the operator through a monitoring camera. We constructed a telerobotic testbed which includes a connection through the international ISDN and typical space structure (space robot, truss structure land ORU). We conducted trans-Pacific teleoperation experiments using the testbed in ETL as the remote site and a telerobotic console at JPL (Jet Propulsion Laboratory in Pasadena, California) as a local site. Experimental results showed intelligent monitoring to be effective for the above problems.

Combination of vision servoing techniques and VR-based simulation for semi-autonomous microassembly workstation

Remote control of microrobotic workstations dedicated to microassembly of hybrid MEMS consisting of several separate components are necessary. The operator controls the motion of the microrobots remotely on the motion level. In the future view of industrial applications, these approaches are not suitable. We proposed a concept of a desktop micro device factory for task-level remote control using the combination of vision servoing techniques and virtual reality (VR) environment. It is composed of two micromanipulators equipped with micro tools operating under a light microscope. First, we present the cooperation control strategy of the micro handling operation under vision-based position/force control integrating a sensor fusion framework approach. A guiding-system based on virtual micro-world exactly reconstructed from the CAD-CAM databases of the real environment being considered is presented for the imprecisely calibrated micro world. Finally, some experimental results of microassembly tasks performed on millimeter-sized components are provided.

Intelligent monitoring system for limited communication path: telerobotic task execution over Internet

Time delay and limited communication capacity are the primary constraints in super long distance telerobotic systems. Extension of the intelligent monitoring system is discussed from the viewpoint of the temporal domain. Details of an experimental system which includes real overseas connection through the Internet are shown. Results for a robotic pick-place task experiment using the system are reported.

Information Sharing via Projection Function for Coexistence of Robot and Human

Various kinds of service robots will be used in environment coexisting with humans. To realize the coexistence of a robot and a human, the robot must share task information with the person via an adequate interface. We propose a “projection function” as the interface for information sharing. We construct a projection system to prove the information sharing efficiency, and develop several modes of interaction between the robot and the human in the projection system.

Automatic Camera-work Control For Intelligent Monitoring Of Telerobotic Tasks

In t elerobot ic task execii t ion visual inforinat ion, such as TV monitors, is most important. However, in order that TV monitor supports human operator effectively, the monitor should display scenes relevant to task situation. In converitiorial systems, the human operator controls camera direction and viewing range (zooming) manually along with robot task control. It severely increases the burden of the operator. In this paper, we propose an intelligent control system of the monitoring camera for telerobotic task execution. The system supports the human operator by offering automated camera control synchronized with the process of the task execution. The robot tasks are commanded and executed using teleoperation-oriented robot-language. The proposed framework for the intelligent control system utilizes the hierarchical structiire of the language system. Prototype system with two cameras has been built up. Some results from experiments are also included.

Monitoring of breath sound under daily environment by ceiling dome microphone

This paper proposes a new method for monitoring normal breath sounds in a daily environment where many kinds of noises exist. In a typical conventional breath monitor, a thermistor, an accelerometer, or a contact-type microphone must be attached directly to a persons body such as a part around the nose and the mouth, or a part over the chest wall and the trachea. This paper focuses on key issues stemming from a factor of the environment: extraction of breath sounds from sensor signals contaminated by environmental noise; and realization of a monitoring system suitable for our homes. Features of normal breath sounds are clarified by frequency analysis. Based on the features, a method for extracting normal breath sounds is discussed. This paper also describes a ceiling dome microphone which is a kind of a parabolic sound concentrator. It realizes a function of detecting normal breathing sounds with high sensitivity while keeping a rooms appearance natural. The gain obtained by such a concentrator is analyzed theoretically from the acoustic point of view. Experimental results proved the feasibility of the proposed method.

The surrounding sensor approach - application to sleep apnea syndrome diagnosis based on image processing

This paper describes a new approach to understanding human physiological status through unrestrainedly monitoring his/her body movement. The sensors used for monitoring the human subject are not attached to the body but embedded into the surroundings. This paper clarifies the advantages of the surrounding sensors from the view point of system construction, monitoring of human body movement, and understanding of physiological status. This paper also reports a system based on this approach and proposes a new method of non-invasive and unrestrained diagnosis for sleep apnea syndrome (SAS) using the system. The effectiveness of the proposed method is proved by applying the experimental system to three SAS patients with the help of doctors and nurses at a hospital.

A prototype of standard teleoperation systems on an enhanced VRML

We have been developing a prototype of teleoperation systems via a standard protocol with a standard human interface, where an enhanced VRML is employed as the protocol and a Web browser as the human interface. We clarify what are missing from VRML2.0 to realize teleoperation systems and show how to enhance it. We believe that the employment of a standard communication protocol and a standard human interface can realize teleoperation systems that are able to be used by nonspecialists from any Internet site, or by any people from anywhere.