Shoji Tatsumi

Q-MAP: a novel multicast routing method in wireless ad hoc networks with multiagent reinforcement learning

Multicast plays an important role in ad hoc networks, and multicast algorithms have the goal of directing traffic from sources to receivers maximizing some measure of network performance combining the processes of routing and resource reservation. This paper discusses some current literature about multicast routing in mobile ad hoc networks. Further, by investigating the swarm-based routing method and the multiagent reinforcement learning applications, this paper analyses the possibility and merit of adopting the reinforcement learning method in a multicast routing protocol for wireless ad hoc networks. And based on the above, this paper presents a novel multicast routing method, the Q-MAP algorithm, that ensures the reliability of the resource reservation in the wireless mobile ad hoc networks. The features and efficiency of the Q-MAP multicast routing method are also illustrated in this paper.

A Competitive Information Recommendation System and Its Behavior

Information recommendation systems draw attention of practitioners in B-to-C electronic commerce. In an independent recommendation system such as in www.amazon.com, a user cannot compare the recommended item with ones from other information sources. In a broker-mediated recommendation system such as in www.dealtime.com, the broker takes the initiative of recommendation, and the information provider cannot recommend its item directly to the user.In this paper, we propose a competitive information recommendation system consisting of multiple animated agents that recommend their items competitively, and discuss the advantages through showing a prototype developed for restaurant recommendation. Each agent recommends restaurants from its own point of view and the user tells good or bad about them. In our competitive information recommendation system, the user can compare items recommended from multiple agents, and the information providers can recommend their items directly to the user through its animated agent. We also show that the competitive nature affects the output depending on the number of participating agents.

A pattern defect inspection method by parallel grayscale image comparison without precise image alignment

For automatic visual inspection of patterns on printed wiring boards and/or patterned wafers, this paper presents a new defect detection method for grayscale images without precise image alignment. Most of the conventional visual inspection algorithms based on grayscale reference comparison require precise image alignment with precision of subpixel or within /spl plusmn/1 pixel; however, it is difficult to succeed in the precise image alignment in every image. While a defect inspection method without precise image alignment has been previously proposed for binary images, the expansion to grayscale images we discuss is indispensable for detecting more minute defects. We propose dynamic tolerance control based on grayscale morphology to reduce false defects on pattern edges, and use gray dilation operation so that a weakness of the original method for binary images, an inability to detect the absence of minute patterns, is overcome. Theoretical analysis and experimental results show that the proposed method is capable of detecting subpixel-sized defects, and has practical detection performance.

Multiagent reinforcement learning method with an improved ant colony system

Multiagent reinforcement learning has gained increasing attention in recent years. The authors discuss coordination means for sharing episodes and sharing policies in the field of multiagent reinforcement learning. From the point of the view of reinforcement learning, we analyse the performance of indirect media communication among multi-agents on an ant colony system which is an efficient method that uses pheromones to solve optimization problems. Based on the above, we propose the Q-ACS method, modifying the global updating rule in ACS for learning agents to share better episodes benefited from the exploitation of accumulated knowledge. Meanwhile, taking the visited times into account, we propose T-ACS by presenting a state transition policy for learning agents to share better policies, benefiting from biased exploration. To demonstrate the coordination performance of learning agents in our methods, we conducted experiments on an optimization problem, the traveling salesman problem. Comparison of results with ACS, Q-ACS and T-ACS show that the improved methods are efficient for solving the optimization problem.

Application of multiagent reinforcement learning - to multicast routing in wireless ad hoc networks ensuring resource reservation

Mobile Ad hoc Networks (MANETS) are selforganized wireless networks, and each mobile node in the network acts as a router and forwards packets on behalf of other nodes. Multicast routing is becoming an important networking service in MANETS, which objective is to find optimal routes from a source node to all multicast destinations and use the network resource effectively. This paper investigate the possibility and merit of applying Reinforcement Learning (RL) into the multicast routing in MANETS. And, taking advantage of the multiagent RL, this paper proposes a novel multicast routing algorithm, the Q-MAP method, that ensures the resource allocation and delay-bounded in mobile ad hoc wireless networks. Further, this paper analyses the convergence and rationality of the Q-MAP method from the point of view of RL, and verifies the efficacy of the proposed method by simulations of route creation.

Assigning Vocation-Related Information to Person Clusters for Web People Search Results

Distinguishing people with identical names is becoming more and more important in searches on the Web. This research assigns useful labels to help users select person clusters that are separated into different people from the result of person searches on the Web. We proposed a method to label person clusters with vocation-related information (VRI). VRI includes broader terms that may not be considered vocations as well as terms that are useful to infer vocations, not only those rigorously defined as vocations. Our method is comprised of two processes: (a) extraction of VRI candidates using HTML structure and heuristics, and (b) VRI generation using term frequency, clustering synonyms, and calculation using a Web search engine. Experimental results revealed the usefulness of our proposed method.

A detection method of mura on a coated layer using interference light

Here, we describe a method to detect mura during the display devices manufacturing process on a uniformly coated thin photoresist layer. A mura is an irregular variation of lightness on a uniformly manufactured surface. Display devices are manufactured through photolithographic process, and every imaging process requires a uniformly coated photo resist layer. Mura detection on the layer is necessary to keep the device quality high. The mura has been visually inspected by tilting a glass under a sodium lamp in order to create the largest amount of interference. However, tilting the glass is difficult due to the growing device size currently four square-meters in the latest line. Rather than tilting the glass, we have developed an apparatus which illuminates the glass via a set of narrow bandpass filtered light of different wavelength. The apparatus observes the interference of light reflected from the surface and from the bottom of the thin layer in order to inspect the mura on the glass. The reflection intensity as a function of the layer thickness shows a sinusoidal periodic characteristic, which means that the mura detection sensitivity depends strongly on the optical path length in the layer. We have developed a method to compensate the periodic sensitivity fluctuation by employing a reflection ratio that enables us to detect the local thickness fluctuation with an accuracy of one nanometer.

Assigning Location Information to Display Individuals on a Map for Web People Search Results

Distinguishing people with identical names is becoming more and more important in Web search. This research aims to display person icons on a map to help users select person clusters that are separated into different people from the result of person searches on the Web. We propose a method to assign person clusters with one piece of location information. Our method is comprised of two processes: (a) extracting location candidates from Web pages and (b) assigning location information using a local search engine. Our main idea exploits search engine rankings and character distance to obtain good location information among location candidates. Experimental results revealed the usefulness of our proposed method. We also show a developed prototype system.

Convergence of the Q-ae learning under deterministic MDPs and its efficiency under the stochastic environment

Reinforcement learning (RL) is an efficient method for solving Markov decision processes (MDPs) without any priori knowledge about an environment. Q-learning is a representative RL. Though it is guaranteed to derive the optimal policy, Q-learning needs numerous trials to learn the optimal policy. By the use of the feature of Q value, this paper presents an accelerated RL method, the Q-ae learning. Further, utilizing the dynamic programming principle, this paper proves the convergence to the optimal policy of the Q-ae learning under deterministic MDPs. The analytical and simulation results illustrate the efficiencies of the Q-ae learning under deterministic and stochastic MDPs.

Single-agent and Multi-agent Approaches to WWW Information Integration

The WWW is a most popular service on the Internet and a huge number of WWW information sources are available. Conventionally we access WWW information sources one by one by using a browser, but WWW information integration gives a unified view to users by integrating multiple WWW information sources elaborately. In this paper, we introduce our single-agent and multi-agent approaches to WWW information integration.