Masashi Sugimoto

The proposal for deciding effective action using prediction of internal robot state based on internal state and action

For a robot working in a complicated environment, it is virtually impossible to predict all possible situations and to pre-program the robot with all suitable reaction patterns for each of the possible situations. Because robots are required to act differently in different situations. Furthermore, robots should be able to adapt to different environments by deciding upon the course of action to take depending on the situation, in addition to pre-registered commands, in a manner similar to humans. However, hardware and the limited computational resources pose a physical limitation, so the robot needs some time to decide its course of action. In this context, if robots will be able to select the most appropriate action quickly and can cancel the time delay caused by the limitations mentioned above. Moreover, if depending on the action a robot takes, the future internal state space will vary infinitely. If we take this point into consideration, we need to simultaneously predict the internal state and the action that the robot adopt. The purpose of this research is to compensate the current action as the appropriate action using next time and future actions that robots will take. For achievement of this, first, we state advance prediction using Online SVR as a learner. This Online SVR predicts the future internal robot state - i.e., the robots next internal state to be taken. Furthermore, this predictor will be useful for predicting the distant future internal robot state, using the internal state that the robot adopt repeatedly. Second, we determine the future action from Optimal Feedback Controller using predicted future internal state - i.e., the robots next action to be taken. In this paper, we designed a controller using LQR (Linear Quadratic Regulator) and use as determine an action. This paper presents the results of these studies and discusses methods that allow the robot to decide its desirable behavior quickly, using the state predicted. As an application example, we used two-wheeled inverted pendulum, and compared the results of the proposed method with the actual response of the inverted postural control task.

Real-time sequentially decision for optimal action using prediction of the state-action pair

We previously reported that an approach to predict the changes of the state and action of the robot. In this paper, to extend this approach, we will attempt to apply the action to be taken in the future to current action. For the achievement of this point, firstly, we will attempt to apply the action to be taken in the future, to the current action, by extending the former approach. We will apply the prediction of the State-Action Pair that has former proposed method. This method predicts the robot state and action for the distant future, using the state that the robot adopt repeatedly. Accordingly, we will obtain the actions that the robot to be taken in the future. In addition, we consider the point that the state and the action of the robot will be changed continuously. In this paper, we propose the method that predicts the state and the action every time when the robot decide an action. By using this method, we will obtain the compensate current action. This paper presents the results of these studies and discusses methods that allow the robot decides its desirable behavior quickly, using the state predicted combined with optimal control method.

The proposal for real-time sequential-decision for optimal action using flexible-weight coefficient based on the state-action pair

For a robot that works in a dynamic environment, the ability to autonomously cope with the changes in the environment, is important. In this paper, we propose an approach to predict the changes of the state and action of the robot. Further, to extend this approach, we will attempt to apply the action to be taken in the future, to the current action. This method predicts the robot state and action for the distant future using the state that the robot adopts repeatedly. By using this method, we can predict the actions that the robot will take in the future. In addition, we consider that the state and the action of the robot will change continuously and mutually. In this paper, we propose a method that predicts the state and the action each time the robot decides to perform an action. In particular, in this paper, we will focus on how to define the weight coefficients, using the characteristics of the future prediction results. By using this method, we will obtain the compensatory current action. This paper presents the results of our study and discusses methods that allow the robot to decide its desirable behavior quickly, using state prediction and optimal control methods.

A Study on the Deciding an Action Based on the Future Probabilistic Distribution

In case of operating the robot in a real environment, its behavior will be probabilistic due to the slight transition of the robots state or the error of the action that is taken at each time. We have previously reported that prediction of the state-action pair, is the prediction method to link the state and action of the robot for future the state and the action. From this standpoint, we have proposed the method that decides the action that tends to take in the future. In this paper, we will try to introduce the statistical approach to the prediction of the state-action pair. From this standpoint, we propose the method that decides the action that tends to take in the future, for the current action. In the proposed method, we will calculate the existence probability of the state and the action in the future, according to the normal distribution.

A Study of Predicting Ability in State-Action Pair Prediction: Adaptability to an Almost-Periodic Disturbance

When a robot considers an action-decision based on a future prediction, it is necessary to know the property of disturbance signals from the outside environment. On the other hand, the properties of disturbance signals cannot be described simply, such as non-periodic function, nonlinear time-varying function nor almost-periodic function. In case of a robot control, sampling rate for control will be affected description of disturbance signals such as frequency or amplitude. If the sampling rate for acquiring a disturbance signal is not correct, the action will be taken far from its actual property. In general, future prediction using machine learning is based on the tendency obtained through past training or learning. In this case, an optimal action will be determined uniquely based on a property of disturbance. However, in this type of situation, the learning time increases in proportional to the amount of training data, either, the tendency may not be found using prediction, in the worst case. In this paper, we focus on prediction for almost-periodic disturbance. In particular, we consider the situation where almost-periodic disturbance signals occur. From this perspective, we propose a method that identifies the frequency of an almost-periodic function based on the frequency of the disturbance using Fourier transform, nearest-neighbor one-step-ahead forecasts and Nyquist-Shannon sampling theorem.

ADAPTABILITY TO PERIODIC VARIABLE DISTURBANCE USING PROBABILISTIC STATE-ACTION PAIR PREDICTION

When operating a robot in a real environment, its behavior is probabilistic because of slight transition of the robot’s state or error in the action taken at a given time. In this case, it is difficult to operate the robot using rule-based-like action decision methods. Therefore, ad-hoc-like action decision methods are needed. A method is proposed for deciding on future actions based on a robot’s present information. The state-action pair prediction method has been reported; it links the state and future actions of a robot using internal information. A statistical approach to state-action pair prediction has been introduced previously, in which the existence probability of a state and action in the future is calculated according to the normal distribution. This paper considers the situation where a command input is sent to an inverted pendulum. Based on this command input, the shape of the floor is changed from flat to undulating. The results of verification experiments confirm that the proposed method can adjust the shape of the floor autonomously.