Jacek Malec

A new Q-learning algorithm based on the metropolis criterion

The balance between exploration and exploitation is one of the key problems of action selection in Q-learning. Pure exploitation causes the agent to reach the locally optimal policies quickly, whereas excessive exploration degrades the performance of the Q-learning algorithm even if it may accelerate the learning process and allow avoiding the locally optimal policies. In this paper, finding the optimum policy in Q-learning is described as search for the optimum solution in combinatorial optimization. The Metropolis criterion of simulated annealing algorithm is introduced in order to balance exploration and exploitation of Q-learning, and the modified Q-learning algorithm based on this criterion, SA-Q-learning, is presented. Experiments show that SA-Q-learning converges more quickly than Q-learning or Boltzmann exploration, and that the search does not suffer of performance degradation due to excessive exploration.

On the integration of skilled robot motions for productivity in manufacturing

Robots used in manufacturing today are tailored to their tasks by system integration based on expert knowledge concerning both production and machine control. For upcoming new generations of even more flexible robot solutions, in applications such as dexterous assembly, the robot setup and programming gets even more challenging. Reuse of solutions in terms of parameters, controls, process tuning, and of software modules in general then gets increasingly important. There has been valuable progress within reuse of automation solutions when machines comply with standards and behave according to nominal models. However, more flexible robots with sensor-based manipulation skills and cognitive functions for human interaction are far too complex to manage, and solutions are rarely reusable since knowledge is either implicit in imperative software or not captured in machine readable form. We propose techniques that build on existing knowledge by converting structured data into an RDF-based knowledge base. By enhancements of industrial control systems and available engineering tools, such knowledge can be gradually extended as part of the interaction during the definition of the robot task.

Knowledge-Based Reconfiguration of Automation Systems

This article describes the work in progress on knowledge-based reconfiguration of a class of automation systems. The knowledge about manufacturing is represented in a number of formalisms and gathered around an ontology expressed in OWL, that allows generic reasoning in description logic. In the same time multiple representations facilitate efficient processing by a number of special-purpose reasoning modules, specific for the application domain. At the final stage of reconfiguration we exploit ontology-based rewriting, simplifying creation of the final configuration files.

How to Make a Challenging AI Course Enjoyable Using the RoboCup Soccer Simulation System

In this paper we present an AI programming organised around the RoboCup soccer simulation system. The course participants create a number of software agents that form a team, and participate in a tournament at the end of the course. The use of a challenging and interesting task, and the incentive of having a tournament has made the course quite successful, both in term of enthusiasm of the students and of knowledge acquired. In the paper we describe the structure of the course, discuss in what respect we think the course has met its aim, and the opinions of the students about the course.

Knowledge and Skill Representations for Robotized Production

Model-based systems in control are a means to utilize efficiently human knowledge and achieve high performance. While models consisting of formalized knowledge are used during the engineering step, running systems usually do not contain a high-level, symbolic representation of the control and most of its properties, typically named numerical parameters. On a system level and beyond the plant data, there is also a need to represent the meaning of the data such that deployment and fault analysis could be augmented with partly automated inference based on the semantics of the data. To that end, we extended the formalized knowledge traditionally used in control to include the control purpose, engineering assumption, quality, involved state machines, and so on. We then represented the control semantics in a format that allows an easier extraction of information using querying and reasoning. It aims at making knowledge in control engineering reusable so that it can be shipped together with the control systems. We implemented prototypes that include automatic conversion of plant data from AutomationML into RDF triples, as well as the automated extraction of control properties, the conversion of parameters, and their storage in the same triple store. Although these techniques are standard within the semantic web community, we believe that our robotic prototypes for semantic control represent a novel approach.

On Distributed Knowledge Bases for Robotized Small-Batch Assembly

The flexibility demands in manufacturing are severe, e.g., for rapid-change-over to new product variants, while robots are flexible machines that potentially can be adapted to a large variety of production tasks. Task definitions such as explicit robot programs are hardly reusable from an application point-of-view. To improve the situation, a knowledge-based approach exploiting distributed declarative information and cloud computing offers many possibilities for knowledge exchange and reuse, and it has the potential to facilitate new business models for industrial solutions. However, there are many unresolved questions yet, e.g., those related to reliability, consistency, or legal responsibility. To investigate some of these issues, different knowledge-based architectures have been prototyped and evaluated by confronting the solution candidates with key application demands. The conclusion is that distributed cloud-based approaches offer many possibilities, but there is still a need for further research and better infrastructure before this approach can become industrially attractive.

Mobile robot cooperation in simple environments

The paper presents the problem of organizing cooperation among heterogeneous mobile robots. It is assumed that cooperation of a number of robots is necessary to achieve the goal. Another constraint is introduced by simplicity of the control system of each robot. A simple model of cooperation is introduced and verified experimentally using Khepera robots with different sets of resources.

Describing constraint-based assembly tasks in unstructured natural language

Task-level industrial robot programming is a mundane, error-prone activity requiring expertise and skill. Since humans easily communicate with natural language (NL), it may be attractive to use speech or text as instruction means for robots. However, there has to be a substantial amount of knowledge in the system to translate the high-level language instructions to executable robot programs. In this paper, the method of Stenmark and Nugues (2013) for natural language programming of robotized assembly tasks is extended. The core idea of the method is to use a generic semantic parser to produce a set of predicate-argument structures from the input sentences. The algorithm presented here facilitates extraction of more complicated, advanced task instructions involving cardinalities, conditionals, parallelism and constraint-bounded programs, besides plain sequences of commands. The bottleneck of this approach is the availability of easily parametrizable robotic skills and functionalities in the system, rather than the natural language understanding by itself. (Less)

Knowledge-Based Industrial Robotics

When robots are working in dynamic environments, close to humans lacking extensive knowledge of robotics, there is a strong need to simplify the user interaction and make the system execute as autonomously as possible. For industrial robots working side-by-side with humans in manufacturing industry, AI systems are necessary to lower the demand on programming time and expertise. We are convinced that only by building a system with appropriate knowledge and reasoning services, we can simplify the robot programming sufficiently to meet those demands and still get a robust and efficient task execution. In this paper, we present a system we have realized that aims at fulfilling the above demands. The paper focuses on the ontologies we have created for robotic devices and manufacturing tasks, and presents examples of AI-related services using the semantic descriptions of the skills to help the user instruct the robot adequately. (Less)

Driver support in intelligent autonomous cruise control

Presents a driver support system developed for an automatic intelligent cruise control test car. The system gathers information to be passed to the driver, processes it and presents it using either a visual or an aural output channel. Although the actual support provided, as well as the ultimate form of the displayed or pronounced messages, remain to be decided upon after a series of well-designed experiments, the design of the system helps programmers to work in an incremental fashion and to alter implementation details easily.