Arkady Zgonnikov

To react or not to react? Intrinsic stochasticity of human control in virtual stick balancing

Understanding how humans control unstable systems is central to many research problems, with applications ranging from quiet standing to aircraft landing. Increasingly, much evidence appears in favour of event-driven control hypothesis: human operators only start actively controlling the system when the discrepancy between the current and desired system states becomes large enough. The event-driven models based on the concept of threshold can explain many features of the experimentally observed dynamics. However, much still remains unclear about the dynamics of human-controlled systems, which likely indicates that humans use more intricate control mechanisms. This paper argues that control activation in humans may be not threshold-driven, but instead intrinsically stochastic, noise-driven. Specifically, we suggest that control activation stems from stochastic interplay between the operators need to keep the controlled system near the goal state, on the one hand, and the tendency to postpone interrupting the system dynamics, on the other hand. We propose a model capturing this interplay and show that it matches the experimental data on human balancing of virtual overdamped stick. Our results illuminate that the noise-driven activation mechanism plays a crucial role at least in the considered task, and, hypothetically, in a broad range of human-controlled processes.

Extended phase space description of human-controlled systems dynamics

Humans are often incapable of precisely identifying and implementing the desired control strat-egy in controlling unstable dynamical systems. That is, the operator of a dynamical systemtreats the current control effort as acceptable even if it deviates slightly from the desired value,and starts correcting the actions only when the deviation has become evident. We argue thatthe standard Newtonian approach does not allow such behavior to be modeled. Instead, thephysical phase space of a controlled system should be extended with an independent phasevariable characterizing the motivated actions of the operator. The proposed approach is illus-tratedviaasimplenon-Newtonian modelcapturingtheoperators’fuzzyperceptionoftheirownactions. The properties of the model are investigated analytically and numerically; the resultsconfirmthattheextended phasespacemayaidincapturingtheintricatedynamical propertiesofhuman-controlled systems.

Double-well dynamics of noise-driven control activation in human intermittent control: the case of stick balancing

When facing a task of balancing a dynamic system near an unstable equilibrium, humans often adopt intermittent control strategy: Instead of continuously controlling the system, they repeatedly switch the control on and off. Paradigmatic example of such a task is stick balancing. Despite the simplicity of the task itself, the complexity of human intermittent control dynamics in stick balancing still puzzles researchers in motor control. Here we attempt to model one of the key mechanisms of human intermittent control, control activation, using as an example the task of overdamped stick balancing. In doing so, we focus on the concept of noise-driven activation, a more general alternative to the conventional threshold-driven activation. We describe control activation as a random walk in an energy potential, which changes in response to the state of the controlled system. By way of numerical simulations, we show that the developed model captures the core properties of human control activation observed previously in the experiments on overdamped stick balancing. Our results demonstrate that the double-well potential model provides tractable mathematical description of human control activation at least in the considered task and suggest that the adopted approach can potentially aid in understanding human intermittent control in more complex processes.

Computer simulation of stick balancing: action point analysis

We analyze data collected during the series of experiments aimed at elucidation of basic properties of human perception, namely, the limited capacity of ordering events, actions, etc. according to their preference. Previously it was shown that in a wide class of human-controlled systems small deviations from the equilibrium position do not cause any actions of the systems operator, so any point in a certain neighborhood of equilibrium position is treated as an equilibrium one. This phenomenon can be described by the notion of dynamical traps that was introduced to denote a region in the system phase space where the object under consideration cannot clearly determine the most preferable of the positions that are similar in some sense. According to this concept, the motion of the system in the dynamical trap region is mainly not affected by the operator. The moments of time when the system leaves the dynamical trap region, or in other words, when the operator decides to start or stop the control over the system, are called action points [1]. These moments are seem to be determined intuitively by the operator, and the purpose of our work is to understand the nature of such intuitive decision making process by investigating the action points data obtained from the experiments.

How the Type of Visual Feedback Affects Actions of Human operators: The Case of Virtual Stick Balancing

Maintaining vertical position of an inverted pendulum is a simple balancing task, which is widely used to study human control behavior. Yet, much about this behavior remains poorly understood even in the context of simple virtual tasks. The purpose of this study was to investigate whether the control behavior of human operators depends on the type of visual feedback from the controlled system. We analyze the experimental data on human stick balancing on a computer screen. The previous studies reported detailed analysis of the task performance of human operators observing only the angular deviation of the stick from the vertical. In this study we augmented the information supplied to the operator by linear displacement of the upper tip of the stick from the reference point. This additional information was suggested to improve the performance of the operators. Surprisingly, the subjects not only exhibited better performance, but also supposedly employed structurally different control mechanisms in the linear displacement condition. The found results may have potential implications both for fundamental research aimed at investigating the basic properties of human control, and applied research on human factors.

Unstable Dynamics Of Adaptation In Unknown Environment Due To Novelty Seeking

Learning and adaptation play great role in emergent socio-economic phenomena. Complex dynamics has been previously found in the systems of multiple learning agents interacting via a simple game. Meanwhile, the single agent adaptation is considered trivially stable. We advocate the idea that adopting a more complex model of the individual behavior may result in a more diverse spectrum of macro-level behaviors. We develop an adaptation model based on the reinforcement learning framework extended by an additional processing channel. We scrutiny the dynamics of the single agent adapting to the unknown environment; the agent is biased by novelty seeking, the intrinsic inclination for exploration. We demonstrate that the behavior of the novelty-seeking agent may be inherently unstable. One of the surprising results is that under certain conditions the increase of the novelty-seeking level may cause the agent to switch from the non-rational to the strictly rational behavior. Our results give evidence to the hypothesis that the intrinsic motives of agents should be paid no less attention than the extrinsic ones in the models of complex socio-economic systems.

Dynamical Trap Effect in Virtual Stick Balancing

We present the experimental evidence of the dynamical traps model describing the human fuzzy rationality in the dynamical systems framework. The results of the experiments on virtual stick balancing are compared to the results of the previous studies on the dynamical trap effect. According to the results obtained, we suggest that the dynamical traps model actually captures certain essential features of human fuzzy rationality and therefore may serve as an alternative to the traditional notion of stable equilibrium in describing the behavior of human as an element of complex social systems.

Complex Dynamics of Single Agent Choice Governed by Dual-Channel Multi-Mode Reinforcement Learning

According to the modern theory of adaption of socioeconomic systems to unknown environments only the interaction between agents can be responsible for various emergent phenomena governed by decision-making and agent learning. Previously we advocated the idea that adopting a more complex model for the agent individual behavior including rational and irrational reasons for decision-making, a more diverse spectrum of macro-level behaviors can be expected. To justify this idea we have developed a model based on the reinforcement learning paradigm extended to including an additional channel of processing information; an agent is biased by novelty seeking, the intrinsic inclination for exploration. In the present paper we demonstrate that the behavior of the single novelty-seeking agent may be extremely irregular and the concepts of chaos can be used to characterize it.

Stick Must Fall: Using Machine Learning to Predict Human Error in Virtual Balancing Task

This work presents a new approach to prediction of human control error in unstable systems. We consider virtual inverted pendulum (stick) as a characteristic example of such system. The proposed approach is based on applying classification via machine learning to distinguish between the samples of human control corresponding to successful balancing and critical control errors (resulting in stick fall). To illustrate the approach, we analyze the previously collected data on human balancing of virtual overdamped stick. The obtained results demonstrate that, at least in the considered balancing problem, as much as 73% of human control errors can be successfully predicted in advance (as early as one second before the stick fall).

Concept of Dynamical Traps: Model Systems of Human Actions and Experimental Evidence

Dynamical traps as a new emergence mechanism related to the bounded capacity of human cognition is considered. It assumes that individuals (operators) governing the dynamics of a certain system try to follow an optimal strategy in controlling its motion but fail to do this perfectly because similar strategies are indistinguishable for them. This is described in terms of some neighborhood of the equilibrium point, the region of dynamical traps, wherein each point is regarded as an equilibrium one by the operators. So when a system enters this region and while it is located in it, maybe for a long time, the operator control is suspended. A simple model of oscillator with dynamical traps and the characteristic features of its dynamics are discussed. Experiments on the balancing of a virtual pendulum were conducted to examine the basic features of human control over unstable systems that are expected to be affected by human fuzzy rationality. It is demonstrated that practically only the dimensions of the phase space region wherein a given pendulum trajectory is located depend on the subject age and skill as well as the pendulum parameters determining the difficulty of the balancing. In contrast, the forms of the distribution functions are the same for all the subjects. The data of the virtual experiments are compared to the results of numerical simulation of the oscillator with dynamical traps. The phase trajectories and the phase variable distributions are shown to be similar for the two systems. In addition a chain of oscillators with dynamical traps which mimics cooperative interaction of human operators is considered also. It is, actually, demonstrated that the human fuzzy rationality can cause complex cooperative dynamics in many-element ensembles.