LotusMenu: a 3D menu using wrist and elbow rotation inspired by Chinese traditional symbol

Abstract

Three-dimensional rotation, including pitching, yawing and rolling, is an important gesture in 3D interaction, with the potential to extend the selection capability in large screen interaction system. For 3D rotation interaction, researchers indicated the design space of pen tilt [1] and wrist tilt as input modality [2,3]. However, the combination of freehand 3D rotation gesture has not been fully utilized for menu control. Menu selection in 3D user interface has long been a challenging task. At present, many 3D user interfaces still use linear menu as their selection technology, which is reported to have defects in time performance and user fatigue [3]. The reason is that most in-air freehand gestures involve the movement of shoulder joint and require a larger range of motion. Rotation information in 3D space has not been fully explored, although studies have indicated that these kinds of information could potentially extend the selection capability [2, 4]. In this article, we present a 3D LotusMenu based on the metaphor of Chinese traditional symbol lotus. As shown in Figure 1, this menu is controlled by three-dimensional rotation of user’s wrist and elbow. The proprioceptive design of the menu allows the user to interact intuitively and effectively [5]. We compared user performances in 3D LotusMenu and traditional three-level linear menu. Results show that the LotusMenu is significantly faster than the traditional linear menu, and there is no significant difference in error rate between the two. In addition, our subjective assessment suggests that LotusMenu technique can significantly reduce users’ fatigue. LotusMenu technique. In order to improve the ability of interactive selection and efficiency of the 3D interactive space, as well as to achieve better user performance, we designed the LotusMenu technique based on a metaphor of mapping user’s hand motion with a lotus. The reasons of choosing such a metaphor come from two aspects. On one hand, the lotus petals are distributed in the circumferential direction around the central axis with a multi-layered structure, which fits consistently with the rotation gesture. This makes the menu intuitive and easy to use. On the other hand, the lotus image is one of the most important plant decorative pattern in the history of Chinese ornaments. This well-known ancient pattern naturally has a high degree of acceptance. Therefore, we believe that this metaphor is affordable for the 3D rotation interaction and familiar to users. LotusMenu technique corresponds the lotus shape to the 3D rotation interaction. We map the selection of circular petal groups to the rolling gesture of elbow (Figure 1(a)), the selection of layered petals to the pitching gesture of wrist (Figure 1(b)), and the selection of circular lotus seeds to the yawing gesture of wrist (Figure 1(c)). Since the range of motion of the wrist rolling gesture is limited [6], and it poses a higher risk of injury and more discomfort than forearm rolling [7], we chose to use the forearm instead of the wrist in the rolling gesture. On this basis, we merge the rolling gesture of elbow and the pitching gesture of wrist to select items in first and second level, which corresponds to the selecting of petals in LotusMenu. In this way, there is almost no shoulder movement in this gesture. So the gesture can be easily performed within a small range of motion. In addition, because the rotation of the wrist and arm is merged as much as possible, the fatigue caused by gestures is further reduced. To use the technique, the user first needs to wake up the LotusMenu, and then directly present a gesture with certain elbow rolling and wrist pitching angles to select the first and second level menu item. Finally, the user selects the third level menu items by yawing his/her wrist. The design of interaction is within user’s natural working range as well as direct mapping between hand motion and items. These allow the user to operate with a natural rhythm without adding significant cognitive over-head [5]. The interaction sequence and visual feedback of the LotusMenu are shown in Figure S1. Recognition algorithm. The gesture recognition is based on Leap Motion’s hand tracking technique. We will use three