Jan Jaap R. van Assen

Shape, motion, and optical cues to stiffness of elastic objects

Nonrigid materials, such as jelly, rubber, or sponge move and deform in distinctive ways depending on their stiffness. Which cues do we use to infer stiffness? We simulated cubes of varying stiffness and optical appearance (e.g., wood, metal, wax, jelly) being subjected to two kinds of deformation: (a) a rigid cylinder pushing downwards into the cube to various extents (shape change, but little motion: shape dominant), (b) a rigid cylinder retracting rapidly from the cube (same initial shapes, differences in motion: motion dominant). Observers rated the apparent softness/hardness of the cubes. In the shape-dominant condition, ratings mainly depended on how deeply the rod penetrated the cube and were almost unaffected by the cubes intrinsic physical properties. In contrast, in the motion-dominant condition, ratings varied systematically with the cubes intrinsic stiffness, and were less influenced by the extent of the perturbation. We find that both results are well predicted by the absolute magnitude of deformation, suggesting that when asked to judge stiffness, observers resort to simple heuristics based on the amount of deformation. Softness ratings for static, unperturbed cubes varied substantially and systematically depending on the optical properties. However, when animated, the ratings were again dominated by the extent of the deformation, and the effect of optical appearance was negligible. Together, our results suggest that to estimate stiffness, the visual system strongly relies on measures of the extent to which an object changes shape in response to forces.

Visual Features in the Perception of Liquids

Summary Perceptual constancy—identifying surfaces and objects across large image changes—remains an important challenge for visual neuroscience [1, 2, 3, 4, 5, 6, 7, 8]. Liquids are particularly challenging because they respond to external forces in complex, highly variable ways, presenting an enormous range of images to the visual system. To achieve constancy, the brain must perform a causal inference [9, 10, 11] that disentangles the liquid’s viscosity from external factors—like gravity and object interactions—that also affect the liquid’s behavior. Here, we tested whether the visual system estimates viscosity using “midlevel” features [12, 13, 14] that respond more to viscosity than other factors. Observers reported the perceived viscosity of simulated liquids ranging from water to molten glass exhibiting diverse behaviors (e.g., pouring, stirring). A separate group of observers rated the same animations for 20 midlevel 3D shape and motion features. Applying factor analysis to the feature ratings reveals that a weighted combination of four underlying factors (distribution, irregularity, rectilinearity, and dynamics) predicted perceived viscosity very well across this wide range of contexts (R2 = 0.93). Interestingly, observers unknowingly ordered their midlevel judgments according to the one common factor across contexts: variation in viscosity. Principal component analysis reveals that across the features, the first component lines up almost perfectly with the viscosity (R2 = 0.96). Our findings demonstrate that the visual system achieves constancy by representing stimuli in a multidimensional feature space—based on complementary, midlevel features—which successfully cluster very different stimuli together and tease similar stimuli apart, so that viscosity can be read out easily.

Inferring the stiffness of unfamiliar objects from optical, shape, and motion cues

Visually inferring the stiffness of objects is important for many tasks but is challenging because, unlike optical properties (e.g., gloss), mechanical properties do not directly affect image values. Stiffness must be inferred either (a) by recognizing materials and recalling their properties (associative approach) or (b) from shape and motion cues when the material is deformed (estimation approach). Here, we investigated interactions between these two inference types. Participants viewed renderings of unfamiliar shapes with 28 materials (e.g., nickel, wax, cork). In Experiment 1, they viewed nondeformed, static versions of the objects and rated 11 material attributes (e.g., soft, fragile, heavy). The results confirm that the optical materials elicited a wide range of apparent properties. In Experiment 2, using a blue plastic material with intermediate apparent softness, the objects were subjected to physical simulations of 12 shape-transforming processes (e.g., twisting, crushing, stretching). Participants rated softness and extent of deformation. Both correlated with the physical magnitude of deformation. Experiment 3 combined variations in optical cues with shape cues. We find that optical cues completely dominate. Experiment 4 included the entire motion sequence of the deformation, yielding significant contributions of optical as well as motion cues. Our findings suggest participants integrate shape, motion, and optical cues to infer stiffness, with optical cues playing a major role for our range of stimuli.

Cues Underlying Liquid Constancy

Fluids and other deformable materials have highly mutable shapes, which are visibly influenced by both intrinsic properties (e.g. viscosity) and extrinsic forces (e.g. gravity, object interactions). How do we identify a liquid’s intrinsic properties across profound variations in shape caused by extrinsic factors? Previous findings suggest we are surprisingly good at matching viscosity across large variations in shape (“liquid constancy”). Here we ask which visual cues enable us to do this. Somehow the visual system abstracts features that are common to different instances of a liquid, while suppressing large differences in shape caused by extrinsic factors. In this study we tried to specify which geometric features observers use to achieve liquid constancy. We simulated eight variations of pouring liquids with seven different viscosities (‘test stimuli’). Each variation was influenced by a different noise force field, like gusts of wind that changed the way the liquid flowed, leading to substantial shape differences over time. Observers adjusted the viscosity of another variation (‘match stimulus’) until it appeared to be the same material as each test. We tested several time offsets to create volume differences between test and match stimuli. The experiment was performed with static and one-second moving stimuli. We find that observers show a high degree of constancy in matching the viscosity across the different variations. However, volume differences between test and match stimulus, especially with static stimuli, caused large effects of over- and under-estimation of viscosity. We then analyzed the 3D shapes of the samples to extract a wide range of shape measurements related to viscosity. We find that a number of cues related to curvatures, periodic movements of the liquids, and the way they spread out predict aspects of the observer’s performance, but that humans achieve better constancy than the cues predict.