Patricia Search

Interactive Multisensory Data Representation

Large data sets require new forms of data representation that reduce the complexity of the information, and help users identify trends and communicate the meaning of the data to diverse audiences. With multisensory data design, it is possible to increase the number of variables and relationships that can be represented simultaneously. Sound, touch, gesture, and movement can enhance the perception of data relationships. Complex data sets also require new ways of organizing databases that encourage the development of new perspectives and facilitate collaboration. Audiovisual metadata is an alternative to text-based metadata that supports data exploration by providing a flexible format for database organization. With these new approaches to data representation, it is important to understand the semiotics of multisensory data design.

The Semiotics of the Digital Image

Western formalism and postmodernist theory do not provide an adequate framework for interpreting many forms of digital art. Using artwork from the 1950s to the present, the author shows how the semiotic structure of the digital image defines a new visual aesthetic in which symbols become interpretations of symbols, and multiple levels of graphic encoding take on discursive characteristics similar to linguistic syntax. The author examines the semiotics of the digital image within the context of philosophical developments in mathematics and science.

The Rhythm and Structure of Multicultural Communication

Although global computer networks make it possible to quickly exchange words and images, cultural differences that reflect the psychological, historical, and social forces of individual cultures are usually lost in the translation. This paper uses research in cognitive psychology, computer-mediated communication, and the authors own research in hypermedia communication to show how computer networking is changing the psychodynamics of communication. The paper also shows how contemporary artists are an important link to understanding the rhythm and structure of multicultural communication. By studying the works of these artists, we can gain valuable insights into how to design multimedia networks that preserve cultural identities and channel the energy from these diverse perspectives into an expansive view of reality.

Multisensory Physical Environments for Data Representation

This paper reviews theoretical research and projects in data representation that use different sensory modalities, embodiment, physical objects, and immersive environments. Other topics include the impact of cross-modal perception on data representation and the role audiovisual aesthetics play in the interpretation of data. Research has shown that cross-modal perception enhances sensory stimuli. Sound, touch, gesture, and movement engage the user and create holistic environments that provide multi-dimensional representations of complex data relationships. These data representations include data sculptures, ambient displays, and multisensory environments that use our intuitive abilities to process information from different sensory modalities. By using multiple senses, it is possible to increase the number of variables and relationships that can be represented simultaneously in complex data sets.

Usable Content in a Post-Document World

The Usable Content Project aims to develop a set of useful paradigms for the analysis, design, and testing of usable content in a post-document world. In planning work supported by the STC, we have brought together a multidisciplinary team of Rensselaer faculty and students to explore a variety of post-document exemplars and develop an over-arching framework for what makes them usable. Our suggestion is that post-documents move users from control through identity and toward community, using a process clearly different from traditional documents. As a consequence, traditional metrics of usability - efficiency, accuracy, and satisfaction - are no longer adequate for post-documents

Ancient voices and cyberspace: exploring the past to reshape the future

Explores the aesthetics of narrative, visual images, music and action in primitive oral cultures as a foundation for new forms of human-computer interaction in electronic communication. Primitive cultures throughout the world use storytelling, visual designs, music and ceremonial rituals to define personal and social identities as well as to define position and navigation in the physical world. In primitive cultures, communication is defined by time-based structures, not fixed structures. These communicative structures are always in flux, always in the state of becoming. This paper illustrates the parallels between the temporal forms of communication in primitive cultures and the time-based structures in interactive electronic communication. The paper examines how the aesthetics of primitive designs and music can form the foundation for new computer interfaces that build perceptual bridges between cultures.

Electronic Art and the Law: Intellectual Property Rights in Cyberspace

The dematerialization of art that began in the 1960s has reached new heights with the use of electronic media. We are at an important crossroads in defining intellectual property rights that will have a direct impact on the way we create and disseminate electronic art in the future. This paper examines the historical evolution of the definition of author in copyright law. The paper shows how current copyright legislation and recent court decisions do not address the plasticity of the medium and the new forms of authorship that are defined by the artistic use of techniques such as virtual reality, artificial intelligence, hypermedia links and collaborative networking.

The Semiotics of Space and Time in Interaction Design.

Since the 1960s artists have been experimenting with the multiple dimensions of electronic art. Their art can inform other disciplines, including interaction design. Digital art has led to new semantic structures that can be applied to human-computer interface design. This paper presents an analysis of the semiotics of the digital image and interaction design, and the new semantic structures that are defined by digital art research and praxis, kinesthetic design, cognitive mapping, and cross-modal perception. The paper also proposes the development of new software tools for designing and visualizing the information architecture, navigation, and cognitive and sensory relationships in a virtual information space. These tools can facilitate the exploration of the unique semantic dimensions that define these interactive environments.