Florian Thalmann

The Mobile Audio Ontology: Experiencing Dynamic Music Objects on Mobile Devices

This paper is about the Mobile Audio Ontology, a semantic audio framework for the design of novel music consumption experiences on mobile devices. The framework is based on the concept of the Dynamic Music Object which is an amalgamation of audio files, structural and analytical information extracted from the audio, and information about how it should be rendered in realtime. The Mobile Audio Ontology allows producers and distributors to specify a great variety of ways of playing back music in controlled indeterministic as well as adaptive and interactive ways. Users can map mobile sensor data, user interface controls, or autonomous control units hidden from the listener to any musical parameter exposed in the definition of a Dynamic Music Object. These mappings can also be made dependent on semantic and analytical information extracted from the audio.

The Topos of Music III: Gestures

ion & Generalization & Specialization. For example this means that, starting with a datum as a Latin square (or with the datum of a taquin), we abstract by forgetting some specifications, e.g. the fact that it is presented in a square, and keeping in mind other facts, e.g. the fact that we have permutations in the square and the fact that they are acting on Latin squares (or on states of the taquin). We abstract by forgetting: “penser c’est oublier” [1076], and this is an aspect of mathematical pulsation, its aspect of estrangement, as expressed in our conclusion (Section 67.7), which means that firstly if by any chance we do some combinations or substitutions, we forget some data within the problem—at the risk wasting time—and possibly recognize the rest as being essential when it is seen in a new context, and at last decide to choose that renewed rest to pursue our job. As in traditional Zen archery this forgetting or oversight is an initial moment of relaxation: see [414] for an explanation of the pulsation in relation to Zen. With the accent on permutations, and so on elements of groups of permutations, we reach the more abstract structure of a group. The notion of a group itself comes by abstraction from groups of permutations as well as groups of geometrical transformations, by keeping in mind only some properties of the composition law of these permutations or transformations. At this moment we have brought out the part played by the structure of a group into the configuration of a Latin square. But now a group itself is a new type of configuration, which can be studied and classified, and represented as itself. Of course in this general study of groups, it will be permissible and useful to look at special cases, etc. Groups, lattices, topologies, linear spaces, metric spaces, fields, rings, all these classic structures (the “structures-mères” according to Bourbaki), and also local structures, categories and topoi, arise in a historical process of “structuration” and are basically of the same nature as figures, formulas, configurations, designs or diagrams. However, with a very important precision: an object or a structure, or a type of structure, 67.3 Method and Objects, Summarily Explained: II—Data 1027 cannot just be an arbitrary formal free definition; it has to be fruitful, or already it is itself a theorem (a theorem of existence), or it will be quickly clear that it implies interesting facts about previously unsolved questions. Now, let us give an example. In a particular higher state of mathematics, namely in the mathematical area of algebraic topology, we consider the rather intuitive idea (“intuitive” in this area, of course) that projective modules over commutative rings are like vector bundles on compact spaces. This puzzle has two precise formulations and proofs by Jean-Pierre Serre [969] and then Richard Swan [1028]. We consider that these Serre-Swan theorems are not really different from a puzzle like “taquin”, and our effective recourses for solving are the same. The only difference is that we move at different levels, with different preliminary formation, into another “mathematical history” or personal historical scenery for mathematical activity. In both cases we have to move writings, letters of algebra and figures of geometry, and diagrams; to introduce abbreviations and rules of modifications,; to construct and de-construct; and to structure, by synthesis and analysis. Ultimately any theorem says that two paths of constructions and de-constructions arrive at the same result. In recreational mathematics or in higher mathematics in the construction of the “paths” we have the critical moment when simultaneously we have to invent the “space” in which the path is going now, usually a space of configurations or structures into another space used in a previous step of the path. Our tracks are the road. So creative mathematics are possible as well at the level of recreational mathematics, as in very abstract algebraic topology. The point is the pulsative imagination within analysis-and-synthesis; deeper creation in mathematics is exactly invention of pulsations as methods, that is to say invention of new calculus expressing the tension between analysis and synthesis. 67.3.1.6 Undirectness, Synthetic Thinking and Intuitions The gestures of folding, bending, cutting, erasing, gluing, etc. are our concrete accesses to any understanding or construction or de-construction of mathematical objects. In algebra it is known through factorizing and expanding expressions, in geometry it is realized in the set of geometrical constructions. But it is of a much more general scope, at any level with any model of any structure, presented by generators and relations. We can do these gestures within the system of elements or within the system of relations, or both. For example if we read Gaston Tarry [1036] on the problem of the 36 officers, we have to admit that his analysis proceeds by such gestures; in this way he can construct various coordinations of the hypothetical solution object, with various symmetries, modifications, insertions, quotients, group actions, etc., and this allows us to count data of different types, and to conclude when we find impossible values. This is a process of abstraction, combined with structuration, inventions of figures to observe in the given situation, as permutations, paths, or what he calls magic groups (already considered by Euler under the name of “formules directrices”). Also he reduces the analysis by classifying possible Latin squares in a solution up to isotopies or conjugations. The historical process of structuring by abstraction is parallel to a systematic movement of undirectness in the development of mathematics in the 19th century. For example, concerning the geometry of curves, from the abridged method of Gabriel Lamé to Etienne Bobillier and Julius Plücker, and to Max Noether, followed by Emanuel Lasker and then Emmy Noether, we start from classical problems on curves through the intersections of given curves, and we reach the invention of Noetherian rings. For the following generations, they think geometry with, the notion of a Noetherian ring at the beginning of their thoughts; they could forget concrete curves and Italian geometry, etc. The same “synthetic” fact arrives with groups: whatever would be the very good motivations for groups, once assimilated the explanations of geometries in terms of groups according to Klein, the new generations will think directly in terms of groups. We can perhaps consider that mathematical intuition, which is very transversal to the skill for application of logical rules, comes from the habits of working at a given level of abstraction (with numbers, figures, equations, polynomials, spaces, groups, rings, categories, topos, etc.), i.e., in a certain category of objects, which becomes familiar; objects are now very complex configurations which became simple for the mind; it 1028 67 Mathematical Models of Creativity is the skill to move into a given category with such formerly old data. For instance, someone can have or not have the intuition of the object R (as an object of a category C). Definitely we consider that mathematical activity and mathematical intuition are absolutely relative, starting from the datum of a category as an allegedly natural setting of a first organized intuitive world. 67.3.1.7 Categories, Sets, Groups, Lattices, Structures, out of Logical Concern An idea suggested by several logicians or philosophers [115, 653] is the opposition between sets and categories as a proposal for foundations. More precisely, with Jean-Yves Béziau [115] we can analyze the relations of set theory and category theory with respect to their relation to the question of foundations. Béziau remarks that at least we have three distinct problems: (A): axiomatic foundation of mathematics (to describe a set of axioms from which we can deduce mathematics); (C): conceptual foundation (to describe basic concepts and their links, in order to think mathematics); (L): logical foundation (to show that mathematics are not contradictory). We can add a fourth case: (F): functional foundation (description of a system of mathematical actions which are enough to develop mathematics). Roughly speaking, set theory is good for (A) and (L), and category theory is better for (C) and (F). So the opposition of categories and sets is wrong as an epistemological perspective. Category theory is a method of analysis of the production of mathematical works—and this method itself is mathematical, and a mathematical theory—whereas set theory (and logic) is possibly a ground for all mathematical developments and constructions of structures. We have to completely dissociate the question of observation, description and control of mathematical gestures, using categories, from the question of foundations and logic, using set theory. The control of the size of universes is interesting for questions on existence based on set theoretical constructions. Some mathematicians believe that categoricians are mainly interested in foundational questions, and consequently are unable to see that groups are everywhere, that geometrical thoughts innervate good mathematics. In fact the notion of category unified the two geometrical notions of lattice and group. In [1150, p.37] “category theory” and “structures” are opposed; Bernard Zarca said that the concept of category is a rival of the concept of structure; this is a serious mistake from a historical perspective; it is a wrong interpretation of the discussion in Bourbaki about the question of introduction of categories in Bourbaki’s Elements. In fact, the more significative example of category are the categories of fundamental structures (structures-mères), also because each structure-mère “is” a category. So a group is a category, a lattice is a category, and even, indirectly, any object C of a category C becomes a categor

MusicLynx: Exploring Music Through Artist Similarity Graphs.

MusicLynx is a web application for music discovery that enables users to explore an artist similarity graph constructed by linking together various open public data sources. It provides a multifaceted browsing platform that strives for an alternative, graph-based representation of artist connections to the grid-like conventions of traditional recommendation systems. Bipartite graph filtering of the Linked Data cloud, content-based music information retrieval, machine learning on crowd-sourced information and Semantic Web technologies are combined to analyze existing and create new categories of music artists through which they are connected. The categories can uncover similarities between artists who otherwise may not be immediately associated: for example, they may share ethnic background or nationality, common musical style or be signed to the same record label, come from the same geographic origin, share a fate or an affliction, or have made similar lifestyle choices. They may also prefer similar musical keys, instrumentation, rhythmic attributes, or even moods their music evokes. This demonstration is primarily meant to showcase the graph-based artist discovery interface of MusicLynx: how artists are connected through various categories, how the different graph filtering methods affect the topology and geometry of linked artists graphs, and ways in which users can connect to external services for additional content and information about objects of their interest.

Physical and Musical Multiverses

We shortly discuss the question of unicity in music and physics, a question that in physics has been virulent since the advent of string theory, but which in music has been relevant since the approach to music via individual compositions at the end of the Middle Ages.

Gestures: Gestural Interaction and Gesturalization

We have so far seen that the BigBang rubette allows users to visualize and sonify facts, and create and manipulate them using processes. In the previous chapter, we also discussed that the only structures that BigBang represents internally are processes, only one of which refers to facts in the form of denotators (InputComposition). All other facts are generated dynamically, whenever an operation is added or modified. In order to offer an intuitive way of interacting with the software, we need yet another level: gestures.

Categories of Gestures over Topological Categories

We generalize the topological approach to gestures, and culminate in the construction of a gesture bicategory, which enriches the classical Yoneda embedding and could be a valid candidate for the conjectured space X in the diamond conjecture [720]; see also Section 61.12. We discuss first applications thereof for topological groups, and then more concretely gestures in modulation processes in Beethoven’s Hammerklavier sonata. The latter offers a first concretization of answers to Lewin’s big question from [605] concerning characteristic gestures. This research is a first step towards a replacement of Fregean functional abstraction by gestural dynamics.

Models from Music

We discuss contributions from music theory, performance, and technology to gestural modeling.

Mathematical Models of Creativity

We claim that category theory is a mathematical theory, proceeding from the observation of mathematical activities and gestures, and constructing a mathematical theory as a kind of algebra of these gestures. Especially, categoricians observe their own activity, and so category theory is also constructing a mathematical theory of itself, of its own system of gestures. We imagine that this theory can be used to model any activity, by a parallel action with the categorical activity. This categorical modeling is what we need for a mathematical holding of mathematical creativity because every activity is in fact somehow an activity of modeling.

The Topos of Gestures

This is the third part of The Topos of Music and deals with gestures. We summarize the trajectory gestures took from the first edition of The Topos of Music to the present second edition.

Fundamental Concepts and Associated Categories

This chapter introduces the definition, some basic propositions and first examples regarding the mathematical concept of a gesture for topological spaces. It also includes a short discussion of the topostheoretic logic that is implied by the topos of directed graphs.