My Research and Creative Processes

An Improved Mapping Model for Data Sonification
The unreliable detection of information in sonifications of multivariate data is generally thought to be the result of the co-dependency of psychoacoustic dimensions.
This research aims at discovering whether the perceptual accuracy of such information can be improved by rendering the sonification of such data with a mapping-model influenced by the gestural metrics of performing musicians playing notated versions of the data. Conceptually, the Gesture-Encoded Sound Model (GESM) is a means of transducing multivariate datasets to sound synthesis and control parameters in such as way as to make the information in those datasets available to general listeners in a more perceptually coherent and stable way than is currently the case. The approach renders to sound a datastream not only using observable quantities (inverse transforms of known psychoacoustic principles), but latent variables of a Dynamic Bayesian Network trained with gestures of the physical body movements of performing musicians and hypotheses concerning other observable quantities of their coincident acoustic spectra. If successful, such a model should significantly broaden the applicability of data sonification as a perceptualisation technique.

• Authority
• Revelation
• Intuition
• Heuristics, folklore and commonsense
• Inference
  • Deductive inference
  • Inductive inference
  • Bayesian inference
  • Abductive inference
• Reliableness

1. Preparing to perceive. We prepare ourselves to perceive an object by getting into a proper position or attitude in respect to it: we reach towards for to touch, sniff for to smell, smell for to taste, look for to see, listen for to hear.
2. Getting at things. We ready the object to be perceived, by 'getting at' it in some preliminary tentative and easily reversible way that allows us to go on and perceive the object more fully: we touch before grasping, taste before eating, whiff for inhaling, we move (body, attention), to look and listen before (lit. "be for") seeing and hearing.
3. Getting the thing itself. We finally perceive the object - get what we have gotten at; receive what we have made receivable by establishing an affinity between our readied self and our correspondingly prepared object. Hearing is the successful outcome of listening to something listened for.

Stage 2 gives evidence for tentative conceptual conclusions, but the nature of the object is not established until the 3rd stage. Taste is a guide, but the proof of the pudding is in the eating.

Delayed Gratification: Man is no more a slave to his nature in perception than he is in other affairs.

• We make a frequent, enjoyable and significant practice of delayed gratification;
• Holding back this perceptual process so as to prevent its natural completion;
• Perceptual foreplay: stimulating our senses for stimulation's sake;
• A sensualist's way of life; A gourmet does not taste to eat, but eats to taste (savouring).
• It is this perceptual holding-back that is central to the appreciation of the arts as we know them today, as c.f. past times, when the formal structure of works defined cognitive rationalisations of world views.