Tuesday, April 26, 2011

A wish list for neuroscientists from a musician

From the safe distance of a composer with an armchair appreciation for science, I'm something of a fan of the neuroscience of music. It's really exciting stuff and my impression is that we're only beginning to learn about it. Here are some aspects of music that I'd like — as a composer — neuroscience to tell us more about; as music if the temporal art par excellence, it's not surprising that they have everything to do with how we process events in time:

1. The two irreversible arrows of music: in pitch and in time. Inverting a sequence of pitches creates only a weak equivalence, indeed an equivalence which deteriorates as one moves towards extremes. And in time, a reversal of a sequence, last-on-first-out, is also only weakly equivalent.

2. The perceptual "borderlands" between parameters: between pitch and timbre or between form and rhythm or rhythm and pitch, including such phenomena as interference beating*.

3. The "chunking" of musical memory: apparently we take in music in pieces which are then reassembled, mentally, into a continuity. (Which is closely related to:)

4. The gaps in timing between mental anticipation of a physical music-producing action, the execution of that action, the perception and analysis of the resulting sound, as well as any motor feedback processing related to the action. (When, exactly, does the music take place? What does "now" mean in music when all of these events occur, in objective external time, at different points? How the hell do musicians ever reach a unison attack?)

5. What other sensory mechanisms contribute or might potentially contribute to an enhanced perception of acoustic events? Research so far has pointed to some limits in the perceptual apparatus and this has led to some practical applications — from transmission of speech over a narrow band to noise reduction to compression and sampling rates. But I'm far more interested, as a musician, in expansion than limitations. For example, I know that a considerable part of "hearing" music, for me, comes from the sense of touch as a complement to the stuff that goes through the ears. AFAIC, fine tuning, in a live ensemble, is often easier by feel than audition, through the reduction of beats more felt than heard. Musicians with hearing losses also know this well, but could this also explain things like La Monte Young's massive sine wave complexes (which don't resolve to simple 5-limit harmonies but share a common, if sub-audio, difference tone)? Also, the physical placement of sounds in space is a fascinating topic. How about echolocation? Some humans — particularly the visually impaired — have become virtuoso echolocaters, but I think I've detected the same in babies crying. What a wonderful extra resource for musicians?


* One of the most interesting results with regard to beating that I've heard about lately is this, a paper on Waves, Beats and Expectancy in speech by Eric Keller. The ways in which speech and music piggy-back on an overlapping set of neural organs is another fascinating topic. While beating in musical contexts is well familiar, that the phenomena was shared with speech was a surprise to me. And while this is wildly prematured and underinformed speculation on my part, wouldn't it be so cool to find a neurological basis, in beating, for poetic and musical metre?


Algorithmic Concepts said...

It's also interesting to think about the Reiki healing effect - probably more of a metaphor than an "effect" per se - in the hands of some masters of the genre such as Liquid Mind (Chuck Wild) or Ali Farka Toure.

I'm also nervous about pattern recognition in music. Roughly: how many notes are sufficient to detect a genre and composer, what minimal amount for a best guess?

It's not unlike a drawing sketch in progress. At the outset you only have two or more lines (which would correspond to an interval as a building block). At this point, it could pretty much become anything.

But once you add more lines to the sketch, the viewer's mind is kind of re-assessing the probabilities of what it is seeing. So after several "runs" the viewer could say: ok, this is starting to look like a silhouette.

Or - ok, with this amount of material provided as sample, I can say with 60% confidence that this is say Roslavets, Krenek or Lady Gaga.

Once the mind has established, with 95% confidence what exactly it is hearing, a certain threshold is crossed and the listener is now evaluating the piece against its understanding of form (template), drawn from its catalog of forms.

Probabilities can get subtle, it's useful to avoid stereotypes and, instead of focusing on limitations, focus on expansion, on creating a shift in probabilities, however small, in your favor.

This is a point laid out so well by Ed Thorp in his "Mathematics of Gambling".

And we are re-assessing probabilities in the creative process as well, aren't we?

When a writer has some initial material, and starts thinking "ok, what do I make of it? where is this taking me?" she is effectively saying to herself: "I am detecting this and this pattern in what I have created, unless I get really electic and/or eccentric, there's a 30% chance that the logic of the plot will take me to point B, a 30% to point B1, and a 40% - to C".

Lisa Hirsch said...

An experienced listener can detect period and often composer in one measure or less. I mean, after you've heard the slow intros to a bunch of classical-period symphonies....

Daniel, I also find this stuff interesting, more so since I realized (duh) how much people vary in their musical abilities. I have had a good musical memory since childhood; I have a friend who can recognize common tunes she has heard frequently but not much more. I know someone who has difficulty picking individual instruments out of an orchestra. My girlfriend claims she can only listen to one part of a multipart piece at once (I'm not sure if this is true but I'm also not sure how to figure that out).

Algorithmic Concepts said...

Re -- An experienced listener can detect period and often composer in one measure or less.

I think you're right. Even if we take famous and well-studied works out of the equation, for more obscure works, still, you can probably make a good guess of period from a surprisingly small sample, from usefully formal stylistic features.

For example, many could tell Metallica by a single riff or maybe even a portion thereof (timbre? manner of playing?).

So I guess what I'm saying is if you start out ONLY with two notes moving up a fifth, as in the two notes opening Bach's Art of Fugue, or WTC Fugue VIII, your options are almost endless.

The same interval open's John Coltrane's "My favorite things" (although the syncopation immediately tells you this is a XX century piece, probably jazz/popular).

Lisa Hirsch said...

In my youth, we played a game where you had to name the piece based on the first chord. :)

Daniel Wolf said...

Lisa, my impression is that there is a lot of good research out there about musical training and brain plasticity. Very good stuff.

The "drop the needle" composer/piece/period/style recognition game is both fun and telling about musical "signatures". Gordon Mumma came up with the idea of an "identicle" (after Lou Harrison's "melodicles") as the smallest unit of music with which a work could be identified. What I find most surprising about such a notion is that both the size and parameters of an identicle appears to be radically variable. The opening chord of the Eroica, for example, has enough information to give away the whole work, but what precisely about that chord does the trick? The key? The chord voicing? The orchestration? Does it work without the pause following it?

Lisa Hirsch said...

Other notable opening chords are the Overture to a Midsummernight's Dream and Mahler's First. In the case of the latter, it's surely the voice, the many-octaves deep D. For the Mendelssohn, the voicing and orchestration - but I would have no problem recognizing either played on a piano, which abstracts out some of the information about the chords.

My friend who has a very poor musical memory had piano lessons at an early age. I can tell you from dancing with her that she has an unusually poor sense of rhythm - cannot feel where the beats are. She has generally poor body awareness, too. I rather think these things go together in some way.

Algorithmic Concepts said...

As far "musical signatures" are concerned, there may also be an interesting legal twist on this.

There's probably a continuum between two extremes - a work entirely different from the original, and a work totally similar to the original, complete equivalence.

There's also "unity of style" - it's ok to copy your own stylistic features.

However, once you get dangerously close to a specific earlier work by someone else, you may be on less firm ground legally.

An interesting example is the "How Deep is Your Love" Bee Gees litigation. (Selle v Gibb).

While the opening chord of Mozart's "Don Giovanni" may be quite characteristic of that particular work, there are many overlaps between Haydn and Mozart.

If Haydn and Mozart had been operating in today's cash-flow driven artistic environment, it would not have been surprising to see a Haydn v. Mozart case with appropriate expert witness testimony and detailed analysis of relevant plagiarism issues.

WARNING: There may also be an Aristotilean 'topoi' twist on the whole issue.

Arguably, the 'topoi' in music, if any, can be used on a gratuitous basis, while their specific manifestations and/or recombinations in a specific work, could result in "striking similarity" between two works, so that access by the suspected plagiarist to the original work (an essential element of the case) can be inferred.