Cognitive Science Program

Reassessing Syntax-Related ERP Components Using Popular Music Chord Sequences: A Model-Based Approach (2021)
Andrew Goldman, Peter M.C. Harrison, Tyreek Jackson and Marcus T. Pearce
Music Perception, 39 (2), 118-144

Abstract

Electroencephalographic responses to unexpected musical events allow researchers to test listeners’ internal models of syntax. One major challenge is dissociating cognitive syntactic violations—based on the abstract identity of a particular musical structure—from unexpected acoustic features. Despite careful controls in past studies, recent work by Bigand, Delbe, Poulin-Carronnat, Leman, and Tillmann (2014) has argued that ERP findings attributed to cognitive surprisal cannot be unequivocally separated from sensory surprisal. Here we report a novel EEG paradigm that uses three auditory short-term memory models and one cognitive model to predict surprisal as indexed by several ERP components (ERAN, N5, P600, and P3a), directly comparing sensory and cognitive contributions. Our paradigm parameterizes a large set of stimuli rather than using categorically “high” and “low” surprisal conditions, addressing issues with past work in which participants may learn where to expect violations and may be biased by local context. The cognitive model (Harrison & Pearce, 2018) predicted higher P3a amplitudes, as did Leman’s (2000) model, indicating both sensory and cognitive contributions to expectation violation. However, no model predicted ERAN, N5, or P600 amplitudes, raising questions about whether traditional interpretations of these ERP components generalize to broader collections of stimuli or rather are limited to less naturalistic stimuli.

Interval patterns are dependent on metrical position in jazz solos (2021)
Andrew Goldman and Peter Cross
Musicae Scientiae,

Abstract

During jazz improvisation, performers employ short recurrent musical motifs called licks. Past research has focused on the pitch, intervallic, and rhythmic characteristics of licks, but less attention has been paid to whether they tend to start in the same place within the measure (metrical dependence). Licks might be metrically dependent, and where a given lick starts in a measure (metrical position) may thus be part of the performer’s mental representation of that lick. Here we report the use of a corpus study to investigate whether licks are metrically dependent. We analyzed a subset of solos, all those in 4/4 time (n = 435), from the Weimar Jazz Database (WJD; Pfleiderer et al., 2017). Using a sliding window technique, we identified melodic sequences (interval n-grams) between 3 and 10 intervals in length. We counted the number of times each interval n-gram occurred, and noted the metrical position of the initial note of each occurrence, using different levels of quantization (8th and 16th note). We compared the entropy of the distribution of metrical positions for each n-gram—with lower values indicating a stronger metrical dependence—against simulated counterparts that assumed no relationship between an n-gram and its metrical position (no metrical dependence). Overall, we found that shorter n-grams were metrically dependent, with varying results for longer n-grams. We suggest two possible explanations: either mental representations of licks may encode their metrical features or the metrical position may make certain licks more accessible to the performer. On the basis of our findings we discuss future studies that could employ our methods.

Improvisation as a way of knowing (2016)
Andrew Goldman
Music Theory Online, 22 (4),

Abstract

This paper proposes a theory of improvisation as a way of knowing. Different musicians may know about similar musical structures in different ways; different ways of knowing facilitate particular kinds of perception and cognition that underlie different performance behaviors. Some of these ways of knowing can facilitate improvisatory performance practices. The details of these improvisatory ways of knowing can be characterized by psychological and neuroscientific experimental work that compares differences in perception and cognition between groups of musicians depending on their training methods and performance experiences. In particular, perception-action coupling is a promising place to begin making such comparisons. This approach provides an alternative operationalization of improvisation for scientific study that is not susceptible to the problems that arise when describing cognitive processes in culturally contingent and music-theoretically relative terms such as novelty, spontaneity, and freedom, as past experimental work has done. Its hypotheses are also more readily falsifiable. This perspective can also connect an understanding of musical improvisation to other domains of improvisation and to historical and ethnomusicological work, as well as square it with more general theories in cognitive science, such as perception-action coupling. Finally, such a formulation has productive implications for work on improvisation that does engage with concepts like novelty, spontaneity, and freedom that are traditionally invoked in improvisation discourse.

Improvisation experience predicts how musicians categorize musical structures (2020)
Andrew Goldman, Tyreek Jackson and Paul Sajda
Psychology of Music, 48 (1), 18-34

Abstract

Western music improvisers learn to realize chord symbols in multiple ways according to functional classifications, and practice making substitutions of these realizations accordingly. In contrast, Western classical musicians read music that specifies particular realizations so that they rarely make such functional substitutions. We advance a theory that experienced improvisers more readily perceive musical structures with similar functions as sounding similar by virtue of this categorization, and that this categorization partly enables the ability to improvise by allowing performers to make substitutions. We tested this with an oddball task while recording electroencephalography. In the task, a repeating standard chord progression was randomly interspersed with two kinds of deviants: one in which one of the chords was substituted with a chord from the same functional class (“exemplar deviant”), and one in which the substitution was outside the functional class (“function deviant”). For function compared to exemplar deviants, participants with more improvisation experience responded more quickly and accurately and had more discriminable N2c and P3b ERP components. Further, N2c and P3b signal discriminability predicted participants’ behavioral ability to discriminate the stimuli. Our research contributes to the cognitive science of creativity through identifying differences in knowledge organization as a trait that facilitates creative ability.

Towards a cognitive-scientific research program for improvisation: Theory and an experiment (2013)
Andrew Goldman
Psychomusicology: Music, Mind, and Brain, 23 (4), 210-221

Abstract

Despite often being conceived as a spontaneous and creative mode of performance, improvisation is predicated on prior knowledge. What characterizes this knowledge, and how is it represented or recalled differently as compared with other modes of music making? Asking about knowledge and trying to distinguish improvisation as a distinct performance process can locate research questions within the theoretical frameworks of cognitive science, but it is not clear how to make such questions experimentally accessible. Differences arising from music–analytical versus cognitive conceptions of improvisation are explored to provide a theoretical framework compatible with experimentation. Experimental research could concern itself with how the embodied interface between performer and instrument, when manipulated, invokes different cognitive processes of music making, helping to describe the cognitive characteristics of various modes of music performance. Here, an experiment is reported that synthesizes previous techniques used to analyze improvisations with experimental strategies from the neuroscientific literature aimed at differentiating performance processes within a given improviser. Jazz pianists improvised monophonically over backing tracks in a familiar and unfamiliar key as well as with their right and left hands. Among other findings, in some of the less familiar performance situations, participants relied more on diatonic pitches and produced more predictable improvisations as measured by entropy and conditional entropy. The nature of the different underlying processes and knowledge at play under these different conditions is explored, and future research directions to better describe them are identified, including incorporating motor theories of perception.

Live coding helps to distinguish between embodied and propositional improvisation (2019)
Andrew Goldman
Journal of New Music Research, 48 (3), 281-293

Abstract

Live coding – creating live music by writing and executing computer code – problematises conceptions of improvisation. Live coding differs from improvisation on most acoustic and electroacoustic instruments in three ways: sensory feedback content is not systematically linked to human movement, sensory feedback is temporally disjunct from human movement, and decisions made while improvising are discrete rather than continuously negotiated. Phenomenological, neuroscientific, and psychological literatures are considered in light of the case of live coding to distinguish between what can be called propositional versus embodied improvisation. This distinction is relevant outside the context of musical performance and can guide future empirical work.

Contact Improvisation Dance Practice Predicts Greater Mu Rhythm Desynchronization During Action Observation (2019)
Andrew Goldman, Thomas Colleen and Paul Sajda
Psychology of Aesthetics, Creativity, and the Arts.,

Abstract

Many studies have examined mu rhythm desynchronization (MRD) as an electroencephalographic (EEG) correlate of action simulation, including recent studies investigating how dance expertise enhances this effect. However, dance activities differ not only in their movement styles and repertoires but also in their methods and cognitive processes. The present study examines how practicing contact improvisation—a form of dance in which small groups use physical contact and transferred weight to fluently and jointly move together without choreography—predicts this effect independent of other kinds of dance practice. In line with recent work on EEG correlates of creative cognition, we also examine changes in alpha power while contact improvising compared with performing a set choreography. EEG was recorded while participants (N = 18) with a range of types and amounts of weekly dance practice completed an action observation/imitation task, and performed a set choreography and contact improvisation with the experimenter using their right arm. Supporting previous findings from the literature, the amount of practice of choreographed dance trended toward predicting greater MRD while observing actions compared with a rest condition; in addition, the amount of practice of contact improvisation significantly predicted this and with a larger effect size. We also report evidence of differences in posterior alpha between choreographed and improvisation performances, suggesting participants overall engaged more visual imagery while improvising compared with performing the choreography. We explain why these findings bear on the ability to improvise and discuss their meaning within the context of work on the neural correlates of creativity.

Hand Shape Familiarity Affects Guitarists’ Perception of Sonic Congruence (2019)
Keith Phillips, Andrew Goldman and Tyreek Jackson
Auditory Perception & Cognition, 2 (1-2), 82-97

Abstract

Musical performance depends on the anticipation of the perceptual consequences of motor behavior. Altered auditory feedback (AAF) has previously been used to investigate auditory-motor coupling but studies to date have predominantly used MIDI piano in experimental tasks. In the present study, we extend the AAF paradigm to the guitar, which differs from the piano both motorically and in its pitch-to-place mapping, allowing further investigation into the nature of this coupling. Guitarists played chords on a MIDI guitar in response to tablature diagrams. In half of the trials, one of the notes in the heard chord was artificially altered. Participants judged whether the feedback was altered or not, responding as quickly and accurately as possible by pressing one of two buttons on a footswitch. Participants ranked the familiarity of the chord shapes and the hand shapes of the stimuli. Judgement of sonic congruence was faster when the chord and hand shape were familiar, and when feedback was congruent, though there was no interaction between these factors. Our findings suggest that guitarists’ auditory-motor coupling is heterogenous with respect to their technique, and that perception-action coupling operates at the abstract level of the gesture. We discuss implications of these findings with regard to forward models and embodiment.

Classical Rondos and Sonatas as Stylistic Categories (2020)
Jonathan De Souza, Adam Roy and Andrew Goldman
Music Perception, 37 (5), 373-391

Abstract

Sonata and rondo movements are often defined in terms of large-scale form, yet in the classical era, rondos were also identified according to their lively, cheerful character. We hypothesized that sonatas and rondos could be categorized based on stylistic features, and that rondos would involve more acoustic cues for happiness (e.g., higher average pitch height and higher average attack rate). In a corpus analysis, we examined paired movement openings from 180 instrumental works, composed between 1770 and 1799. Rondos had significantly higher pitch height and attack rate, as predicted, and there were also significant differences related to dynamics, meter, and cadences. We then conducted an experiment involving participants with at least 5 years of formal music training or less than 6 months of formal music training. Participants listened to 120 15-second audio clips, taken from the beginnings of movements in our corpus. After a training phase, they attempted to categorize the excerpts (2AFC task). D-prime scores were significantly higher than chance levels for both groups, and in post-experiment questionnaires, participants without music training reported that rondos sounded happier than sonatas. Overall, these results suggest that classical formal types have distinct stylistic and affective conventions.

IU's Music and Mind Lab

Science! The Musical (trailer)

Science! The Musical (full production at Columbia University)

2020 study on the neuroscience of improvisation