154th Meeting ASA, New Orleans, Nov 2007

Mirroring, not imitation, for the early learning of L1 pronunciation Piers Messum [email protected] http://p.messum.googlepages.com/home 1

Abstract: Mirroring, not imitation, for the early learning of L1 pronunciation To learn to pronounce, a child must learn to produce speech sounds that listeners take to be equivalent to those that they themselves produce. Once this ‘correspondence problem’ is solved, the child can learn the pronunciation of any word by imitating the sequence of its constituent sounds. The foundational skill - learning to pronounce - has also been assumed to be ‘imitative’, but in the different sense of being a mimicry or matching-to-target process applied to the speech signal after some form of normalisation for vocal tract size. However, attending to ephemeral words and speech sounds as meaningful units bars attending to the latter as a signal to be mimicked. Since a child must recognise what is said, basic perceptual psychology provides one reason among several that preclude the possibility of mimicry in the early learning of speech sound qualities. Instead, the normal pattern of mother-infant vocal interaction points to the mother’s reformulation of the child’s output within ‘imitative’ contexts (i.e. vocal mirroring) as enabling the child to establish the correspondences needed. The nature of the representations thus formed resolves, inter alia, the mystery of fast shadowing and the longstanding question of whether speech is fundamentally an acoustic or gestural code. 2

How do children learn the inventory of L1 speech sounds? Notice, first, that ‘learning to pronounce a word’ is different from ‘learning to pronounce’. The former requires a child to recognise and then say the right speech sounds in the right order. (It is uncontroversial that this is one form of imitation.) ‘Learning to pronounce’ must logically precede this. Here a child learns the actions that will produce speech sounds that his listeners take to be equivalent to theirs.

3

To learn to pronounce it is conventionally believed that the child imitates (mimics) the speech sounds he hears. The child makes the judgment of action-to-sound equivalence, based on the similarity he perceives between the acoustic results of his actions and the adult speech he hears. Kuhl’s work (e.g. 1991) suggests that the objective differences in sound quality caused by different size vocal tracts can be resolved by an infant. There are variants of the conventional view1, but all have the child imitating something, and thus creating a common neural representation linking production and perception. 1There

may be time delays in the process (Kuhl), ‘discovery’ of sounds (Davis and MacNeilage), motor representations (Liberman and Mattingly), direct perception of articulation (Fowler and Goldstein), ‘active intermodal matching’ (Meltzoff and Moore), and so on. 4

My attempt was pretty good ...

Conventional account

juice

[dzu]

5

Two problems (among others): 1 A child must attend to an utterance for its meaning, with the attentional set of evaluation (or recognition). Because the speech signal is ephemeral, this precludes mimicry of any embedded speech sound, for which a sensory attentional set would be required1. 2 The observed developmental pattern for speech sounds does not show the continuous improvement one would expect from an ‘imitative’ or ‘matching to target’ account. 1Cf.

the two-stream - dorsal/ventral - theories of visual and auditory perception. Also, a variety of characterisations of modes of listening as auditory/phonetic (Pisoni), sensory-trace/contextcoding (Durlach and Braida), phonetic/phonological (Strange), and so on. 6

How do a child’s perception and production interact in speech development? Awareness of Sensation (AS) Babble

Conventionally, perception is represented as a unitary sense. Here, for example, in a diagram of Vihman’s ‘articulatory filter’.

Evaluating Perception (EP)

Production L0

L1

called Meaningful Perception (MP) in Messum (2007)

Instead, we need a framework for analysis which takes the ‘two stream’ theories of auditory perception into account. Here the streams are clearly separated, left and right, and each will interact with production in a different way.

7

Vocal motor schemes:

Alternative account

<δ> <δ> <δ>

“… generalized action patterns that yield consistent phonetic forms. In a similar vein, Thelen et al. (1996) demonstrated that children’s successive reaches toward an object at 6 months of age showed random variation in trajectory, whereas by 8 months each child showed a consistent trajectory in repeated reaches, presumably based on a consistent and repeatedly utilized “reaching motor scheme.” The vocal motor scheme concept is applicable to any consistently occurring phonetic pattern developed, in theory, by repeated and regularized child action.” McCune and Vihman (2001)

8

Note that <δ> represents a vocal motor scheme: the child’s motor action not the sound that is produced as a result.

<δ> <δ> <δ>

The sound does have two roles to play. First it will be what his mother hears. Second, it will inform the child whether he has executed the gestures as he intended, in common with other sensorimotor feedback that he can make use of for this. However, he does not attend to the sound to compare it to his mother’s production. Instead he will rely on her judgment of equivalence between the results of his action and her output. He can then use the association he has formed in reverse to assemble the component speech sounds of words (as illustrated in the following slides).

9

She seems to think that what I did and what she said are equivalent ... OK, she’s the expert.

<δ>

/dzu:/

10

Well, I think I now know what to do for the first part ...

juice

<δ>

11

Notice that : • The child discovers the actions (VMS’s) for himself • The judgment of equivalence (not necessarily based on similarity) is made by the expert (adult) not the novice (child) • The fundamental correspondence is between action-andsound, not sound-and-sound. This ‘pushmi-pullyu representation’ (Millikan 1996) has a motor head and a perceptual head • No issue of normalisation arises

12

Awareness of Sensation (AS)

Evaluating Perception (EP)

Production Babble

L0

called Meaningful Perception (MP) in Messum (2007)

L1

Word recognition

Early vocal play inc. vocal motor schemes (VMS’s)

by acquaintance (i.e. unmotivated)

Action/sound correspondences, from mother’s reformulation (mirroring) of VMS’s during imitative play

Listening to own output creation of inverse and forward models

ReRe-creating others’ output

… enable word adoption

first adopted words, progr. phon. idioms, pure echolalia, etc

Parsing for production soneme recognition

Listening for meaning

Word assembly

now motivated by potential for communication

based on equivalence, not similarity, of speech sounds

Processes for RE-CREATION of sounds/ MIMICRY

Processes for word (RE-)PRODUCTION

FAST SHADOWING

Processes for word RECOGNITION

generating 2 phonological lexicons Piers Messum, 2008

2 (or 3) routes to word output

13

Evidence either way We do not know how infants learn speech sounds. There is no direct evidence for the conventional view, or any alternative. However, • there are problems with an account of acoustic ‘matching to target’ (see above); • the elements of the ‘mirrored equivalence’ alternative all function in analogous circumstances, lending circumstantial support (next slide); • the ‘mirrored equivalence’ alternative generates coherent explanations for many of the long-standing problems of speech (following slides). 14

Circumstantial support • ‘Robot’ infants successfully learn Japanese and English vowels and speech sounds through mirrored equivalence interactions (Yoshikawa et al. 2003; Howard and Messum 2007) • Adult L2 learners develop better pronunciation when articulatory advice, but no acoustic model, is provided (Gattegno’s Silent Way; Catford and Pisoni 1970) • Infants’ phonological development is ‘shaped’ by their mothers (Goldstein et al. 2003) • Male cowbirds are ‘tutored’ by their non-singing mothers (Smith et al. 2000) 15

Long standing problems that are solved … (1) Scholars from child development have theorised that there must be just a single phonological lexicon, supporting imitation and providing the ‘common currency’ for speech. Evidence from neuroscience contradicts this, supporting the existence of separate input and output lexicons instead. A ‘mirrored equivalence’ account shows how two lexicons are, indeed, developed in parallel in speech acquisition. The existence of two phonological lexicons solves the fis and puzzle/puddle/puggle problems.

16

Long standing problems that are solved … (2) Is the underlying nature of speech that it is an ‘acoustic code’? Or that it is ‘gestures made audible’ (as asserted, for example, by Stetson)? The question sets up a false dichotomy. In a ‘mirrored equivalence’ account the underlying representation created for a speech sound is both perceptual and motor, at the same time. A pushmi-pullyu underlying representation for speech sounds (instantiated, presumably by mirror neurons) solves other puzzles, including how speech can be shadowed with such short latencies. 17

Piers Messum University College London http://p.messum.googlepages.com/home [email protected]

Please ask me (or visit my homepage) if you are interested in the acquisition of timing phenomena in speech, e.g. pre-fortis clipping, ‘stress-timing’, foot level shortening, tense/lax vowel length differences, VOT’s, etc. I argue that these, too, are not acquired by imitation (modelling). Instead, they arise from the aerodynamics/speech breathing of child speech, which differs markedly from the adult model. 18