From sounds and experiments to models

Psycholinguistic experiments

Experiment types

Lexical decision tasks
spoken or written
word played/displayed
yes/no button
'is this your language?'
accuracy and speed measured

Priming
a word seen or heard second time recognised faster


semantic/conceptual priming
a related word seen or heard recognised faster
(nurse primes doctor, etc.)


syntactical / morphological
endings of words
sentence structures

Cross-modal priming
read first, heard second
less effective than single modality


cross-linguistic priming
1st language, 2nd language
less than single language

Gating
gradually increase duration of a word
determine when word is recognised


hear 1st fragment
make guess/state confidence
hear 2nd fragment

  • we don't need to hear whole words
  • reverse gating shows start more important than end
  • neighbourhood effects - similar words near, more info needed

Shadowing
listen and repeat as fast as possible

  • 25% of women close shadowers
    250-300ms
  • 75% women + all men distant shadowers
    500ms
  • time taken to produce syllable = 150ms
  • subjects typically correct errors
  • shows don't just repeat but process
  • used with dichotic listening - repeat only one stream

understanding barrier to copying

Click detection

  • played speech with click inserted
  • clicks mask speech sounds
  • asked where heard click
  • typically report clicks in different places
  • believe heard sounds were masked (vice versa)
    • phonemic restoration
  • sometimes coughs
  • Words recognised faster than non-words
  • impossible nonwords excluded faster than possible non-words
  • accuracy vs. speed - inverse correlation

general findings

  • word frequency effects
    • more frequent words recognised faster
  • word supremacy effect
    • actual words faster than non-words
  • context effect
    • words recognised in context (primed)
  • distortion effects
    • location of distortion matters
    • beginning more damaging
    • in written, ends and beginnings important
  • clause boundaries have special status
  • phrase boundaries less so
  • we process speech very quickly
    • including corrections, and at same time as speech

Models
a theory of how the mind does certain things
sometimes designed to be used in computers to test

top down or bottom up
logical assumption

  • speech understanding bottom up (sounds processed)
    • Broca
  • speech production top down
    • Wernicke

(Wernicke seems to be activated first)

Modular
language as a module
separate modules within language module

non-modular
fully connectionist

serial modular
module works
send output to next module
module stops working

interactive modular
module can work with previous module

McClelland & Rumelhart

Phonology

Speech sounds = phones
depends on: position in a word, accent, etc.
some variation leads to different meaning


representation of speech sounds (in minds) = phonemes
abstract mental representations

Phonemes
the phoneme /t/ can be
top
stop
butter
in English substitution doesn't alter meaning

babies

  • at 1 mo can distinguish phonemes
    • prefer own name, mothers voice, etc.
  • at 4mo can distinguish phonemes from other languages
    • parents can't
  • 6-8 mo no longer distinguish vowels from other languages
  • 8-12 mo no longer distinguish consonants from other lanugages

problems recognising words

  • speed
    • english 20-30 phonemes per second
    • non-speech sounds at 1.5 per second
    • 160-200 wpm (more inf rench)
  • speaker variation
    • variations in pronounciation (regional, formality, emotion, mode e.g. conversation or reading
    • no two utterances are identical
  • segmentation problem
    • continuous stream of sounds
    • no predictable pauses between words
    • catgorical perception

categorical perception

  • perception of different thigns as the same category
    • different pronounciations of /k/ as all being k
    • why lack of invariance not a problem
  • not limited to speech
  • not limited to humans

impoverished signal

  • vowels perceived better than Cs
    • stronger/louder
  • stop consonants in vowels

highly redundant speech
beeb can be diff from bab

  • vowels duration & coarticulation important
    • lip shapes for koo kee kah

acoustic cues
place of articulation

experiment
listen for errors
errors noticed

  • more at beginnings
  • in predictable sequences more than unpredictable
  • more in certain types of sound switches

Word Recognition

  • word frequency effects
  • (im)possible word effect #
  • priming effect
  • semantic priming effect

other effects

  • beginnings of words more important#
  • more than one word at a time is considered - blends
  • slips of the ear (hear something incorrectly)

Need models that account for these

Need models that account for:

  • word frequency effects
  • (im)possible word effect #
  • priming effect
  • semantic priming effect
  • beginnings of words more important#
  • more than one word at a time is considered
    • blends
    • shadowing
  • slips of the ear (hear something incorrectly)

McGurk effect
visual input helps decide sound