Please enable JavaScript.
Coggle requires JavaScript to display documents.
Reading and eye movements (Y1), Screenshot 2024-03-14 18.45.27, Screenshot…
Reading and eye movements (Y1)
Physical eye structure and acuity
Cornea / lens -> pupil -> retina in the fovea (photoreceptors) -> optic nerve
Vision is clearest where photoreceptor density is highest in the fovea
Periphery is away from the fovea
-> parafovea - moderate acuity - between peripheral and fovea
Measurement is in degrees of retinal angle
Cortical processing emphasises fovea from V1 and up
Studying syntax - Mahowald et al, 2016 - sampled 100 journals and found that 11% of judgements were not significant when participants were in a forced choice or acceptability judgement task
5% showed no significance in either task
Cannot trust linguistics
Writing systems - all spoken languages have phonemes or basic speech sounds which can be combined in various ways, but written scripts differ markedly in the extent to which, as well as the ways in which, this phonetic information is represented
4 main types of script -
Logographic - symbolic language in which morphemes are represented pictorally, or symbols are associated with meaning over sound
Syllabic - using a symbol for each syllable
Consonatal - scripts represent the consonants of the language
Alphabetical - letters represented the phonemes or sounds of the language
How do we read -
Stops - moving the eye on a word-by-word basis across the page
Gilchrist et al (1998) - moving the material has been observed in patients with extraocular muscular fibrosis, but she had the same head movements as the eye movements of reading - function transferred
Some languages combine scripts - e.g. Japanese uses both kana (syllabic) and kanji (logographic)
Early writing systems were pictographic - pictorial or iconic representations of the meaning of the word
Graphemes - written representation of a phoneme - transparent or shallow - orthographic languages because there is one to one correspondence between letters and symbols, but opaque or orthographically deep languages by contrast where the same sound has multiple meanings
Children who learn shallow orthography show advantages over those learning deep orthography
Fixations and saccades
Fixations - where our eyes remain typically still, roughly for 250-300ms
Saccades - the ballistic movement of our eyes to another location (up to 700 degrees a second) - once direction has been decided, saccades are completed - you cannot change direction mid saccade
Saccadic suppression - the apparent inhibition of visual input during saccades
Skipping and re-reading -
-> We only fixate on about 2/3 words when reading for comprehension (in English)
-> In addition, 10-15% of fixations are regressions of fixating on previous text
-> This means we have 4 different metrics that can be measured in order to explore different experimental effects
Fixation durations
Regression rates
Word skipping rates
Saccade distance / speed
Perceptual span - how many characters can we process in one saccade -
Gaze contingent - display changes based on where reader is fixating
Moving window paradigm (Rayner and McConkie, 1975) - found the perceptual span is 3-4 characters to the left and 14-15 characters to the right in English - asymmetric - cannot see other lines
-> However, fovea is symmetrical - perceptual span mismatches - psychological effect of basing attention and how much information we take in - this assymetry is caused by us reading from left to right, as we already have read the left hand side and so need to take in more of the right hand side
Factors that affect the perceptual span - based on the language that you know -
In languages such as Hebrew (right to left) the perceptual span is the same size, but in reverse (Pollatsek et al, 1981)
Japanese readers have a perceptual span that is 13 characters right when reading left to right, but 6 characters down when reading top to bottom (Osaka, 1993)
Chinese readers have a perceptual span 1 character to the left and 3 to the right (Inhoff and Liu, 1998)
Span increases and decreases as a function of text difficulty (Rayner, 1986)
Liversedge et al, 2016 - the attentional span of sentences
-> Found sentences that had the same syntax order - smaller sentence length has a different perceptual span
-> Finnish - informationally sparse; more characters for same amount of stuff
-> Chinese - informationally dense - less characters for more stuff
-> Compared reading times - Chinese takes fewer fixations but are longer, whereas Finnish people take many fixations but are much quicker
In bilingual people - perceptual span depends on the language they are reading:
Psychological effect that is based on what is most efficient for your language - perceptual spans are the same but in different directions as required
Perceptual spans adapt to reading
Adults - when you are a skilled reader, you read at the same speed as any skilled reader in any language
Eye-tracking
Scleral search coil - uses small electrical current changes caused by eye movements to monitor them - < 0.1 degree of error
EOG - electrodes placed around the eyes - as an eye movement is made, the changes in electrical field measure direction of movement
Dual purkinje image eye tracker - infrared light illuminates the eye and a combination of lenses and mirrors reflect this light, showing any movements made
Eyelink - infrared light illuminates the eye, the corneal reflection is monitored using a camera that samples the location of the eye at 1000Hz
Tobii eye trackers - typically has a 90-120Hz sampling rate of the eye, allows for head movement
How it works - camera and IR illuminator (this is pointed at iris, and the pupil and corneal reflection is measured)
Other methods -
EEG - use of electrophysiological changes on the scalp during language comprehension to detect processing differences - excellent temporal resolution, poor spatial resolution
Self-paced reading - one or a few words are presented at a time, and the time is takes the participant to press a button for more of the text is used as an index of reading time; very poor temporal resolution - yields many different findings to eye movements, which is a more ecologically valid method of reading as it does allow for word skipping and regressions
Lexical decision tasks - making a categorisation decision of a word / non-word - yields interesting effects, but tells us only of word recognition, not the interpretation of a sentence
Doing questionnaire experiments
Computational analysis of text corpora
Eye movement findings of word recognition
Word frequency effect - reaction times are faster to words seen more frequently;
experiment in which there was a common word, a medium word and less used word (low, medium and high vocab rarity conditions) - those with better vocab had shorter reaction times and higher recognition as the word frequency decreased
Up to 300ms difference in lexical decision task, 67ms in eye movement and reading studies (Brysbaert, Mandera and Keuleers, 2018)
-> Probability of word known increased the probability of word frequency - lower word frequency meant faster recognition from those with better vocab, and low recognition in low vocab people and a longer response time
Measures of word frequency -
Kucera-Francis Corpus - used in most studies up until the late 90s - only features 1.014 million words, from sources that are the most typical of our language use
SUBTLEX - subtitles of 10,000 films and TV programmes and features 51 million words (Brysbaert and New, 2009)
Word length - longer words take longer to recognise
Word predictability - if a word is contextually constrained, it is recognised quicket
Phonology - when given the preview of a homophone, readers recognise words 20ms quicker than a control word
Phonological code is automatically accessed during word recognition (Rayner et al, 1995)
Makes reading easier
Phonological neighbourhood size - a larger phonological neighbourhood makes word recognition quicker
Neighbours are words that differ by one phoneme
Suggests that letter position is not equal in importance during word recognition - word-beginning letters are important
Transposed letter effect - as long as first and last letter are the same, all other letters in the word can be whatever order
Disappearing text - we still see a frequency effect for words even when they are not on the page, and this is because we are comprehending them still and we can recognise them when they appear again
Mindless reading - words are not processed in this case, and so there is no frequency efect (Foulsham et al, 2013)
Eyes move in a sequential pattern, but no actual reading is happening - stop engaging in linguistic processing
Context effects on visual word recognition -
word superiority effect; shows letters within a letter string are detected more readily if they are presented within a word compared with a non-word in a non-letter way
Parallel processing of letter information - letter position does not effect word recognition - pseudowords are over nonwords
Context has considerable effects - semantic priming produces faster word recognition
Text comprehension
Syntactic effects - parsing; assigning syntactic roles to the components of sentences
Garden path sentences emerge when the original parsing of the sentence is shown to be incorrect due to its syntactic ambiguity
Reanalysis of a sentence to determine prediction
Chomsky - aimed to establish a set of rules, a grammar, that would describe a well-formed grammatical sentence but produce no ungrammatical ones
Distinction between deep and surface structure - same underlying structures and meaning, but different appearances
Active and passive voice sentences - same meaning, different surface
irreversible passive sentences take same processing time as active; reversible passive is easier
Active sentences - where the person is first
Passive sentences - object first
Broca's aphasics - struggle with reversible sentences due to agrammatism
Wernicke's aphasics - struggle with both as they have semantic inability
Phrase structure tree - syntactic processing - a graphic representation of the syntactic structure of a sentence
Sentence -> noun -> article noun (the girl) ->
OR - sentence -> verb phrase -> verb ->
OR sentence -> verb phrase -> noun phrase -> article + noun
The girl ate an ice cream
Parsing role - assign incoming words the appropriate role in the sentence as simply and effectively as possible - strategies:
Minimal attachment - simplest tree structure consistent with the grammar of the language by using as few syntactic roles as possible
Late closure - attaches incoming material to the phrase currently being processed, as long as that is grammatically permissible, reducing the load on working memory - clause we process remains open as long as possible, and therefore incoming input is associated with the more recent clause
Frazier (1979) - serial models make parsing incremental in allocating a word to a syntactic role as the word is perceived
Interactive view suggests semantics influence syntax, and that the levels of language interact
Garden path sentences require a revision of the initial interpretation of a sentence as new, conflicting information is presented, but this does not always cause an ideal sentence structure, and revision may not be consistent (good enough but not ideal)
Sentence processing - incremental interpretation - readers semantically interpret and syntactically parse text on a word-by-word basis
-> Minimal attachment - readers try to interpret sentences with the simplest possible syntactic sentence
-> Incremental interpretation - visual world paradigm - participants eye movements are tracked whilst participant listens to narrative (Altmann and Kamide, 1999)
Semantic processing - plausible outcomes - we immediately detect semantic anomalies suggesting that we are incrementally interpret semantic aspects
Rayner et al, 2004 - implausible and anomalous sentences noticed - stop when we notice something is implausible and regress back, but we stop immediately for anomalous sentences
Can be overriden by context due to pragmatics (Filik, 2008) - peanuts in love, only realise when we have more context the real semantic meaning
Rayner et al, 1986 - we preview a set of words based on the word before, and we read sentences quickly when they fit the preview; we take the semantic meaning of the word and we are semantically primed for certain word expectations
-> When the semantic prime mismatches, it takes longer to read a sentence
Readers store words based on their semantic associative meaning
Text comprehension - background knowledge, vocabulary, language structures, verbal reasoning and literacy knowledge
Word recognition - phonological awareness, decoding and sight recognition
Eye movements and attention is driven by cognition
Levy (2008) - realistic theory of sentence comprehension -
Processing of input that is not perfectly formed
How we resolve ambiguity
How we make inferences from incomplete input
How we overcome difficulty in sentence-grammatical structures etc
Word recognition is a lexical access process - retrieve meaning from lexical store
Two main models of lexical access -
Serial search models - orthographical -> phonological -> semantic occurring in three separate stages, where they are accessed one at a time in a sequence - perceptual / pattern analysis - entries arranged by frequency in the lexicon
Direct access models - logogen model (Morton, 1970) - propose parallel word access - activation of the word process from the information in the 3 planes is processed in a parallel manner
Methodologies - word naming and sentence verification tasks
Frequency effects apply to open-class words and closed-class words
Priming effects - repetition priming
Lexical ambiguity - have multiple representations and are treated differently - more than one meaning will be activated at the same time and then it is eliminated - context-dependent (connectionist triangle model)
Models of word recognition - rely on the notion that words have a mental lexicon (Rayner et al, 2004)
A store of all lexical representations with their semantic meaning, syntactic role and phonology encoded within these representations
There is not really a single neural structure that can be attributed to all of this language knowledge
Interactive activation model - feature units inform letter units which then inform word units (McClelland and Rumelhart, 1981) - each identification layer activates the next
Higher frequency words (used more often) are said to have a lower threshold for activation
More predictable words are said to be primed so they have already received from activation, hence the threshold is reacher quicker
This is also the explanation for semantic and phonological priming
Dual route cascade model - Colthart et al, 2001 -
printed speech - feature analysis -> letter analysis -> orthographic input lexicon (route 2 - meaning, route 3) -> phonological output lexicon or grapheme-phoneme translation (route 1) -> phoneme system -> speech
Route 1 - grapheme-phoneme conversion; grapheme is the visual unit that corresponds to a phoneme
-> Conversion rules used to convert each grapheme into a phoneme
-> Rules determined by the most common grapheme-phoneme association in the language
-> Good for regular words and non-words, bad for irregular words
Route 2 - Lexicon + semantics (semantic is vague in this model)
Route 3 - lexicon only
-> Orthographic lexicon stores the spelling of all words you know
-> Activates meaning and or phonology (hence semantic and phonological priming effects)
-> Good for reading all familiar words, bad for unfamiliar words - frequency and word length effects
Connectionist triangle model - two possible routes to take from you spelling to sound (Harm and Seidenberg, 2004)
Direct pathway from orthography to phonology
Indirect pathway from orthography to phonology via semantics and context
-> highly interactive model - all types of knowledge can influence meaning
Eye movements and neurodevelopmental disorders
Eye movements and dyslexia - oculomotor dysfunction hypothesis - everyone with dyslexia has atypical or incorrect eye movements
Despite early data showing comparable performance in savage latency, accuracy, velocity as well as fixation location and duration for general eye movement behaviour
In fact, general eye movement performance is fairly consistent regardless of reading level in terms of acuity
The differences are a reflection of information processing systems - it is not the eyes but the language process
Dyslexic readers make more fixations, shorter fixations and have a larger frequency and length effect
Phonology deficit hypothesis - those with dyslexia struggle to sound out words and dyslexia is from deficits accessing and manipulating phonological information - changing the first two words causes a delay - incorrect phonological information harms them more (Kirkby et al, 2021)
Not all readers replicate this (Castles and Colthart, 1996)
Others show different deficits in contrast sensitivity, working memory or attention
Dyslexia is a multiple deficit issue - multiple risk factors lead to different cognitive and behavioural issues (Pennington, 2006)
Eye movements and autism - ASD readers show no differences in frequency or syntactic attachment effects during reading compared to neurotypicals (Howard et al, 2017)
Some differences come in semantic anomalies - passage level (whole passage is undermined) or sentence level anomalies (in those words) - Au-Young et al, 2017
Individuals with ASD process contextual information less efficiently than local information - detect local anomaly more efficiently and were delayed in detecting context anomalies - weak central coherence theory (Frith et al, 1989)
Eye movements and DCD - children and adolescents with DCD perform more poorly in terms of educational attainment - median of 2 GCSEs compared to 7 in controls (Harrowell et al, 2018)
Can be attributed to poorer handwriting, manipulation speed and increased fatigue
However, they also have poor reading and spelling ability
Higher levels of reading delay
Frequency results - DCD readers seem to lexically access words similar to TD readers
Attachment results - syntactic errors; only TD readers replicate the attachment effect of immediate disruption (anomalous sentences, syntax attached is wrong) - recognise it later and takes longer to renalayse in DCD
Reading relies on executive control and working memory and cognitive control when processing information
Less skilled readers struggle more with having more fixations and retention fixating on less important information more often, also due to weaker memory
Summary - there is a wealth of evidence to suggest that how people read has a direct effect on their eye movement behaviour
This can range from word identification effects, such as those of word frequency and predictability to more 'higher level' semantic and pragmatic processes of sentence interpretation
There are several models of word recognition, but the idea that this is the entire story with regard to reading is not the case
We read by fixating on words, accessing the lexicon / orthography / phonology to understand the word and then move on - models of access, word frequency improve speed of reading
Reading and eye movements - Key papers
Liversedge and Findlay (2000) - Saccadic eye movements and cognition
Scanning the visual scene is an important selective process in visual perception
Eye movement / tracking can be vital to understanding reading and cognitive processes
Whitford and Titone (2014) - Perceptual span in second language
As exposure to second language increases, the ease of processing increases, but the ease of processing language 1 decreases, thus balancing in processing speed
Bilinguals with high v low current L2 exposure exhibited increased L2 reading fluency but decreased L1 fluency (longer durations and slower reading rates)
Also more affected by reduction in window size during L2 reading, indicative of a larger perceptual span for L2, but not as affected during L1 reading (smaller perceptual span)
Individual differences tend to modulate more general aspects of reading behaviour, including reading difficulty and allocations of visual attention into the parafovea during both language sentence-level reading
Rayner et al, 2004 - the effect of plausibility on eye movements in reading
When a word is anomalous, it has an immediate effect on eye movements, but the effect is not immediate
Noun phrase arguments of the verbs are such that the thematic assignment occurred at the critical target word, and the sentence was plausible, implausible or anomalous - whereas target word in anomalous condition was immediately noticed, this was delayed in the implausible condition
Fitzsimmons et al, 2020 - reading on the web - we lexically process words when they are linked when skim reading, and we fullly lexically process both linked and unlinked words when reading for comprehension
Only fully processed linked words when navigating a web page, not just reading one statically, regardless of the type of reading happening
Readers should engage in an efficient reading strategy where they attempt to minimise comprehension loss while maintaining a high reading speed - hyperlinks should be markers of important information to navigate the text effectively and efficiently
Task of reading on the web causes readers to lexically process words much differently to typical reading
Neuropsychology of speech comprehension
Wernicke's area - profound problems with comprehension of even relatively simple sentences
Aphasia - content words are problematic but overall syntactic structure of the sentence remains intact, suggesting relatively independent processes within speech production
Comprehension is what is impacted - neologisms, retrieval problems (anomia), and paraphasias (word substitutions) - difficulty understanding others language and do not appreciate errors in theirs
Broca's aphasia - generally produces comprehension deficits only sentences with more comple morphosyntactic structures
Other areas that impact language comprehension affected by left hemisphere strokes; anterior superior temporal gyrus, superior temporal sulcus, angular gyrus, middle temporal gyrus, mid-frontal cortex in Brodmann's area 46 & 47, middle temporal gyrus
Pure word deafness - deficit affecting the ability to recognise speech sounds, while comprehension of non-speech sounds remains intact; pure as other aphasias are absent (patient can read and speak still) - her words as though they are far away
Pure word meaning deafness - can phonetically repeat words but not understand them but can recognise a written word (Franklin, 1994)
Ellis and Young (1996) - three route model for processing spoken word that helps understanding of these disorders
First route - direct phonemes access from auditory analysis, allowing repeats of words we do not know
Second route - access from semantic system and understand the word
Third route - bypasses semantic access and connects lexicon to speech output directly
In pure word meaning deafness, patient is compromised along route 2
Pure word deafness is an issue in route 1
Neuropsychology of reading - acquired dyslexia - evidence for dual route cascade model - studies of people who have lost the ability to read due to brain injury
Pattern of difference between surface and phonological dyslexia is dissociation of function between indirect sound-based routes and direct lexical routes
Surface dyslexia - deficit in the reading of irregular or exception words, while reading of regular words is spared - use grapheme-phoneme route as others are compromised
-> Lexical route is needed for exception words, and so regularisation errors are made
Phonological dyslexia - inability to pronounce non-words or pseudowords but can read real words regardless of regularity
Visual errors of reading non-words or pseudowords but can read real words regardless of regularity
Normally only partial deficits, but they are nowhere near normal functioning level
Also supports two stages of the lexical route - retrieval of the lemma and access to phonology -
Temporary dyslexia - errors of orthographic stage prior to lexical access, disruption to lexical access and errors assembling phonology
Evidence for the third route from orthography to phonology via the lexicon but no the semantic system comes from a pattern of deficit known as non-semantic reading
Some people can read irregular words without understanding them - generally associated with dementia diagnosis
How is reading achieved in orthographically consistent languages:
Orthographically consistent - low grapheme-phoneme ratio
English speakers show strong activation in the left posteior inferior temporal area and in part of the interior frontal gyrus, particularly when reading non-words
Italian (consistent) has greater activation of an auditory area associated with phonological processing, planum temporale, while reading words and non-words, suggesting there are left hemisphere commonalities in reading both orthographically deep and transparent scripts, with specific brain regions being orthographically specific
Chinese - common network for reading that compares two script types in a single language group; reading is sound from letters in one, and the other uses little of the letter sound assembly
Alphabetic and non-alphabetic scripts activate a common brain network - also some specific differences, such as activation of the inferior parietal cortex during reading of pinyin and fusiform gyrus for non-alphabetic characters
Stroop bias - where naming of ink colour is slower when emotionally charged words are used rather than emotionally neutral words, suggesting valence impacts processing
