Web Analytics

Theoretical Rationale For Neurolearning SPC Dyslexia Screening Test App

The most commonly used definitions of dyslexia in the English-speaking world[i],[ii]agree in two key respects:

  • That dyslexia’s core diagnostic feature is difficulty developing fast and accurate reading and spelling skills (particularly at the level of sounding out and spelling words) which are unexpected in relation to an individual’s intelligence, age, and education.
  • That challenges with the phonological component of language (essentially the way the brain identifies and processes sounds in words) are a key component of the brain-based differences underlying dyslexic reading and spelling challenges.[iii] (See Shaywitz, S. Overcoming Dyslexia for a good review of the phonological aspects of dyslexia.)

The Rose Review, which is used in the UK, also notes that challenges with verbal memory and processing speed have also been shown to play a role in dyslexic learning challenges. (See ref. 2)

Both definitions were based on a comprehensive review of the research on dyslexia available at the time of its formulation, and each reflected the broad consensus of understanding current among dyslexia researchers at the time of its formulation.[iv]

As additional research has been accumulated, several important additional points about dyslexia have been recognized:

  • Dyslexia has increasingly been recognized as a syndrome that is multifactorial in origin, having many possible underlying cognitive contributors rather than just one (i.e., phonological impairments). These factors include not only verbal memory and processing speed (as per the Rose Review), but also word retrieval and naming speed[v], visual attention[vi], procedural learning and memory[vii],and overall language ability. (See ref. 4)
  • Consistent with the growing recognition of this multifactorial origin, it has been increasingly recognized that the overall likelihood that an individual will show dyslexic reading and spelling challenges rises with the number of risk factors that the individual possesses.[viii] As a result, dyslexia risk assessment requires a clear and well-defined way of integrating the results from a wide variety of assessments of different cognitive skills.
  • It has also become increasingly clear that students with high verbal ability may experience highly significant challenges with reading and spelling (especially with speed or fluency of reading, stamina for reading, ability to decode unfamiliar words, spelling and writing) that are clearly dyslexic in nature, despite scoring at or above population means on tests of phonological awareness, reading comprehension, and/or other traditional dyslexia assessments.[ix] (Verbal IQ, as measured on the WISC, has a correlation of approximately 80% with Vocabulary[x], as measured both on the WISC and our app, so our app uses Vocabulary as a stand-in for Verbal IQ.) For such students, proper identification depends the recognition of discrepancies between their own personal areas of strength and weakness, rather than on comparisons of their weaknesses with population norms.[xi] For example, a very bright third grade student who tests into a gifted program with a verbal IQ of 135 (99th percentile), and who tests at the 80th percentile on an untimed reading comprehension test (i.e., reading short stories or informational paragraphs) but who also tests at the 50th percentile on tests of phonological awareness and single word decoding and spelling, will very likely also show academically significant difficulties keeping up class work in reading and writing, and will show difficulties spelling, making “silly mistakes” in reading test items, etc., that are dyslexic in nature and require special intervention, even though by most traditional measures they don’t qualify for a diagnosis of dyslexia. Similarly, among the age 5-6 pre-reading population, individual discrepancies between verbal ability and word and sub-word level reading skills can already be seen, and these often provide the only tip-off that the child is at high risk of dyslexia-related reading and spelling challenges. No other assessment currently in use identifies these children.
  • In addition, our series of more than 150 consecutive dyslexic students examined in our own clinic has revealed patterns of relationships between scores on different cognitive measures that are highly distinctive and robust for dyslexic students, and highly predictive both of academic challenges and potentially successful interventions[xii]. We’ve appended a graph to this document [see figure below] showing how these students scored on WISC-IV IQ and WIAT-III achievement tests. This graph reveals the characteristic relationships between scores on the various subtests that dyslexic students show, and which are almost entirely preserved across the range of IQs. Note, for example, the consistent broad gap between relatively higher verbal and non-verbal comprehension scores on WISC IQ, and the relatively lower working memory and processing speed scores. Also note the across-the-board reductions in academic fluency skills in such areas as math calculation, reading rate, and essay production. These internal relationships are highly consistent at the individual as well as the population level, and form another signature for dyslexic processing that we have found to have value both in diagnosis, and in recommending interventions.
  • Finally, we have found that an individual student’s variations from these characteristic patterns have predictive value in deciding upon what form of interventions they’ll be most likely to benefit from. For example, in the areas of reading and spelling, there are many different kinds of interventions available, some of which use muscle memory, others speech-related memory, others training of auditory processing functions, others visualization or verbal memory (mnemonics), etc. Based upon a student’s unique combination of results in the kinds of subtests featured in our app, we can guide instructors toward the use of interventions that will be especially likely to work successfully with an individual student (and away from less successful methods). The specificity with which we can deliver these results also means that our highly individualized recommendations will fit quite in instructional settings using UDL concepts and methods. [These patterns will be used to make recommendations, but will mostly not be part of the risk assessment algorithm]

ABOUT THE SCREENING TEST APP

Our dyslexia screening test app provides users with 3 kinds of information:
  1. the likelihood that a particular test subject will experience dyslexic reading and spelling challenges, and that they would qualify for a diagnosis of dyslexia if evaluated formally by a qualified assessment professional
  2. information about performance on certain kinds of cognitive and academic tasks, with regard to both how they express dyslexia risk and may point the way to helpful interventions
  3. individualized recommendations regarding instruction, accommodations, remediation, and/or need for additional assessment

TEST SCORING AND DEVELOPMENT OF RISK ASSESSMENT ALGORITHM:

The app test segregates users into 5 different risk or likelihood tiers:
  • Very low
  • Low
  • Moderate
  • High
  • Very high
This segregation is made on the basis of nine subtests, which have been chosen to provide information on both four of the basic or “foundational” cognitive skills described above—phonological processing, working memory, naming speed, and visual attention—as well as several “achievement” (acquired) skills in reading (real and nonsense word and passage level). These subtests are used to generate data which includes both time of response and correct/incorrect/incomplete response results, and they consist of:
  • A vocabulary knowledge task (as rough proxy for verbal IQ)
  • A sound discrimination task
  • A visual attention task
  • A rapid naming task
  • A sequential memory task with combined auditory and visual components
  • A sound elision task
  • A word reading task
  • A nonsense word reading task
  • A reading comprehension task
The selection of these tasks was based initially on the clinical experience of, and literature review conducted by, the test developers, which supported the hypothesis that these nine tasks would be sufficient to provide an efficient but accurate tool for screening individuals for dyslexia risk. This hypothesis was validated with a pilot study of approximately 50 individuals whose dyslexia status was known from previous assessments, which found that these measures generated a suitable body of data from which to generate reliable dyslexia risk measures. To further validate the app, 814 individuals aged 7 to 70 were then given the app test, and the data generated was evaluated in the following fashion:
  • The data for each individual was examined by an expert professional who then assigned each a dyslexia risk score based on the professional’s determination of the likelihood of dyslexia risk in that individual. These scores ranged from 1 (very low risk) to 10 (very high risk).
  • Both the raw data and the professional assessment score for each of these individuals were then used as inputs into the Damon machine learning system developed by a professional psychometrician Dr Mark Moulton (Educational Data Services). [The total pool was divided in halves, and each half was in turn used as a training and testing group; then the groups were switched and the process repeated. The overall process is described in a paper on our website (https://neurolearning.com/about-our-dyslexia-screening-test-app/).] From this process a scoring algorithm was generated.
  • Using the expert professional results as true results (TP and TN), the statistical results of the screener were as follows:
    • For our results, High and Very High risk scores are considered as “positive” results; Low and Very Low scores are considered as “negative” results; and moderate (about 15% of sample) is considered as neither positive nor negative, but unable to reach a definitive determination.
    • For sensitivity,
      • if we consider only the 85% of “positive” and “negative” results and exclude the 15% of “moderate” results (narrow definition), sensitivity is .980.
      • If we include the moderates (ie, the whole population screened, broader definition) then sensitivity is .907
    • For specificity,
      • narrow definition gives us .956;
      • if we include cases the test considered + which the professional judged “moderate” we get .887
    • For PPV,
      • our narrow rate is .964, and
      • the broader gives .905
    • For NPV,
      • the “narrow” definition gives us .976;
      • the broader definition gives us .889
  • In subsequent tests with an additional 569 users, using the similar expert professional approach, and with a slight fix to a big we found in our system, the results were:
    • For sensitivity
      • Narrow = .978
      • Broad = .884
    • For specificity
      • Narrow = .981
      • Broad = .897
    • For PPV
      • Narrow = .993
      • = .962
    • For NPV
      • Narrow = .938
      • Broad = .724 (reflects a somewhat higher percentage of TNs in the moderate range after the big fix)
In addition to the Total Dyslexia Score (ie, overall dyslexia risk score), the app scoring outputs also include 6 subscale scores, which reflect the following:
  • Four “foundation” subscales reflect performance on some of the low-level processing tasks that contribute to dyslexia risk. These subtests are:
    • Subword Skills (phonological processing/phonemic awareness, formed by the combination of the auditory discrimination and elision tasks)
    • Working Memory (formed by the sequential memory task)
    • Naming Speed (formed by the rapid naming task)
    • Visual Attention (formed by the visual matching task
  • Two “achievement” subscales, reflecting performance on the learned skills of reading at the:
    • Word Level (formed by the combination of whole word and nonsense word reading tasks)
    • Passage Level (formed by the reading comprehension task)
The scores for these subscales are not generated by the total dyslexia scoring algorithm, nor are they used directly by that algorithm in generating the total score. Instead, they are generated by use of a scoring table that was created based on normal distributions of the scores on the relevant subtests of those individuals in the original testing groups who were found by the algorithm to be at low or very low risk of dyslexia. The mean score for this population is 5, and higher scores (1 subscale point usually equaling 0.5 SD of raw score distribution) correlate with higher dyslexia risk. While these scores are not used in calculating overall dyslexia risk, they are used in generating our reports. I’ve attached a file showing the form in which the raw data is printed out from the app (decimalized numbers in the sections labeled “.T” are times; non “.T” columns are scores, with 0 indicating incorrect response, 1 correct response, and blank no response.

[i] National Institute of Child Health (2002). Definition of dyslexia. Washington, DC: National Institute of Child Health and Human Development. The key part of the definition reads: ‘Dyslexia…. is characterized by difficulties with accurate and/or fluent word recognition, and by poor spelling and decoding abilities. These difficulties typically result from a deficit in the phonological component of language that is often unexpected in relation to other cognitive abilities and the provision of effective classroom instruction’.

[ii] Rose, J. (2009). Identifying and teaching children and young people with dyslexia and literacy difficulties. Available from: http://www.teachernet.gov.uk/wholeschool/sen/ [last accessed 5 July 2009]. This definition reads: ‘Dyslexia is a learning difficulty that primarily affects the skills involved in accurate and fluent word reading and spelling. Characteristic features of dyslexia are difficulties in phonological awareness, verbal memory and verbal processing speed’.

[iii] Shaywitz, S. (2003). Overcoming Dyslexia. New York: Simon and Shuster.

[iv] Snowling, M, & Rose, J. (2012). Annual Research Review: The nature and classification of reading disorders – a commentary on proposals for DSM-5. Journal of Child Psychology and Psychiatry 53:5. (2012), pp 593–607.

[v] Wolf, M, & Bowers, P. (1999). The double-deficit hypothesis for the developmental dyslexias.. Journal of Educational Psychology 91:3. pp 415–438.

[vi] Schneps MH, Thomson JM, Sonnert G, Pomplun M, Chen C, et al. (2013) Shorter Lines Facilitate Reading in Those Who Struggle. PLoS ONE 8: e71161 doi:10.1371/journal.pone.0071161.

[vii] Ullman, MT, Earle, FS, Walenski, M, Janacsek, K. (2020) The Neurocognition of developmental disorders. Annual Review of Psychology. 71:5. Pp 5.1-5.21.

[viii] Pennington, B.F. (2006). From single to multiple deficit models of developmental disorders. Cognition, 101, 385–413.

[ix] Eide, Bl, & Eide, F. (2006). The Mislabeled Child. New York; Hyperion. Eide, BL, & Eide, F. (2011) The Dyslexic Advantage. New York: Hudson Street Press. Eide, BL, & Eide, F., 2e Newsletter. October 2005. Hoeft, Fumiko, in preparation.

[x] WISC-IV. (2003). San Antonio: Pearson.

[xi] Turner, M. (1997). Psychological Assessment of Dyslexia. London: Whurr.

[xii] Eide, BL, & Eide, F. Unpublished observation.