Assessing the Diagnostic Accuracy of a Dyslexia Screening Tool
The Challenge
Dyslexia is a learning difficulty estimated to affect up to 1 in 10 of the UK population. It can cause people difficulty with learning to read and write, comprehension of written instructions, carrying out sequences of directions, and planning and organisation. By getting the appropriate diagnosis, those with dyslexia can benefit from support with various management techniques. Children with dyslexia can make the most of educational interventions such as one-to-one lessons with a specialist teacher, and extra time on exams, allowing them to reach their full potential. Assistance is also available in the workplace where employers are required by law to make reasonable adjustments for those with dyslexia.
In order to receive a formal diagnosis, people with dyslexia must have an in-depth assessment carried out by an educational psychologist or qualified dyslexia specialist. These assessments can cost upwards of £500, which can be a barrier to those seeking a diagnosis and getting support with their learning difficulties.
Pico Educational Systems have developed the QuickScan Questionnaire and QuickScreen Dyslexia Test (QS Dyslexia Tests) in order to offer an accessible, online dyslexia screening tool. The QS
Dyslexia Tests, which are part of the British Dyslexia Association (BDA) Assured Scheme, provide an indication of dyslexia (as well as a detailed report of the test results and valuable insights into learning styles, abilities and difficulties) at a much lower cost than a full educational psychologist’s assessment. This can help individuals to identify whether they have positive dyslexia symptoms and obtain suggestions for training and development. Those who are identified as being at a higher risk of having the condition may then choose to undertake an in-depth assessment to formally confirm their diagnosis.
An essential step in the evaluation process of any diagnostic or screening test is to assess its accuracy. How useful a screening tool such as this is, depends in part on how much confidence we can have in the indication it provides.
The Solution
In order to help assess the accuracy of the QuickScreen Dyslexia Test, we carried out an analysis of observational data compiled by Pico Educational Systems to produce a number of diagnostic accuracy measures.
- Sensitivity (true positive rate),
- Specificity (true negative rate),
- Positive predictive value (PPV), and
- Negative predictive value (NPV).
The sensitivity of a diagnostic test indicates how good it is at identifying people with the condition in question. It tells us the probability that someone who is dyslexic is identified as such by the test. The specificity of a diagnostic test indicates how good it is at identifying people who do not have the condition, so, the probability that someone who is not dyslexic is identified as not dyslexic by the test.
The predictive values tell us the “post-test probabilities”. So, the positive predictive value (PPV) tells us the probability of a participant being dyslexic given a positive indication from the test. Likewise, if someone has a negative indication from the test, the negative predictive value (NPV) will tell them the probability that they do not have dyslexia, i.e., that the indication was correct.
In addition to the accuracy measures, we provided confidence intervals to capture the uncertainty in the estimated measures. Rather than taking a naïve approach to calculating the measures (i.e., taking observed proportions), we also used a transformation and applied a so-called continuity correction to provide more reliable estimates and confidence intervals.
The QuickScreen Test has five possible outcomes (None, Borderline, Mild, Moderate and Strong). Therefore, each of the four diagnostic measures were calculated for each possible indication, as well as combinations of these outcomes. For example, we estimated that 96.6% of non-dyslexic individuals will receive a QuickScreen indication of “None or Borderline”.
The Value
By calculating reliable diagnostic measures, we can describe how well the QuickScreen Test performs; assessing its accuracy in providing an indication that is likely to be consistent with an independent education psychologist’s assessment. Furthermore, the predictive values provide important information for a particular participant, answering the question “How likely is it that I have or don’t have dyslexia given the indication that I have received?”
“We are really pleased with the report Select provided on their initial analysis of our QuickScreen dyslexia test results. It doesn’t get much better than that for a first round – beautifully presented and very clearly set out.”
Dr Dee Walker – Dyslexia Consultant, Pico Educational Systems Ltd