On the Limitations of Using Vocabulary Tests with School-Aged Students

Those of you who read my blog on a semi-regular basis, know that I spend a considerable amount of time in both of my work settings (an outpatient school located in a psychiatric hospital as well as private practice), conducting language and literacy evaluations of preschool and school-aged children 3-18 years of age. During that process, I spend a significant amount of time reviewing outside speech and language evaluations. Interestingly, what I have been seeing is that no matter what the child’s age is (7 or 17), invariably some form of receptive and/or expressive vocabulary testing is always mentioned in their language report.

Many of you may be wondering, “What’s wrong with having a vocabulary test as part of an assessment battery? Isn’t vocabulary hugely correlated with both language and literacy outcomes?”  The answer is, “It is more complicated than that.” Here’s why.

Children with robust lexicons formulate longer sentences and more interesting stories, better comprehend complex texts, and even compensate to some degree for reading deficits (Colozzo et al, 2011; Law and Edwards, 2015; Rvachew and Grawburg, 2006).

In contrast, studies have found that children with Developmental Language Disorder (DLD) (formerly known as Specific Language Impairment or SLI) have limited expressive vocabularies (Leonard, 2014), have trouble learning new words (Alt & Spaulding, 2011; Storkel et al, 2016), and have clinically significant word retrieval deficits (Dockrell, Messer, George, & Wilson, 1998).

Due to these deficits, one-word vocabulary tests are often used in the assessment process to qualify children for speech and language services (Betz, Eickhoff, & Sullivan, 2013). However, studies have found that single word vocabulary tests have poor psychometric properties and/or are not representative of linguistic competence embedded in life-activities (Gray et al., 1999; Ukrainetz & Blomquist, 2002; Bogue, DeThorne, Schaefer, 2014).

Furthermore, because of this, single word vocabulary tests can overinflate testing scores and not represent the child’s true expressive language competence. Finally, even when a student truly has solid or even superior vocabulary knowledge and naming skills, doesn’t mean that s/he can effectively utilize these abilities during the narrative production as well as reading and writing tasks.

Don’t believe me?  Consider reviewing language evaluations of current or former students who received outstanding scores on one-word vocabulary tests, yet who were unable to utilize these words to perform semantic flexibility tasks (e.g., name antonyms, synonyms, provide clear definitions as well as define multiple meaning words), produce coherent and cohesive narratives, comprehend these words in the context of read texts, or utilize them during writing composition tasks.

The problem is that numerous SLPs overuse these tests and rely on them for qualification purposes when diagnosing language impairment (Betz, Eickhoff, & Sullivan, 2013). However, the practice of qualifying students based on single-word vocabulary testing in conjunction with psychometrically weak comprehensive testing (visit HERE for a compilation of psychometric data on major SLP testing), can often result in many language-impaired students not being qualified for language therapy services despite desperately needing them.

Now it’s important to understand that I am not recommending elimination of vocabulary tests from SLP assessment batteries.  I am merely suggesting that SLPs use these tests wisely during the assessment process, and utilize them with children who truly benefit from their administration. Such populations include toddlers and preschoolers (under 5 years of age) as well as any children presenting with severe language deficits regardless of age, secondary to intellectual and neuro/developmental impairments such as ASD, DS, FXS, FASD, etc.  They are especially relevant for children with limited vocabularies who are unable to effectively participate in semantic flexibility tasks or produce narratives. As such, we want to learn more about the types of words they know and use on a daily basis to express their wants/needs, so we can increase their lexicon for functional communication purposes and prepare them for effective engagement in both semantic flexibility as well as narrative tasks, in order to further improve their language abilities.

In contrast, for children age 5-6 and older, it is far more practical for SLPs to functionally determine their linguistic flexibility skills as pertaining to the use of language.  This can be accomplished via standardized as well as informal measures. As mentioned above, broadly speaking, linguistic flexibility tasks focus on the manipulation of language.  Tasks such as generation of attributes, production of synonyms and antonyms, formulation of clear and precise definitions of words as well as explanations of multiple meaning, figurative, and ambiguous words and sentences are all examples of language manipulation tasks.

As such, these tasks are far more representative of the student’s language ability in an academic setting versus selecting a picture out of a visual field of four items (receptive identification) or naming a word in the presented picture (expressive generation).

Now there are numerous tests which possess subtests relevant to this purpose.  I, personally, often use select subtests from the below tests:

• The WORD Tests (Elementary and Adolescent)
  • Associations
  • Antonyms
  • Synonyms
  • Definitions
  • Flexible Meanings
• Language Processing Test – 3 (LPT-3)
  • Similarities and Differences
  • Multiple Meaning Words
  • Attributes
• Expressive Language Test – 2 (ELT-2)
  • Metalinguistics
  • Defining Categories
• Test of Integrated Language and Literacy 
  • Vocabulary Awareness
• Clinical Evaluation of Language Fundamentals – 5 Metalinguistics (CELF-5M)
  • Multiple Meanings
  • Figurative Language

There are a number of other tests which contain subtests suitable for this purpose. SLPs can also easily create their own informal assessment procedures, similar to the above, for clinical assessment purposes.

However, even these tasks, though a huge improvement over one-word vocabulary tests are not sufficient. In addition to these, research strongly recommends the inclusion of narrative assessment (which is highly correlated with social, reading, as well as academic outcomes), as part of SLP assessment battery.

Narrative language skills have routinely been identified as one of the single best predictors of future academic success (Bishop & Edmundson, 1987; Feegans & Appelbaum 1986; Dickinson and McCabe, 2001). Language produced during story retelling is positively related to monolingual and bilingual reading achievement (Reese et al, 2010; Miller et al, 2006) Narratives provide insights into child’s verbal expression by tapping into multiple language features and organizational abilities simultaneously (Hoffman, 2009; Ukrainetz, 2006;Bliss & McCabe, 2012). They encompass a number of higher-level language and cognitive skills (Paul et al, 1996) such as event sequencing, text cohesiveness, use of precise vocabulary to convey ideas without visual support, comprehension of cause-effect relationships, etc. Narratives bridge the gap between oral and written language and are needed for solid reading and writing development (Snow et al, 1998).

Contrastingly, poor discourse and narrative abilities place children at risk for learning and literacy-related difficulties including reading problems (McCabe & Rosenthal-Rollins, 1994), while narrative weaknesses significantly correlate with social communication deficits (Norbury, Gemmell & Paul, 2014). As a result, narrative analyses help SLPs with distinguishing children with DLD from their typically developing (TD) peers (Allen et al 2012).

So the next time you are tasked with selecting appropriate language testing to determine whether a student presents with language and literacy deficits, don’t be so hasty in picking up that single-word vocabulary test.  Take a moment to carefully consider its utility for the student in question. After all, it may very well be a determining factor in deciding whether the student will qualify for language therapy services.

References: 

1. Allen, M,  Ukrainetz, T & Carswell, A (2012) The narrative language performance of three types of at-risk first-grade readers. Language, Speech, and Hearing Services in Schools, 43(2), 205-221.
2. Alt, M., & Spaulding, T. (2011). The effect of time on word learning: An examination of decay of the memory trace and vocal rehearsal in children with and without specific language impairment. Journal of Communication Disorders, 44(6), 640–654
3. Betz, Eickhoff, & Sullivan,( 2013) Factors Influencing the Selection of Standardized Tests for the Diagnosis of Specific Language Impairment. Language, Speech, and Hearing Services in Schools, 44, 133-146.
4. Bishop, D. V. M., & Edmundson, A. (1987). Language-impaired 4-year-olds: Distinguishing transient from persistent impairment. Journal of Speech and Hearing Disorders, 52, 156–173.
5. Bliss, L. & McCabe, A (2012, Oct) Personal Narratives: Assessment and Intervention. Perspectives on Language Learning and Education. 19:130-138.
6. Bogue, E. L., DeThorne, L. S., & Schaefer, B. A. (2014). A psychometric analysis of childhood vocabulary tests. Contemporary Issues in Communication Science and Disorders, 41, 55-69.
7. Colozzo, P., Gillam, R. B., Wood, M., Schnell, R. D., & Johnston, J. R. (2011). Content and form in the narratives of children with specific language impairment. Journal of Speech, Language, and Hearing Research, 54(6), 1609-1627.
8. Dickinson D. K., McCabe A. (2001). Bringing it all together: the multiple origins, skills and environmental supports of early literacy. Learning Disabilities Research and Practice. 16, 186–202.
9. Dockrell, J. E., Messer, D., George, R., & Wilson, G. (1998). Children with word-finding difficulties: Prevalence, presentation and naming problems. International Journal of Language & Communication Disorders, 33, 445–454.
10. Feegans, L.,& Appelbaum, M (1986). Validation of language subtypes in learning disabled children. Journal of Educational Psychology, 78, 358–364.
11. Gray, S., Plante, E., Vance, R., & Henrichsen, M. (1999). The diagnostic accuracy of four vocabulary tests administered to preschool-age children. Language, Speech, and Hearing Services in Schools, 30(2), 196–206.
12. Hoffman, L. M. (2009). Narrative language intervention intensity and dosage: Telling the whole story. Topics in Language Disorders, 29, 329–343.
13. Law, F., II, & Edwards, J.R. (2015). Effects of vocabulary size on online lexical processing by preschoolers. Language Learning and Development, 11, 331–355.
14. Leonard, L. B. (2014). Children with specific language impairment. Cambridge, MA: MIT Press.
15. McCabe, A., & Rollins, P. R. (1994). Assessment of preschool narrative skills. American Journal of Speech-Language Pathology, 3(1), 45–56
16. Miller, J et al (2006). Oral language and reading in bilingual children. Learning Disabilities Research and Practice, 21, 30–43
17. Norbury, C. F., Gemmell, T., & Paul, R. (2014). Pragmatics abilities in narrative production: a cross-disorder comparison. Journal of child language, 41(03), 485-510.
18. Paul R, Hernandez R, Taylor L, Johnson K. (1996) Narrative development in late talkers: early school age. Journal of Speech and Hearing Research, 39(6):1295–1303
19. Reese E., Suggate S., Long J., Schaughency E. (2010). Children’s oral narrative and reading skills in the first three years of reading instruction. Reading & Writing: An Interdisciplinary Journal, 23, 627–644.
20. Rvachew S., Grawburg M. (2006). Correlates of phonological awareness in preschoolers with speech sound disorders. Journal of Speech, Language, and Hearing Research, 49: 74–87.
21. Snow, C.E., Burns, M.S., & Griffin, P. (eds.) (1998). Preventing reading difficulties in young children. Washington, DC: National Academy Press
22. Ukrainetz, T. A. (2006). Teaching narrative structure: Coherence, cohesion, and captivation. In T. A. Ukrainetz (Ed.), Contextualized language intervention: Scaffolding PreK–12 literacy achievement (pp. 195–246). Austin, TX: Pro-Ed.
23. Ukrainetz, T. A., & Blomquist, C. (2002). The criterion validity of four vocabulary tests compared with a language
    sample. Child Language Teaching and Therapy, 18, 59–78.