Test Review: Clinical Assessment of Pragmatics (CAPs)

Today due to popular demand I am reviewing the Clinical Assessment of Pragmatics (CAPs) for children and young adults ages 7 – 18, developed by the Lavi Institute and sold by WPS Publishing. Readers of this blog are familiar with the fact that I specialize in working with children diagnosed with psychiatric impairments and behavioral and emotional difficulties. They are also aware that I am constantly on the lookout for good quality social communication assessments due to a notorious dearth of good quality instruments in this area of language.

I must admit when I first learned about the existence of CAPs in May 2018, I was definitely interested but quite cautious. Many standardized tests assessing pragmatics and social language contain notable psychometric limitations due to the inclusion of children with social and pragmatic difficulties into the normative sample. This, in turn, tends to overinflate test scores and produce false negatives (a belief that the child does not possess a social communication impairment due to receiving average scores on the test).  Furthermore, tests of pragmatics such as Test of Pragmatic Language -2 (TOPL-2) tend to primarily assess the child’s knowledge of rules of politeness and knowing the right thing to say under a particular set of circumstances and as such are of limited value when it comes to gauging the child’s ability to truly assume perspectives and adequately showcase social cognitive abilities.

The CAPs is a unique test as compared to others with a similar purpose, due to the fact that the testing administration (which can take between 45-60 mins) is conducted exclusively via videos. The CAPs consists of 6 subtests and 3 indices.

Subtests (You can read up more on the comparison of the CAPs subtests HERE ):

Instrumental Performance Appraisal (IPA) subtest (Awareness of Basic Social Routines) is a relatively straightforward subtest which examines the student’s ability to be polite in basic social contexts. The student is asked to first identify “if anything went wrong in the presented scenario?” After that, the student is asked to explain, what went wrong and how s/he knows? Targeted structures include greeting and closure, making requests, responding to gratitude, requesting help, answering phone calls, asking for directions, asking permission, etc.  Goals: can the student discern between appropriate and inappropriate language and then provide a verbal rationale in a coherent and cohesive manner.

Score types: (2) correct identification of problem or lack of thereof + correct justification; (1) correct identification but incorrect rationale; (0) incorrect identification.

Social Context Appraisal (SCA) subtest (Reading Context Cues) requires the student to engage in effective perspective taking (assume mutual vs. individual perspectives) by identifying sarcasm, irony, and figurative language in the presented video scenarios. The student is then asked to provide a coherent and cohesive verbal explanation and effectively justify own response.

Score types: (3) correct identification of the problem or lack of thereof + identification of idiom or sarcasm + reference to both characters actions; (2) correct identification of the problem or lack of thereof + identification of idiom or sarcasm + reference to one character’s actions; (1) correct identification of the problem or lack of thereof but an inability to verbalize the problem in the situation; (0) for incorrect identification.

Paralinguistic Decoding (PD) subtest (Reading Nonverbal Cues) assesses the students’ ability to notice and interpret micro-expressions and nonverbal language.  The aim of this subtest is to have the students grasp what went wrong vs. well in the presented videos, assume mutual perspectives,  as well as verbally justify their responses providing adequate and relevant details.

Score types: (3) correct identification of the problem or lack of thereof + explanation of situation + reference to both characters facial expressions and tone of voices; (2) correct identification of the problem or lack of thereof + explanation of situation  + reference to one character’s facial expression and tone of voice  (1) correct identification of the problem or lack of thereof but an inability to explain actions and/or nonverbal body language; (0) for incorrect identification.

Instrumental Performance (IP) subtest (Use of Social Routine Language) assesses the student’s ability to use rules of politeness (e.g., make requests, respond to gratitude, answer phone calls, etc.) by providing adequately supportive responses using first-person perspectives relevant to various social situations.

Score types: (2) appropriate introduction + use of supportive statements; (1) appropriate introduction without the use of supportive statements; (0) inappropriate intent of message or use of impolite language 

Affective Expression (AE) subtest (Expressing Emotions) assesses the student’s ability to effectively display empathy, gratitude, praise, apology, etc., towards affected peers in the video scenario. It requires the usage of relevant facial expressions, tone of voice, as well as stating appropriately supportive comments.

Score types: (2) expresses empathy, praise, apology, gratitude, etc. along with supportive statements +appropriate facial and prosodic affect; (1) expresses empathy, praise, apology, gratitude, etc. + appropriate facial and prosodic affect without relevant supportive statements;   (0) provides an approrpiate response but lacks adequate prosody and affect, or message contains inappropriate intent 

Paralinguistic Signals (PS) subtest (Using Nonverbal Cues)assesses the student’s ability to appropriately use facial expressions, gestures, and prosody (act out vs. recognize and interpret facial expression and gestures). This includes showing appropriate expression of empathy, frustration, alarm, excitement, gratitude, etc., exhibiting relevant inflection in prosody as well as showing appropriate to the situation facial expression (vs. having inappropriate message intent, be monotone, have flat affect, etc.)

Score types: (2) appropriately expresses urgency, empathy apology, etc. +exhibits inflections in prosody and shows relevant facial expressions; (1) appropriately expresses urgency, empathy apology, etc. +exhibits inflections in prosody without showing relevant facial expressions  (0)   inappropriate intent of message or monotone prosody. 

Indices (information regarding the student’s pragmatic proficiency):

1. Pragmatic Judgement (Sum of IPA, SCA & PD scaled scores)
2. Pragmatic Performance (Sum of IP, AE & PS scaled scores)
3. Paralinguistic (Sum of PD, AE & PS scaled scores)

Based on the administration of this test the following goals can be formulated for remediation purposes:

Long Term Goal: Student will improve pragmatic abilities for social and academic purposes

Short-Term Objectives: 

1. The student will verbally identify instances of politeness or impoliteness in presented social routines
2. The student will provide relevant justifications explaining which aspects of the presented scenarios were appropriate vs. inappropriate
3. The student will verbally identify sarcasm, irony, and figurative language in presented social scenarios
4. The student will effectively explain sarcasm, irony, and figurative language in presented social scenarios
5. The student will verbally interpret micro-expressions and nonverbal body language  (e.g., they feel disgusted; the girl is smirking, the man’s hands are crossed, etc.)
6. The student will effectively use rules of politeness and provide adequately supportive responses using first-person perspectives pertaining to various aspects of social scenarios
7. The student will display a range of emotional expressions via the use of relevant facial expressions, tone of voice when providing supporting responses
8. The student will state appropriately supportive comments regarding relevant social scenarios
9. The student will use a range of facial expressions, gestures, and relevant prosody pertinent to the provided social scenarios

Furthermore, this test comes with a Contextualized Assessment of Pragmatics Checklist as well as a downloadable  Free Report Template.

Multiple videos posted by the Lavi Institute showcasing individual subtest administration can be accessed by clicking on the above-highlighted links as well as on YouTube.

Psychometrics: the normative sample consisted of 914 individuals out of which 137 (or 15%)  included individuals with atypical language development: ASD: N-18; SLI: N-27; Other (Learning Disabilities): N-92.

Excellent Sensitivity and Specificity Cut Scores (at 1, 1.5 & 2 SD) for clients with ASD ONLY:

Impressions:  To date, I have used this assessment with only 3 students. As such, expect multiple updates of this post as I continue to document how well it suited to identify children with social communication difficulties. Below are my preliminary impressions on how well this test is suited for children with varying pragmatic profiles.

A. Initial Assessment: 8-3-year-old male diagnosed with Autism

The CAPs had captured the student’s display of pragmatic deficits extremely well.  It was able to highlight the student’s relative strengths as well as pervasive pragmatic needs.  Based on the results of the CAPs, I was able to generate relevant pragmatic goals to target with this student in therapy.

B.  Yearly Reassessment: 8-11-year-old diagnosed with Anxiety:

I definitely had some trepidation about how well the CAPs will be able to capture this student’s pragmatic difficulties. This student was initially assessed via the Social Language Development Test-Elementary (SLDTE), which did show deficits in the areas of making inferences, interpersonal negotiation, as well as multiple interpretations of social situations. However, subsequent to his assessment that student did exceptionally well in treatment and had improved exponentially. While I knew that the student was not done with the treatment quite yet, I wasn’t certain if the CAPs was capable of picking up his subtle social pragmatic difficulties. Much to my surprise, the CAPs was effective in highlighting my student’s difficulties on a number of subtests including those pertaining to the effective reading and use of context and nonverbal cues, comprehension  and interpretation of irony and sarcasm, effective support of peers via a variety of statements relevant to social situations (coherent and cohesive sentence formulation given relevant details), as well as use of relevant prosody, facial expressions, tone of voice, and nonverbal cues.

C. Initial Assessment: 11-year-old student with suspected language and literacy deficits 

This was definitely the trickiest assessment subject from my small sample. Based on the collected data I suspected the student had social communication deficits, however, given his relative strengths in a variety of areas and that the fact that no one had previously brought it up, I truly did not anticipate that CAPs will effectively and accurately identify his pragmatic needs. As expected, the student did quite well on that “easier” subtests of the CAPs: (IPA, IP, and AE). However, I was very pleasantly surprised that the CAPs had accurately picked up on the fact that the student presented with difficulty reading both context and nonverbal cues as well as using nonverbal cues to effectively answer the presented questions.

Summary: While my sample of subjects has been quite small to date, I fully intend to continue using the CAPs with students of varying ages with varying diagnoses in order to continue refining profile of students who will significantly benefit from CAPs administration for assessment and reassessment purposes.

MISC:

Current Cost on WPS Website $225

There you have it! These are my impressions of using the CAPs in my settings. How about you? Have you used this test with any of your students to date? If yes, what are some strengths and limitations you are noticing?