The importance of Multi-Rater Assessments in ADHD evaluation

The Conners 4th Edition is designed to measure symptoms and impairments associated with Attention-Deficit/Hyperactivity Disorder (ADHD), along with common co-occurring problems and disorders in youth ages 6 to 18 years. It is also designed to collect this information from multiple sources including one or more parents/guardians (using the Parent form), one or more teachers (using the Teacher form), and from the youth themselves (using the Self-Report form). These three rater forms were developed such that wherever possible, the scales and item sets are aligned with each other to facilitate comparison of information collected from all respondents.

Why is it important to collect data from multiple perspectives?

The primary reason for multiple raters is to enhance the quality of the evaluation and to generate a full and well-rounded assessment of a youth being assessed with the Conners 4. While ADHD symptomology may include difficulty paying attention, impulsiveness, and restlessness, these symptoms may vary depending on the environment and context the youth is in, as well as the perspective of the person observing or experiencing them. Comparing scores of the Conners 4 from different raters helps gain a better understanding of the intersection of the behaviors across context, environment, and perspectives.

For instance, a youth in a structured home or school environment may be able to manage their emotions and behaviors more effectively than if they were in a noisy or distracting home or school environment. Understanding this variability in symptom presentation as a function of environment can help provide insight into how a youth’s ADHD functions, how to manage their impairments caused by their symptoms, and how they may be able to leverage their skills based on their surroundings.

While symptoms and impairments of ADHD can be affected by a youth’s environment, the perspective of the raters themselves is also critical to the difference in scores. Each rater provides a unique viewpoint. A parent/guardian or teacher will perceive the youth’s symptoms and impairments differently based on the rater’s own personal experience with the youth.

For example, symptoms that may appear heightened to a teacher may appear as more typical to a parent/guardian, and vice versa. Alternatively, behaviors reported as concerning across all raters provide an indication of how pervasive they are in the youth’s day-to-day functioning. The degree of insight may also vary by rater; a teacher may be monitoring many students at once while a parent has been observing their singular child their entire life, but a parent also has limited reference points of observation while a teacher has other peers for comparison.

Moreover, information that is not available for one rater may be provided by another. This advantage is especially salient for self-report ratings. The youth’s ratings of their own thoughts, feelings, and behaviors can supplement the observer’s ratings because it provides an intrapersonal perspective into the youth’s own functioning. While parents/guardians and teachers can provide valuable information from an externally observable perspective, the youth self-report provides insight about their feelings and thoughts that may not be easily observable by others, which is just as valuable. The youth also has the unique opportunity to report on their own behaviors across multiple settings, whereas parents/guardians and teachers can only report from the setting they are able to observe the youth.

How do raters differ?

To investigate the differences between raters, scores were generated from a sample of 62 parents, teachers, and youth who provided ratings of the same youth (for details on the description of the sample and scoring methods, (see the Conners 4 Manual; Conners, 2022). Figure 1 below presents a comparison of each rater type’s average scale scores for the Conners 4 Content, Impairment & Functional Outcome, and DSM Symptom Scales. Table 1 presents the percentage of endorsements for the Self-Harm Critical Items by rater type.

Figure 1a: Average Score by Rater Type: Conners 4 Content Scales

Figure 1b: Average Score by Rater Type: Conners 4 Impairment & Functional Outcome Scales*
*Note: Family Life is not assessed on the Conners 4 Teacher form

Figure 1c: Average Scores by Rater Type: Conners 4 DSM Symptom Scales

For all raters, the average scores across the Conners 4 scales varied in their congruence. Scales with higher congruence highlight the consistency of being able to observe these behaviors across multiple settings. For example, the DSM Conduct Disorder Symptoms scale has a high level of congruence between raters, indicated that parents, teachers, and the youth are able to report similar levels of aggressive, destructive, and deceitful behaviors, as well as rule violations. Figure 1c shows that scores from parents, teachers, and youth were typically within 1 point of the average score across all three raters. This high level of agreement is likely due to how readily observable the behaviors are; they tend to be noticeable and outwardly expressed, such that parents, teachers, and youth will provide a similar report.

Scales with lower agreement (for example, the Schoolwork scale) highlight the differences in reports from each setting. Figure 1b shows that scores from parents, teachers, and youth were more spread out, where each individual rater could range approximately 3 points above or below the average score across all three raters. These reporting differences may reflect the various perspective provided by each rater. For instance, parents may report on the child’s behavior at home but do not have the same level of insight as teachers regarding the child’s behavior in the school setting. In contrast, the youth can report on their own behaviors from multiple settings, so ratings from a youth and a parent might be reasonably misaligned when it comes to Schoolwork. Furthermore, the youth has the unique perspective of being able to report on their internal feelings and cognitions, rather than just the externally expressed behaviors.

For the Self-Harm Critical Items, in particular, differences in amount of endorsement show not only that the youth is able to report on this sensitive content, but they are able to report more on these behaviors and experiences than other respondents. As seen in Table 1, for the Self-Harm Critical Items, there was greater endorsement on the Self-Report relative to Parent and Teacher raters. Most notably, the Critical Item about deliberate self-harm is endorsed more than twice as often in Self-Report ratings (19.4%) relative to Parent ratings (8.1%). These items reflect deeply sensitive and personal behaviors or attitudes, to the extent that a youth may not wish to disclose this information to a parent or teacher. This finding underscores the importance of self-reports because they can include information that may not be readily observable by others and can be better described by the youth themselves.

Table 1. Endorsement of Self-Harm Critical Items by Rater Type

Overall, the differences observed across the rater types provide support for the need for multiple informants on the Conners 4 ADHD assessment. As a multi-informant assessment, the Conners 4 provides insight into the contextual variations where ADHD symptoms and impairments may present themselves. This feature greatly aids in diagnosis and eligibility considerations.

How can I use multiple informants in my evaluation process?

The Conners 4 Manual suggests a few things to consider when using multiple raters to complete the assessment.

• Rater forms must be completed independently by each respondent.
• It’s important to ask respondents to complete the Conners 4 around the same time period. Longer periods of time elapsing between assessments could reflect changes in the youth’s functioning or potential development of symptoms, ultimately affecting comparability of scores.
• It’s recommended that teacher ratings of the student be solicited from more than one teacher to evaluate the degree of consistency of behavior in different classroom settings.

Please see the Conners 4 Manual for other important recommendations regarding the administration of the
Conners 4.

While a single Conners 4 assessment will provide you with useful information regarding the youth, multiple assessments will provide a superior evaluation, gaining more insight into a youth’s ADHD-related symptoms and features. Although there is some expected similarity in the reports across raters, perfect alignment is not anticipated, because having multiple sources is meant to provide unique information that can reveal subtle differences in behaviors. This approach leads to a fuller and more effective assessment of ADHD overall and a more thorough and tailored treatment or intervention plan, as compared to an evaluation process that relies only on a single source of information.