Using Professional Judgment and Overrides in Risk/Needs Assessments

Due to its role in informing parole and probation decisions, the use of professional judgment or overrides in risk/needs assessment and management can become scrutinized after tragic and violent events.

What is a risk/needs assessment? Why do assessors override risk assessment scores? Is the use of this override common? What does the Level of Service/Case Management Inventory (LS/CMI; Andrews, Bonta, & Wormith, 2004) manual instruct us to do? We hope to answer these questions below.

A brief history of risk assessment

A risk assessment, which differentiates those who have offended according to their risk to re-offend, can help guide decisions regarding supervision and treatment. There have been four generations of risk assessment. The first generation was based on professional judgment by correctional staff and clinical professionals (Bonta & Andrews, 2007). The second generation consisted of evidence-based actuarial risk assessment instruments (ARAs) that focused mostly on static risk factors (e.g., criminal history and past substance abuse). These factors were assigned a quantitative score and summed, resulting in a final risk classification (Bonta & Andrews, 2007). Third-generation tools considered dynamic risk and need factors (e.g., employment, friends involved with criminal activity, family relationships) that are more sensitive to changes in someone’s circumstances. These third-generation ARAs became known as risk/needs assessments. Finally, fourth-generation ARAs integrated the assessment of a broader range of responsivity factors (i.e., personal characteristics) with systematic intervention and case management considerations (Bonta & Andrews, 2007).

The development of the third and fourth generations of ARAs would not have been possible without the Risk-Need-Responsivity (RNR) model for the assessment and rehabilitation of justice-involved individuals.

The Risk-Needs-Responsivity model and risk assessment

“Good offender assessment is more than making a decision on level of risk.” (Bonta & Andrews, 2007, p. 7)

The RNR model is an influential model for the assessment and treatment of those who have offended based on three principles. Firstly, the risk principle emphasizes that the level of service should match the level of risk. For example, intensive correctional programs are more effective when delivered to high-risk individuals (Viglione, 2019). Secondly, the needs principle underlines the importance of assessing dynamic criminogenic needs that can be targeted in programming. The needs principle, therefore, helps determine what interventions and services are the most suitable and should be prioritized based on the highest identified need areas (Cobb, Mowatt, & Mullins, 2013). Lastly, the responsivity principle focuses on how to maximize the ability of the individual to benefit from programming. In other words, the responsivity principle provides direction on how to choose services and interventions that increase the likelihood that those who have offended will have positive outcomes (Cobb, Mowatt, & Mullins, 2013). The theory behind the responsivity principle is that matching programming and intervention to an individual’s unique characteristics (e.g., gender, age, ethnicity, learning style, cognitive ability, mental health, and culture) increases the chance of success. By considering criminogenic needs and responsivity factors, newer risk/needs assessment tools acknowledge that change is an important aspect of life that cannot be ignored and can be facilitated by appropriate intervention and treatment.

The use of overrides and professional judgment in Actuarial Risk Assessment Instruments

The move away from first-generation unstructured risk assessment based on professional judgment towards ARAs has resulted in discussions on the use of overrides. ARAs are based on a variety of weighted factors and use statistical models to calculate a total risk score, typically in a range from low to high, which indicate the probability of reoffending (Kamorowski, Schreuder, Ruiter, Jelicic & Ask, 2018; Miller & Maloney, 2013). However, some clinical professionals and correctional staff have contended that the ARA process diminishes their expert voice and advocate for the use of overrides—using professional judgment to override a risk assessment score—to account for factors that the ARA process does not consider (Hannah-Moffat, Maurutto, & Turnbull, 2009; Miller & Maloney, 2013).

What does the research say about the use of professional judgment and overrides?

Numerous studies have summarized the superiority of ARAs, when used correctly, compared to unstructured professional judgment (see Grove et al., 2000; Guay & Parent, 2017; McCafferty, 2016; Miller & Maloney, 2013; Schmidt, Sinclair, & Thomasdottir, 2016). Additionally, research on the effect of overrides on predictive validity has established that the inclusion of clinical discretion hinders the overall validity of an ARA (Hannah-Moffat, Maurutto, & Turnbull, 2009; McCafferty, 2016; Orton, Hogan, & Wormith, 2020).

Perhaps the most well researched aspect of overrides is the use of overrides to increase risk levels. A plethora of studies have largely confirmed that most overrides are used to raise, rather than lower, risk levels (Cohen et al., 2020; Gobeil et al., 2014; McCafferty, 2016; Wormith, Hogg, & Guzzo, 2012). Some studies have even reported override rates as high as 74% for sexual offenders and 41% for non-sexual offenders (Schmidt, Sinclair, & Thomasdottir, 2016). The use of upward overrides can represent a form of “double dipping.” Double dipping occurs when an offender is, for example, identified as having a low score on a Mental Health section, but an assessor overrides the score to High or Very High risk because they feel that this section was not correctly weighted for this client. The assessor feels that the individual’s problems are worse than the score indicated for this client, and thus count one section twice. Practitioners interviewed by other researchers stipulate that selecting a checkbox for mental health “inhibits [their] ability to adequately assess, detail, and highlight the mental health problems and challenges supervisees face when starting supervision” (Cohen et al., 2020, p. 17).

It seems, then, that there is tension between the intention of ARAs and the dominant risk paradigm, and this tension can have a negative influence on how ARAs are used to inform programming (Viglione & Taxman, 2018). As a result, ARA developers must perform a balancing act when considering ways to develop ARAs that allow for more “clinically based and practitioner-driven assessment” (Hannah-Moffat, Maurutto, & Turnbull, 2009, p. 406). Risk scores should not be overridden to avoid addressing needs or responsivity considerations.

In third- and fourth-generation risk/needs assessments, documenting a full needs analysis should help identify the areas where additional concerns are present. Treating the problem rather than raising the risk score should be the fundamental approach. Double dipping can be addressed in training for risk/needs assessments, in order to avoid issues that decrease the predictive validity of these ARAs.

Professional discretion in the LS/CMI

The Level of Service/Case Management Inventory (LS/CMI) is one of the most widely used risk/needs assessment tools in the United States and Canada (Davidson et al., 2015). Combining risk/needs assessment and case management, this fourth-generation tool is designed to assist in management and treatment planning for justice-involved adults and late adolescents (Andrews, Bonta, & Wormith, 2004).

Section 1 of the LS/CMI, the General Risk/Need section, is considered the most critical for the overall assessment of risk, while the other sections provide important information for case management considerations and can be used for override decision-making. Indeed, the LS/CMI does allow assessors to use professional discretion to override the actuarial risk level that is generated from the Section 1 Total Score.

It may come as no surprise that assessors are cautioned about the use of this override. It has indeed been well-established that actuarial assessments are superior in terms of predictive accuracy compared to more subjective, clinically based assessments (Grove et al., 2000). The reason why the use of an override cannot be outright rejected is that, in a small number of scientific studies, the clinical approach has been found to be superior to the actuarial approach (Grove et al., 2000). Allowing room for some subjective judgment or professional discretion through the override is therefore a compromise between the actuarial and the clinical approach to risk (Frechette & Lussier, 2021).

How often and why is the LS/CMI override used?

The LS/CMI manual indicates that overrides should be used in less than 10% of all cases (Andrew, Bonta, & Wormith, 2004). When making an override decision, assessors should use information from Sections 2 (Specific Risk/Need Factors), 3 (Prison Experience – Institutional Factors), 4 (Other Client Issues) and 5 (Specific Responsivity Considerations), and both aggravating and mitigating circumstances can be used to justify an adjustment to the final risk level. An override could, for example, be used after consideration of gender-specific responsivity factors.

Most research shows that the LS/CMI override is used sparingly. One area that has, however, been under scrutiny is the use of overrides for sexual offender risk assessments. Wormith, Hogg, and Guzzo (2015) found that the LS/CMI override was used more frequently for sex offenders (35%) compared to other types of offenders (15%) and this use of the override decreased predictive validity for general, violent, and sexual re-offending. Because the override led to a decrease in the tool’s predictive validity, the authors recommend cautious use, a written justification, and more specific guidelines for its continued use. Studies in Ontario, Canada, have found that the override is used more frequently for individuals serving a sentence in the community compared to those serving a sentence in a correctional facility (Andrew, Bonta, & Wormith, 2004; Brews, 2009).

Why is the LS/CMI override used more often in specific circumstances?

When trying to understand why a professional might use the override, it is important to consider the difference between adjusting a risk prediction generated by a risk assessment instrument and developing a risk management intervention strategy (Frechette & Lussier, 2021; Hanson, 2009). Most would argue that the latter is more thorough and all-encompassing; it can include the scoring of a risk assessment instrument as part of a more extensive process (Serin, 2020).

In other words, risk prediction refers to the quantification and qualification of risk probabilities, whereas risk management refers to criminal justice initiatives, safeguards, and interventions to prevent recidivism (Frechette & Lussier, 2021). If the two tasks are instead considered one and the same, it is possible that assessors who are expected to score a risk assessment instrument use the override to be able to apply their preferred risk management strategy (Miller & Maloney, 2013).

Frechette and Lussier (2021) found evidence for this theory in their study of 15,744 individuals sentenced to community terms (at least 6 months) or incarceration (less than 2 years) in Quebec, Canada, between 2008 and 2011. All individuals had been assessed using the LS/CMI at the beginning of their sentence, and the override had been used for 650 (4.1%) of those assessed. Most of these overrides resulted in a risk increase (76.5%), whereas fewer than a quarter of the overrides (23.5%) resulted in a decrease in the risk category. One group of justice-involved individuals was found to be 10 times more likely to receive an override that increased their risk category: low-risk individuals serving a sentence for a violent crime or sex crime who had displayed a negative attitude (e.g., being intimidating or controlling). It is possible that the assessors used the override to increase community-based supervision beyond what the actuarial risk assessment suggested because this combination of offender characteristics was considered a red flag when it comes to expected compliance with community-based supervision and treatment.

The use of the override then seems to be based on a perceived gap between risk prediction and risk management, particularly for offenders in the community who are seen as a threat to society. This perceived gap indicates that assessors are not immune to a phenomenon called “dread factors,” which describes a tendency to put an emphasis on more catastrophic but rare events rather than more common events characterized by higher occurrence probabilities (Frechette & Lussier, 2021). If a risk assessment focuses on recidivism in general, the assessor may feel that the risk of reoffending is more specific or severe in nature than what is suggested by the generic risk assessment instrument and therefore prefers to focus on risk management-oriented outcomes instead (Frechette & Lussier, 2021).

Even though the previously discussed RNR model intends to provide a framework to establish appropriate treatment opportunities for those who have offended, it seems that assessors focus predominantly on risk.

In one study, which involved 1,085 hours of observation and interviews, Viglione (2019) found that probation officers frequently deviated from the way in which they were trained to use risk/needs assessments and the most common deviation indeed involved overriding results to increase the risk level. Additionally, even though probation officers were identifying criminogenic needs and responsivity considerations, they did not use those to inform treatment and supervision decisions (Viglione, 2019).

The need for further research

The override function can be seen as a safety net that allows assessors to change an individual’s risk category and, by extension, change the intervention to fit this risk (Frechette & Lussier, 2021). What we have discussed so far, however, does not help us understand why assessors would use an override to reduce a risk score. In other words, we might be able to explain why assessors may use the override to increase a risk score to justify more restrictive measures, but not why assessors would make the decision to use an override to decrease an offender’s risk category. Additionally, we know little about if and how the use of overrides introduces bias. In a study of justice-involved youth in Québec, Canada, who had been assessed with the Youth Level of Service/Case Management Inventory (YLS/CMI ™), it was found that Black youth were more likely to see their initial risk level increased through an override than White youth (Parent et al., 2022). This concerning finding warrants further attention.

There is a need for further research on the use of overrides, and this research should consider decision-making strategies, the potential introduction of bias, as well as sources of error (Frechette & Lussier, 2021). Additionally, we need to improve our understanding of the relationship between perceptions of professional and personal liability, risk, and the use of practices aligned with the RNR model (Viglione, 2019).

In the meantime, the evidence suggests that overrides cause imprecision and that assessors should include a defensible rationale when choosing to dismiss empirical predictions (Cohen et al., 2016). Indeed, as per the LS/CMI manual, overrides should be used sparingly, should be supported by logical arguments and reasonable evidence, and should be recorded alongside a rationale. Additionally, a risk prediction generated by a risk assessment instrument should be considered a piece of a larger puzzle, a source of information that can help inform the development of a risk management intervention strategy based on an extensive and comprehensive evaluation of all the information available. The LS/CMI is not intended to be used in isolation to assess the level of service required for an individual. Instead, the LS/CMI is designed to assist in implementing the least restrictive and onerous interpretation of a criminal sanction and help identify dynamic areas of risk and needs that can be addressed by programming to reduce risk.

Have more questions about this topic or about the Level of Service/Case Management Inventory (LS/CMI)? Reach out to us at [email protected] and our team will be happy to answer any questions you may have.

References