Part 2, Strategy 2
Action Item 2: Establish quality-assurance practices for the use of risk and needs assessments.
Why it matters
Simply having a risk and needs assessment tool is not enough to ensure that it is used correctly. Risk and needs assessment instruments are not merely computerized reports; they are administered by people. Quality assurance practices are essential to ensuring that risk assessments are used properly.
Risk and needs assessment questions are devised and weighted based on the importance of each item in the prediction of recidivism for the population. Supervised and prison populations can change over time, so risk and needs assessment tools must be validated routinely to ensure their accuracy. Validation studies need to determine the tool’s predictive accuracy by race and gender, as well as for the correctional population that is assessed.
Scoring guidelines provide instructions for how each question on a risk assessment should be scored based on the information collected. The scoring guidelines of proprietary and public domain risk assessments are regularly reviewed and updated by their creators to reflect the latest research and practices in the field. Corrections agencies should check with the assessment tool developers every two to three years to learn of any changes to scoring guidance.
Often, there is a difference between the policies and procedures that guide use of risk and needs assessment and the way the tool is actually being used. Staff need initial and ongoing training on how to use a risk and needs assessment tool properly. Training ensures that people who administer the assessment understand risk-need-responsivity principles, develop a common understanding of each question in the tool, and consistently adhere to the scoring guidelines. Ongoing training helps reduce the likelihood of staff administering the tool differently from one another over time, which distorts assessment results. A quality assurance process can determine whether the tool is being used for its intended purpose, including to inform case planning and direct people to programming.
What it looks like
- Require in statute the routine validation of risk and needs assessment tools, including checking for bias across race and gender.
- Ensure that supervision staff are trained on the proper use of risk and needs assessments.
- Ensure that corrections agencies regularly check for scoring guidance updates from the risk and needs assessment tool developer.
- See Case Study: Hawaii updates scoring and staff training
- Ensure that agencies regularly review how risk and needs assessments are conducted.
- See Case Study: Pennsylvania uses RAQI to improve use of risk and needs assessment
- See Case Study: Rhode Island uses quality assurance practices to improve use of risk and needs assessment
Key questions to guide action
- What are the requirements in your state to ensure that risk and needs assessments are validated regularly and checked for bias?
- What can your state do to ensure that people who conduct risk assessments are properly trained on their use?
- What quality assurance practices do correctional agencies use to ensure that risk and needs assessments are properly conducted?
Use the information that follows to inform your answers to these questions.
Only about half of states that use risk assessments for probation populations report validating the tools on their populations within the last five years.
Only about half of states that use risk assessments for parole populations report validating the tools on their populations within the last five years.
Additional Resources
Improving risk assessments
Validation studies should examine the risk and needs assessment instrument’s ability to identify groups of people with different probabilities of reoffending and should demonstrate similar degrees of accuracy across race and gender. A properly validated tool may result in different average scores by race or gender, which may reflect true disparities in criminal justice involvement or enforcement in a jurisdiction. To be valid, risk and needs assessments must predict reoffending (e.g., new arrests and/or convictions), and the degree of accuracy in making those predictions should be close to equal by race and gender groupings. To learn more, see: Three Things You Can Do to Prevent Bias in Risk Assessment.
An objective review of policies, procedures, and practices can help agencies identify areas for improvement. The CSG Justice Center developed a multi-point inspection protocol to help state and local corrections agencies, community supervision agencies, and treatment/programming providers assess how well their risk and needs assessment tool is working. The Risk Assessment Quality Improvement (RAQI) checklist helps identify potential problems with assessment tools quickly and efficiently. State agencies can then use this information to ensure that taxpayer dollars are targeted correctly and used effectively on supervision, programming, and treatment. To learn more, see RAQI: Risk Assessment Quality Improvement Checklist.
Risk-Need-Responsivity (RNR) Principles The RNR principles have three major components:
- Risk Principle: Match the intensity of services to a person’s level of risk for criminal activity
- Need Principle: Target criminogenic needs (factors that contribute to the likelihood of new criminal activity)
- Responsivity Principle: Account for a person’s abilities and learning styles when determining appropriate services and tailoring them to his or her needs.
Case Study
Hawaii updates scoring and staff training
Hawaii has a long history of using risk and needs assessments to inform case planning and supervision; however, in 2012, CSG Justice Center staff discovered that the state’s scoring guide had not been updated in years. They also found inconsistencies in the scoring, and staff were unable to determine predictive accuracy for distinct groups of people (i.e., people on probation and parole, and those released from prison without supervision). As a result, CSG Justice Center staff worked closely with the state to improve staff skills and enhance the risk assessment’s accuracy.
Hawaii worked with the creators of their risk assessment tool to adopt updated scoring rules that reflected state-specific circumstances and to issue a revised scoring guide. Approximately 200 staff responsible for conducting assessments were retrained statewide and successfully passed a state-based recertification process that required them to demonstrate proficiency on an audited assessment. To learn more about how other states are improving the quality of risk assessment, see How Justice Reinvestment States are Improving the Quality of Risk Assessment.
Case Study
Pennsylvania uses RAQI to improve use of risk and needs assessment
An objective review of policies, procedures, and practices can help agencies identify areas for improvement. The CSG Justice Center developed a multi-point inspection protocol called the Risk Assessment Quality Improvement (RAQI) checklist, which can be used to assess and improve the quality of an agency’s use of risk and needs assessments. Pennsylvania used the RAQI to identify deficiencies in its risk assessment scoring guide and inaccuracies in scoring. As a result, the Pennsylvania Board of Probation and Parole (PBPP) worked with the creators of the risk assessment to update their scoring guide. PBPP instituted a series of monthly trainings, held at each regional office, that focused on providing clarification and updates to the scoring guide.
Case Study
Rhode Island uses quality assurance practices to improve the use of risk and needs assessment
Agencies need to create protocols to adequately train new assessors and then to regularly review, via direct observation or recording, how assessors conduct the assessment and calculate the score. Assessments should also be checked for inter-rater reliability (i.e., that two different staff members score the same individual the same way on the risk instrument.)
Inter-rater reliability exercises highlight scoring differences among assessors, and focus groups with assessors may provide additional information about why practices no longer align with written policies and procedures. For example, in Rhode Island all staff who conduct risk and needs assessments are required to submit monthly taped interviews to a trainer who is responsible for reviewing the tapes to ensure that appropriate interview techniques were used and that scoring was done accurately. The trainer provides written feedback on scoring and interviewing skills for each staff member responsible for conducting assessments and addresses deficiencies through additional on-site coaching. Staff are also required to participate in a refresher training course every two years.