This article was originally published in The Weekly Times, and was republished with permission.
A controversial new NDIS assessment tool has been slammed as mathematically flawed, unreliable, and potentially “dangerous” for Australians relying on disability funding.
The government will roll out an adapted version of the Instrument for the Classification and Assessment of Support Needs (I-CAN) tool in April next year.
The assessment involves a guided three-hour interview by a non-clinical assessor, who could ask questions like, “Can you go to the toilet by yourself?”.
The answers are fed into a computer and an algorithm determines how much funding that person needs. A further questionnaire will be undertaken to glean information about the context of someone’s life.
However, despite the government crowing that the adapted I-CAN is the “gold standard” for assessing people with a disability, a PhD student has investigated the published evidence and found it doesn’t stack up.
The registered psychologist, who does not want to be named, presented his analysis in an anonymous submission to a senate inquiry.
He told this masthead that “as a general rule, it is dangerous to use a tool for high-stakes decisions when that tool has not been shown to be reliable or valid for that particular use”.
He said the adapted version has not been shown to be scientifically reliable or valid, and that using it without very strong evidence risks giving some participants too much funding – and more worryingly, others too little.
“A lack of precision will cause risks of further fiscal waste and human suffering,” he said.
He said he would like to know whether the government’s algorithm for turning I-CAN scores into participant plans meets the definition of a software-based medical device under the Therapeutic Goods Act. The algorithm is still being designed.
Dr Aaron Frost, also a psychologist, posted his views on the academic’s analysis on YouTube, saying “someone saw what looked like a massive government cock-up,” and decided to investigate.
He said what was found, “should worry all of us; every clinician, every person with a disability, and every taxpayer.”
“The government has taken that qualitative clinical tool and plugged it into their secret black box and they’ve decided to let that undisclosed algorithm automatically spit out whether a person with quadriplegia gets $40,000 or $400,000 a year,” Dr Frost said.
Australian Psychological Society chief executive Dr Zena Burgess said the anonymous submission backs their view “that the evidence for the use of the tool is indeed weak”.
I-CAN was developed over 20 years ago to help qualified clinicians figure out what people actually needed to support their needs in group care. The government is adapting the tool, with the University of Melbourne and the Centre for Disability Studies.
Earlier this year, Health Minister Mark Butler announced 160,000 people would be removed from the NDIS – and about the same again denied entry – over the next four years as part of a plan to cut costs. As people’s plan assessments are due, the I-CAN tool will help determine who goes and who stays.
At the moment, eligibility and funding are based on reports from health professionals.
Dr Burgess said the government’s intention to reduce the use of medical reports was risking “undermining professional advice in the NDIS overall”.
A spokesman for the agency said the new NDIS Support Needs Assessment is built around a conversation between people, not a single stand-alone tool, and that further testing and research would be undertaken.