After the recent review of Child, Youth and Family (CYF), the Office of the Children’s Commissioner (OCC) has come to the conclusion that the system is failing children on far too many fronts. In fact, the Children’s Commissioner goes as far as to doubt that children are better off in state care at all.

Of the many areas where CYF has failed to meet expectations is in the collection of data. One of the conclusions of the report is that: “Better collection and analysis of data is essential for CYF to improve its services and for the Government and the public to have confidence that CYF and other state agencies are improving outcomes for vulnerable children.”

There is currently little reliable data on the outcomes of children in the system, or even whether state intervention is preferable to keeping children with their families. It makes sense then that the OCC recommends using data to drive improved outcomes.

But will the Minister for Social Development allow the collection of such evidence?

Last month, Anne Tolley rejected a predictive modelling proposal with the words "not on my watch, these are children not lab rats". This won her accolades from both the left and right.

The proposal concerned a predictive tool that would have been used to identify children at risk of abuse. The Ministry of Social Development sought ethical approval to use the tool on newborns. The cohort of children would then be given a risk rating, and observed to see whether those deemed high-risk actually went on to suffer abuse.

Like the now internationally acclaimed social investment approach, this would allow the ministry to target its resources where it can be most effective. But Tolley was not having a bar of it. She even went as far as to accuse those heading the project of not differentiating between an academic study and real life.

Keep in mind, this is the same government trialling social bonds, a model which requires comparison with a control group to measure performance. This is a government actively seeking new approaches to fund and deliver social services.

The way the recommendation was reported in the media made it sound like Tolley may have a point. The ministry’s recommendation, taken out of context, seemed to suggest agencies should not intervene with the observed group, in order to see whether the predictive model was correct. Such an approach would be abhorrent – to remove all support for vulnerable children in order to test a model.

But that is not what the ministry proposed. The observational study would see children receive exactly the same treatment as they would if there had been no study. There would not be any addition of services, no withdrawal of services, and responsiveness to at-risk children was still mandatory. So as a purely background activity, all the study would do is collect the data necessary for evidence-based policy.
It seems to be that any new intervention, anything that involves the collection of data and evidence, or any programme that involves a control group, reduces people to lab rats or guinea pigs.

It is possible that Tolley simply misunderstood the ministry’s proposal. That does not distract from the fact her comments have been hugely damaging. When not even the Minister for Social Development can see the value in testing whether models work, the success of evidence-based reforms seem bleak.

Tim Dare, who was contracted by the ministry to provide an ethical analysis of predictive risk modelling tool, argued that “the problem is not the shelving of the study – though that was a mistake too – rather it is the chilling effect of the knee-jerk and political response to an attempt to produce evidence for important social policy.”

And that is the real concern. Tolley, and those who supported her rejection of the proposal, no doubt believe they are acting in the best interests of some of society’s most vulnerable children. Their commitment to respecting the dignity of children as people, rather than subjects of academic scrutiny is commendable.

But to believe the collection of data is ethically questionable will make any overhaul of the CYF system difficult. Of course, quantitative evidence will need to be supplemented with the qualitative experiences of those involved.

The OCC has made it clear that the CYF system needs to change. An independent panel has already begun work on a new operating model. It can only be hoped that Dare’s advice is heeded: “we must not stumble along relying on experience and anecdote: it leads us astray.”