The coalition's academy programme is not the same as Labour's, so why is our old research being used to support them, asks Stephen Machin
University researchers are under increasing pressure to make their findings policy-relevant. At the same time, research evidence is increasingly quoted either to support or subvert government policies across a wide range of areas and this has featured heavily in policy debates about education. On the whole, this is a good thing, especially for evaluating the success or otherwise of initiatives like Sure Start, the education maintenance allowance and the literacy hour.
One education policy that has been discussed extensively is the introduction of academy schools. This is an initiative on which James Vernoit and I published some research in late 2010, not long after the coalition government came to power and the new education secretary, Michael Gove, started pushing for a rapid expansion of the academies programme.
Our results showed a reasonably positive impact of the policy – under the last government. So we have been somewhat surprised to see it used extensively by supporters of the coalition's policy on academies: for example, by the Department for Education in a recent debate with the Local Schools Network, and again by Jonathan Hill, the under-secretary of state for schools, last month. This seems rather hard to justify, given that the new academy programme is different in a number of ways.
The first academies were introduced under the Labour government in 2002. The model was very clear: take the worst-performing school in a local education authority and grant it academy status in the hope of improving academic standards.
By the time the coalition came to power, 203 academies had opened or were in the process of conversion. Since then, the programme has been massively expanded as a result of the 2010 Academies Act. As of last month, there were 1,635 academies and coverage has been widened to enable primary schools (and other types of schools) to become academies.
In research carried out with my colleague, James Vernoit at the centre for economic performance at LSE, we conducted an evaluation of Labour's academies programme. This was not as straightforward as it sounds; the schools that had converted to academies under Labour were all previously poorly performing schools, so we needed a comparison group.
It turned out that schools that had been granted academy status, but not yet converted (what we called "future academies"), were very similar to the already converted academies, so we could use them as a well matched "control group" to evaluate the impact of conversion.
Our findings painted a reasonably optimistic picture of Labour academies. Over an eight-year period, we found improvements in the quality of the intake and in GCSE performance in the academies that had converted in the first five years of the programme relative to the comparison group. We saw the most significant improvements in schools that made the biggest shift in terms of autonomy, for example, those that changed from a community school into an academy.
So what of the new academies set up under the coalition? The first point to make is that they are different from the Labour academies in that they are not necessarily about combating disadvantage. And unlike the poorly performing schools in deprived areas that converted to academies prior to the 2010 act, these new academies tend to be better-than-average schools. In fact, in the early days of the coalition government, schools rated outstanding by Ofsted actually had their applications "fast tracked" by the Department for Education.
And under the new programme, it is not only secondary schools that can convert – primaries can, too. So drawing comparisons with the Labour academies is simply not comparing like with like.
Nevertheless, our research has been widely cited in policy circles and in the media, usually as evidence for the success of the coalition's academies programme. Sometimes it is said that our research refers only to the Labour academies; more often this goes unstated. And unfortunately, our evidence on Labour academies has frequently been marshalled in support of the new academies programme, usually (though not always) without offering the caveat that the new academies are rather different.
We do not yet have robust, academically rigorous evidence on the coalition academies. For one thing, it is very early days, and as research on US charter schools also shows, time needs to pass before it is possible to evaluate their impact in a meaningful way.
It may be, in due course, that these new academies do deliver performance improvements. But we know nothing of this yet, and translating the evidence from the old programme over to the new, without appropriate reservations about whether the findings can be generalised, is, at the moment, a step too far.
It is good to see rigorous evidence being called for and used in a thoughtful way, but what is not so impressive is seeing research generalised beyond its research scope in this way. And one thing is clear: researchers, policy-makers and the media need to be clearer and think carefully about how they make practical use of research evidence.
• Stephen Machin is professor of economics, University College London and research director at the centre for economic performance, London School of Economics