A slice of frozen PISA

Climb into your bath, track your arteries with a razor blade and bring on The Smiths.

It's PISA time, and there's nothing the Welsh enjoy more than a good moan, revelling in the doom as we all – once again - rubber neck the slow motion car crash that is the Welsh education system.

PISA, as you know, is an OECD initiative. Since 2000, it's tested 15 year olds in 60+ nations and territories every three years in reading, mathematics and science.

Although Wales has been included as part of regional statistics before, Wales has only sat the PISA tests separately since 2006, so today's results are the third set where Wales has featured specifically.

They're important because PISA is the closest we have to a direct international comparison in terms of education, as all pupils sit the same test at near enough the same time. It's not perfect, but it's the best measure we have and its results shouldn't be dismissed out of hand.

2012's Results

How Wales has performed in the last three PISA tests.
(Click to enlarge)

You can find all the data here, but the specific part relevant to Wales is here (pdf).

One crumb of comfort is that reading performances have improved slightly - up 4 points from 476 in 2009 to 480 in 2012. However, this still lags behind England (500), Scotland (506) and Northern Ireland (498). Northern Ireland was the only Home Nation to have seen a decline.

The bad news is that performances in maths and science in Wales have again declined – though not as sharply as 2009. Maths scores dropped by 4 points from 472 to 468, while science scores dropped by 5 points from 496 to 491. Again, the other Home Nations outperform Wales, with a ~20 point gap between Wales and the rest of the UK that shows no signs of closing.

For want of comparison, 2009 saw a 12 point drop in maths and 9 point drop in science. So the latest figures are hardly as "alarming" as The Western Mail screamed today.

Wales also remains below average compared to the rest of the OECD, placing an equivalent of 41^st out of 65 for reading, 43^rd for maths and joint 36^th place for science.

The changes to the raw scores are barely significant enough to make a big difference - we're talking 1% falls compared to 2009 (or a 1% rise in case of reading) - so it's fair to conclude that Welsh results have stagnated or frozen.

Examining Welsh Performances

International Comparisons

The Welsh education system is roughly as "awful" as
Sweden's, based on the 2012 results.
(Pic : Aberdeen University)

This is the whole point of PISA, but it seems everyone's determined to compare pencils with England alone. Wales – while far from brilliant – isn't the worst by a long shot, though there's clear need for improvement.

• Wales' score in maths places us in the bottom half of the table alongside Israel (466) and Croatia (471) but well ahead of Greece (453), Malaysia (421), UAE (434) and Qatar (376).
• In reading, Wales scored similarly to Sweden (483), Israel (485), Iceland (483) and Slovenia (481).
• In science, Wales scored similarly to Croatia (491), Italy (494), Luxembourg (491) and Russia (486) and was just 10 points off the OECD average.

However, in addition to being behind the other Home Nations on the whole, Wales is light years behind many other similar nations like New Zealand, Finland, Ireland and Slovenia.

That's before mentioning far-east nations like Singapore, Taiwan and South Korea who are in a league of their own, due to very intensive, competitive education systems and heavy use of private tuition. Those bring their own problems - like high suicide rates.

So, internationally speaking, Wales is a "C student who needs to buck their ideas up" and needn't sport a dunce's cap, while the UK as as whole isn't doing that much better. There are plenty of developed nations in the same boat we are.

To make a significant jump up the rankings, Welsh performances would have to improve by up to 10% (~40points).
Judging the overall figures, Welsh education standards are directly comparable with Israel, Sweden and Croatia.

You know...."third world" Sweden. Where the UK Conservatives went to get ideas for their school policies.The same Sweden which has an education system that's near enough as "awful" as Wales', just to put things in much-needed perspective.

Economic Performance

This is the first thing people point to when they analyse the results, (somewhat) logically deducing that PISA influences international investors when deciding where to invest, or impacts the economy in some great way.

Is there a correlation between good PISA scores and higher economic productivity?

2012 mathematics scores for the 65 nations/territories plotted against GDP per capita.
(Click to enlarge)

Using the mathematics scores in isolation - no. There's no strong correlation at all. If there is one it's incredibly weak.

All PISA does is compares test results, giving you a good idea of the strength of respective education systems. As pointed out, Wales scored similarly to Sweden, and performed better than many nations we've lost foreign direct investment to over the years, including parts of eastern Europe.

There's a whole host of factors that determine whether businesses make an investment. Education and skills is one key part of that, but it's not the be all and end all.

There is, however, a clear correlation between socio-economic background and good PISA scores. According to Volume II of the report itself (p38), the difference in performance – on average - between the top advantaged quarter and bottom disadvantaged quarter is up to 90 points, with disadvantaged students twice as likely to wind up in the bottom quarter than advantaged students.

In economies where there is greater "resilience" (pupils who perform better than their socio-economic background suggests they would), the scores are higher. That means in order to improve performances, more chances need to be given to disadvantaged students.

If you are interested I suggest you read the whole report. However, it cements the link between individual motivation, deprivation (not money spent on schools) and poor educational performance.

The Welsh Education System

Time to invest more capital in the classroom, not on classrooms?
(Pic : Wales Cnline)

As ATL's Philip Dixon explained on Click on Wales, turning this around doesn't come down to selective schools, the Welsh language, not enough time teaching "the Three Rs" or whatever knee-jerk response the media and commentators want to come up with to explain things.

The report says higher performing nations "allocate school resources more equitably amongst advantaged and disadvantaged schools" and "grant more autonomy over curricula and assessments".

They also value teachers more. We see them as workhorses who can never do their job properly.

Policies outlined to improve educational equity include :

• Targeting low-performance schools, or low-performing students within schools - This could include a special curriculum, early prevention programmes or, in some circumstances, grade repetition ("holding students back a year").
• Targeting disadvantaged students – This includes, once again, a special curriculum, including disabled students in mainstream classes, and conditional financial incentives to make sure children regularly attend school. Technically, Wales is already doing some of this through the Flying Start scheme and Pupil Deprivation Grant.
• Raising standards for all students – Altering the curriculum, increasing class time, changing the length of the school day and improving teacher training.

I'm not convinced this is something you can "throw money at", as the Welsh Government and opposition parties might be minded to do. Clearly the answers don't lie in things like selective schools either.

The answer, in Wales' case, probably lies in the curriculum and how subjects are taught. It's no good just "teaching to the test", the curriculum itself is going to have to change fundamentally and teachers are going to have to be given the freedom, and trust, to innovate.

You can still do all that while setting down national guidelines and targets. The control freakery in Cathays Park needs to stop.

PISA puts theory in a practical, problem-solving context making it different from any other exam. Welsh schools generally just teach the theory and give straight-forward number/word-only questions, which are never placed in their practical, "real world" use. So when pupils come across a real world example, they might not fully understand the question or might miss key bits of information.
Successive Welsh Governments might well have pumped a lot of money into new schools, but they've clearly neglected the teachers and the curriculum itself. There's no point having lots of shiny new buildings if they're not properly equipped in terms of trained teachers who can deliver a meaningful curriculum.

So if there's one area I would expect more money and policy capital to be spent on, it would be professional development for teachers and developing a brand spanking new National Curriculum – not deprivation grants and school extensions.

And, of course, all the existing measures need to be given time to work too. Big changes take time to filter through though in education – perhaps up to a decade or more. Any measures taken in the last 5-10 years might not make a noticable impact until the 2020s.


Burnt by melted cheese? Let it cool first

The results are embarrassing. However, some of the the reaction to the results has been even more embarrassing. Everyone needs to CALM DOWN.

There's nothing in this worth getting excited about, and we're pretty much in exactly the same place we were last time around. Wailing and gnashing teeth without offering any practical solutions isn't going to help anyone. That's going to take time and thought.

Yes, it's disappointing news, and the Welsh Government/Welsh Labour deserve to be roundly criticised. The First Minister should be pretty red-faced right about now. I suspect if Leighton Andrews were still in his old role, when you add the tuition fee stuff to this, he would be looking at the sack.

However, the decline isn't as pronounced as 2009's and there's been a noticeable turnaround in reading. The worst of it might be over, and once various policies like the Literacy and Numeracy Framework begin to filter through, I suspect we might see modest improvements in scores (but not rankings) in 2015.

Luckily (or conveniently) for Welsh Labour, the results of 2015's tests won't be made public until after the 2016 National Assembly elections. Having said that, it also means there's no opportunity for an improvement between now and then to put on election leaflets.

The backdrop to that election campaign in terms of education is now boiling down to two areas - PISA and higher education funding.

It's all well and good that I can take a step back and look at it as part of the "big picture". Try telling parents that.
Everyone's going to have to take a deep breath, think rationally and think of long-term, transformative changes. Policy makers are going to have to look beyond England for inspiration, and I would recommend Finland and Estonia as the best places to start.

Unfortunately, as the media and political reaction to this has proven, there's plenty of knee-jerk politics out there, but precious little patience.