By Mary Hamilton, Lancaster University, UK and Co-Director of the Lab for International Assessment Studies   

A key rationale for carrying out international comparative surveys of skills such as the Organisation for Economic Co-operation and Development (OECD) Programme for International Student Assessment (PISA) is that the findings can positively influence policy and therefore educational outcomes. Such claims implicate the media as part of a chain of influence. The argument runs that the media publicise the findings, which influence public opinion and in turn this puts pressure on politicians to respond. The media can also compare past successes, failures and improvements through a running commentary on trends in the test scores.

However, the impact of media on educational policy is assumed but not widely researched. My colleagues and I have followed media coverage of the OECD Programme for the International Assessment of Adult Competencies (PIAAC) in France, Japan and the United Kingdom as well as in Greece, New Zealand, Singapore and Slovenia, which took part in the second wave of (PIAAC-2).

While we can envisage positive roles for the media in policy formation, in practice journalists are often blamed for partial and sensational coverage of international survey findings. Researchers and agencies voice frustration at this, searching for ways to prevent misinterpretation of data and poor commentary. Our research suggests that we need a better, more sympathetic understanding of the constraints under which journalists work along with willingness to share responsibility for the ways in which data from international assessments are translated in the public sphere. 

Firstly, it is important to recognise that educational stories are not generally seen as newsworthy. There are many other pressing issues that claim a place on front pages and one of the great achievements of PISA has been its visibility in the news media and ability to sustain coverage by promoting controversy. Media coverage of PIAAC, on the other hand, was uneven and short-lived. The findings were not always treated as new information. Three of the four countries reviewed in our PIAAC-2 research had poor results in at least one dimension of the survey, yet the way the media perceived and the government responded to these findings differed considerably. In only one country, Slovenia, did it seem likely that the results would be used to develop positive change in adult skills policy.

Secondly, the relationship between media and governments varies across countries. In European Union (EU) countries, pluralistic media driven by a combination of economic gain and ideology, compete to offer the most newsworthy stories, which will maximise visibility of the media outlet. In other countries, the media may operate under the more or less close control of the state and are therefore more likely to operate as a mouthpiece for government policy than a dissenting voice.

Thirdly, the media do not act independently, but are part of networks that stretch from the international to the local. Transnational agencies, politicians and other interest groups as well as the economics of the media industry can actively shape the coverage of international survey findings. In some cases, the coverage is polarised by the selection of politicians and interest groups who are invited to comment on the findings. Across the board, a restricted range of expert voices appeared in the media reports. This not only excludes certain less powerful groups from the public conversation, especially teachers and students, but exacerbates polemic between politicians advocating different strategies and blaming of the absent groups.

The OECD produces selective commentary for each country, which highlights significant features of the complex data and their potential policy relevance within national contexts. This framing is backed up by promotional interventions across all media. Our research shows that journalists in all of the countries we studied follow the OECD’s guidance closely in terms of the policy implications discussed and the data features mentioned.

Reporting the surveys challenges journalists to create data-driven stories within a very tight time-scale. Given such time pressures lack of specialist expertise, it is unsurprising that journalists would rely on readily available summary material that is easy to translate into press reports and headline news. New specialist ‘data journalists’ are needed, trained to scrutinise, analyse or re-analyse, and summarise the findings from such datasets if reports are to move beyond the level of superficial comparative rankings.

In the case of PIAAC, the press coverage reverted to more dominant and familiar national discussions about initial schooling and their implications for vocational preparation. Other national debates entered the stories, with coverage centring on controversial issues not mentioned by the OECD, such as patterns of immigration.

The implications for reducing inequality and for improving citizen well-being emphasised by the OECD were largely ignored. The value of the surveys was seen to be related to employment and monetary returns and, as observed in other survey coverage, there is extensive use of external reference countries, which act as significant markers of national identity. Attention is deflected away from systemic within-country policy issues like inequality and towards the relative performance of competitor states.

With great differences in the media industry structures across countries, caution is needed in making generalisations about the power of the media to hold governments accountable. The conditions under which the media operate mean that they cannot be relied upon to contribute to productive public debate within the domain of educational policy. However, more research is required to unpack the complexity of the media’s role, especially the increasing availability of online and interactive social media, which offer possibilities for journalists, policy-makers and the public to delve more deeply into the findings.

We should see the media as just one element in the space for public debate and conversation that may be thriving or highly constrained in a particular context. Opening spaces for collective thinking and conversation are crucial to truly hold governments to account for their educational commitments and to increase the validity of international assessments and their potential positive impact on policy and practice in national contexts.