By Jenny Bradshaw and Francesco Avvisati, OECD

The international community is intensively working on a set of goals and targets to be reached by 2030. Among them, the Education for All Steering Committee on Education Post-2015 has identified “knowledge and skills for decent work and life” (Target 4). But which skills are needed for work in future societies? Are they relevant across countries and can they be validly measured on a global scale?

OECD’s PISA student assessment tackles these questions in its tri-ennial assessment of the skills and competencies of 15-year-olds in over 70 countries. There is no doubt that reading, mathematics and science, which form the core of the PISA assessments, continue to be important foundations on which other skills can be built. The challenge is to identify other key transversal skills, and to track whether young people and adults possess them.

Equipping students with the skills to confront and overcome complex, non-routine challenges in order to succeed in employment and life more generally, is increasingly important for countries worldwide. In the future, few of today’s youth will work on tasks that require them to repeat the same procedures that they are learning at school in their families, or on the job: these will increasingly be performed by machines. In contrast, new and dynamic problems that require constant attention, controlled execution, and strategic interaction – such as troubleshooting a faulty mobile phone, deciding the best moment to harvest, or diagnosing a rare disease – remain difficult to fully automatise.

For that reason, PISA 2012 set out to investigate how well the world’s 15-year-olds do when it comes to tackling real-life, interactive problems through creative problem solving. The aim was to assess how well they could resolve problems with no immediately obvious solutions, so demonstrating their openness to novelty, ability to tolerate uncertainty and a capacity to reason and learn outside of school contexts. An earlier measure of problem-solving skills in PISA 2003 focused on students’ ability to analyse problems and make decisions in static situations; PISA 2012 expanded the scope of the assessment to dynamic (or interactive) problems, while recognising that successful problem solving requires more than good reasoning; the willingness to tackle problems in the first place is an important element too.

The PISA definition of problem solving is not specific to 21st century societies in which technology permeates all aspects of life. In fact, to picture a great problem solver, you should rather think of Robinson Crusoe on his desert island. He overcame several practical problems, such as building a canoe (which he had never done before), not only by drawing well-thought-out plans and executing them, but also through a great readiness to learn from mistakes and a strong willingness to engage with problems that were not familiar to him – all aspects that are acknowledged by the PISA framework for the assessment of problem solving.

In PISA, the assessment makes reference to specific contexts, in an attempt to make these tasks relevant and interesting to the 15-year-olds who took the test. Among the tasks used to assess problem-solving skills were figuring out the fastest route on a map, operating an air-conditioner and using consumer feedback to prepare popular consumer products (click on the links or here to take the tests yourself).

The PISA 2012 assessment of problem solving included tasks in which students had to explore the problem situation and initiate experimental interactions in order to gather all necessary information to solve the problem, as they would have to do in real life (and as Robinson Crusoe did almost daily on his island). The computer-based delivery made the inclusion of such tasks possible in a large-scale assessment: students explored virtual problem scenarios while sitting at their desk.

The results of the PISA 2012 problem-solving assessment for the 44 countries and economies around the world that participated were released on 1 April 2014, and showed how important it is to measure such skills for developed and emerging economies alike. Some countries did not participate because of the logistical challenge of it being a computer-based test. While logistical challenges are real, results show that the computer-delivery did not give an unfair advantage to some countries. The assessment was successful in measuring the cognitive skills required for successful problem solving, rather than the level of ICT familiarity. Among the countries that did better in problem solving than in mathematics, for instance, is Brazil. And a finer analysis revealed that Brazilian students had relatively good success on items where curiosity, perseverance and some creativity was required – but lagged behind students in other countries of similar performance when they were required to draw logical conclusions from the evidence or to exercise other abstract reasoning skills.

In PISA 2015, problem solving is being taken one step further, with an assessment of “collaborative problem solving”, which over 50 countries are participating in.

We believe that a focus on learning for the new global development agenda post-2015 is important but that it is only a first step. Identifying the specific skills to focus on and working out how they can be measured in a comparative manner across the globe is the second major step that must now take place. The Technical Advisory Group on the post-2015 education indicators suggested proficiency in problem solving as an outcome indicator for the skills for work target. With its 2012 and 2015 problem-solving assessments, PISA illustrates how to move from a visionary target to a globally agreed framework and to the actual measurement of skills that are key for successful employment and life.