By Ariel Fiszbein, Chair, Board of Directors, PAL Network, and Suman Bhattacharjea, Director (Research), ASER Centre/Pratham Education Foundation, India and Chair, Advisory Group, PAL Network
One of the first tasks of any education system is to help children acquire foundational literacy and numeracy in the initial years of schooling, to equip them with the tools necessary for their journey through school and beyond. The vital importance of this task is acknowledged in Sustainable Development Goal (SDG) Indicator 4.1.1a, which tracks the proportion of children and young people in grades 2/3 achieving at least a minimum proficiency level in (i) reading and (ii) mathematics, by sex. Understanding children’s proficiency levels is key to designing programs and mechanisms to target support to where it is most needed, even more so in the post-pandemic context.
As Silvia Montoya and Luis Crouch have noted in a recent blog, one problem is that robust, large-scale, internationally comparable data on children’s learning outcomes are available mainly at higher levels of schooling. Few sources of assessment data cover early primary grades, as required by SDG Indicator 4.1.1a. The authors note that while international assessments such as the Early Grade Reading Assessment (EGRA) and the Early Grade Mathematics Assessment (EGMA) have been implemented in many countries, these are not designed to provide internationally comparable data, leaving only two regional large-scale assessments (PASEC in francophone Africa and ERCE in Latin America) that are currently used to report against SDG 4.1.1a.
Citizen-led assessments have tried to fill the large data gap on foundational learning
The cornerstone of the authors’ argument – that there is a paucity of robust, large-scale assessment data on children’s foundational reading and arithmetic proficiency – was the reason for the establishment of the Annual Status of Education Report (ASER) in India almost 20 years ago, and the spread of the Citizen-Led-Assessment (CLA) model. CLAs are designed to focus on basic reading and arithmetic skills among school-age children (typically 5-16 years), based on the recognition that in many countries the problem of poor foundational learning is not solved in the early years of primary education.
In addition, CLAs are simple to implement, and results are reported in ways that are easy to understand. Rather than a domain or test score, assessment results clearly convey what an individual child or a group of children can and cannot do. This makes the connection between assessment results and action to improve them easy to visualise, a characteristic that is often missing from large-scale assessment models. As Montoya and Crouch pointed out in another blog, ‘assessment markets’ tend to be inefficient and inequitable, and “consumers are not well informed about how different learning measurement products are made and how they correspond to their specific needs”.
The CLA model was designed to generate assessment data that responded to the specific contexts and learning objectives of the individual countries where it was implemented rather than to provide internationally comparable estimates. For example, although the highest level of the reading assessment is pegged to Grade 2 in all countries, an analysis of the Grade 2 curriculum in each implementing country guides the design of the specific items that are used, and country experts are consulted on appropriate sampling strategies. Therefore, although these assessments generate robust estimates that are, in many cases, both nationally representative and repeated at regular intervals (for instance in India, Pakistan, Kenya, Tanzania, and Uganda), cross-country comparisons using CLA datasets have been more difficult, as Montoya and Crouch point out. While tests often share similar items across countries, they are adapted to each country context.
The establishment of the People’s Action for Learning (PAL) Network in 2015 was the catalyst for the design of a new set of common assessments based on the CLA model. The PAL Network brings together 17 organizations across 15 countries located on 3 continents, with the common mission to generate evidence through assessments and action to inform, influence and improve children’s learning. Building on previous tests which shared similar items across countries, the new tests are fully comparable across countries. This effort is intended to address the paucity of internationally comparable large-scale assessment data on foundational literacy and numeracy.
Citizen-led assessments are adjusting and improving
The network’s first such initiative, the International Common Assessment of Numeracy (ICAN), was recently selected as a winner of a 2023 World Innovation Summit for Education (WISE) award. An open-source, easy to use assessment that was designed to align with the Global Proficiency Framework for mathematics, ICAN embraces all CLA principles but comprises common items that allows for comparisons across geographies. It was first implemented in 2019 in one district each of 13 low- and middle-income countries across Africa, Asia, and the Americas. A second assessment round was carried out in a subset of these geographies, in sub-Saharan Africa and South Asia, in 2022 to assess the impact of the COVID-19 pandemic on children’s foundational numeracy. ICAN can be used both in schools and households.
In 2019, supported by a Knowledge and Innovation Exchange (KIX) global grant, PAL Network, Pratham Education Foundation in India and the Australian Council for Educational Research (ACER) began work on developing the next entrant to the PAL toolkit of common assessments. While ICAN mirrored the original CLA vision of assessing foundational skills among school-aged children (age group 5-16), the Early Language, Literacy and Numeracy Assessment (PAL-ELANA) focuses on children in the age group of 4-10 years, looking specifically at children’s acquisition of pre- and foundational literacy and numeracy skills in the period that bridges SDG target 4.2 (early childhood education) and SDG indicator 4.1.1a (grades 2/3 of primary education). Currently, there are no sources of large-scale assessment data that bring evidence to bear on this critical period of schooling and learning.
The PAL-ELANA assessment includes tasks that assess oral language acquisition, early literacy and early numeracy skills that are comparable across the 9 languages and 12 countries where it will be implemented. These tasks can be mapped up to grade 4 of the Global Proficiency Framework and ACER’s Learning Progression. Data collection is scheduled for early 2024 in three districts in each participating country (Bangladesh, India, Nepal and Pakistan in South Asia; Kenya, Mozambique, Tanzania, and Uganda in East Africa; Mali and Senegal in West Africa; Mexico and Nicaragua in the Americas).
Finally, the newest kid on the PAL Network common assessment block is currently in early stages of conceptualisation. This internationally comparable assessment of foundational literacy and numeracy is being designed to generate nationally representative estimates aligned to SDG 4.1.1a in each participating country in the coming years. The numeracy assessment will be based on an updated version of ICAN, modified in accordance with changes to the Global Proficiency Framework for mathematics. The language and literacy assessment, named the International Common Assessment of Reading or ICARe, will draw on the extensive work done to develop an item bank of comparable items in 9 languages for the PAL-ELANA assessment, and will expand this work to cover additional languages as needed. Together, ICAN and ICARe are more than words that reflect the spirit and capability of the PAL Network and its members; these assessments will also address the final concern expressed by Montoya and Crouch in their blog regarding the absence of nationally representative estimates in some CLA datasets.
With the 2030 SDG deadline rapidly approaching, the determined South-South partnerships for SDG 4.1.1a are growing and evolving to enable cross-country measurements, where simplicity, speed, rigour and equity go hand in hand, ensuring not only comparability and representation, but also ease of communication around the key issues identified. If all children are to acquire basic reading and arithmetic skills in well under a decade, these features are essential to enable rapid uptake of both the tools as well as the findings they generate.
