By Armen Nurbekyan, Gevorg Minasyan and Naneh Hovanessian, Central Bank of Armenia

The COVID-19 pandemic led to a massive shutdown of in-person education across the world forcing teachers and students to adapt quickly to a “new normal” of online learning. While many studies analysed the effectiveness of online classes for student learning, to our knowledge, none have focused on how well teachers can learn new skills online.

In Armenia, the pandemic hit in the midst of the introduction of new financial literacy topics into the school curriculum, just when teachers were lining up to be trained on the subject. While the training was conducted in person before the pandemic, it had to finish remotely afterwards. This natural experiment enabled us to compare how efficient online teaching is relative to traditional face-to-face teaching. We found that online teacher training is no match to traditional in-person teacher education, an important lesson for others to learn.

In our experiment, using detailed data from Armenian schools, we looked for answers to three questions:

1. Did remote learning impact the teachers’ ability to learn?

2. Does the effectiveness of remote training depend on the complexity of the topic?

3. Which individual teacher characteristics determine the ability to adapt?

Meta-analyses of a number of studies in the field reveal that, while e-learning is successful in sustaining academic interest and continuing development, it suffers from a lack of face-to-face interaction with the instructor and classroom socialization.

A recent experiment conducted amongst students of the Lomonosov Moscow State University using web-based workshops, for instance, showed that online learning is no full-fledged alternative to traditional educational settings, as cognitive and behavioural components of professional competencies do not transfer to a virtual environment. Another experiment analysed the effectiveness of online vocational training and found that maintaining student engagement, facilitation of practical skills acquisition and organization of work-based learning proved difficult in online learning.

In Armenia, a new financial education programme was introduced in 2018 into the mathematics and social science curriculum in grades 2 to 11 in 350 schools. Before the start of each academic year, teachers receive training on core financial concepts, as well as teaching techniques and roadmaps. The training has two main objectives: ensure that teachers absorb the necessary financial knowledge; and provide the skills for proper integration and delivery of the new material in the classroom.

Training participants answered tests both before and three months after the training, which have enabled us to measure its success. The training covered two modules on factual knowledge and implementation skills. The module on factual knowledge covered the ‘big three questions’, as defined by Lusardi and Mitchel, which check for understanding of compound interest rates, inflation and risk diversification. The module on implementation skills covered four questions testing integration, lecture planning, concept selection, and grading.

We first looked at the overall improvements three months after the training. The impact was significantly positive for both in-person and remote learners. However, the post-training scores were significantly smaller in 2020 than in 2019, indicating that online training was less effective than conventional face-to-face training.

Figure 1. Learning improvement among online learners in 2020 and in-person learners in 2019

Was remote learning less effective when it came to  acquiring factual knowledge or implementation skills? To answer this question, we came up with two indices on each to measure progress. We found that the level of additional financial knowledge attained by the teachers was almost the same for both years, implying that virtual training can be very effective for sharing general information. In contrast, the improvement in implementation skills was significantly lower for teachers of the 2020 cohort, which would indicate that such skills are difficult to learn through online learning platforms in the absence of appropriate social communication.

Can any of the teachers’ characteristics explain these differences? Several interesting patterns emerge from our findings. Unmarried and divorced teachers did as wellas their counterparts in the offline group, but married teachers performed worse online than when they receive in persontraining. In addition, teachers from richer households performed better during the online training, while performance was independent of household income for the face-to-face training. And lastly, teachers who are the main household earners performed better in the offline training. These factors plausibly proxy for teachers’ learning environments including the quality of online access, as well as workspace conditions. Moreover, they might also be correlated with pandemic-related stress, in particular childcare problems.

Figure 2. Improvement of implementation skills by characteristics

To conclude, the COVID-19 pandemic has forced education systems worldwide to adopt alternatives for face-to-face teaching and learning. While the crisis offers an exceptional opportunity to reimagine education and to start realizing a vision for the future of online learning, the proper functionality of the latter is still a major concern. Our research, based on a financial education programme for teachers, shows that although e-learning works well for factual knowledge attainment, it does not have the same effectiveness when it comes to instructional competencies. Moreover, different groups of learners are affected very differently. Some groups perform equally well in online environments and traditional face-to-face instruction, while others do a lot worse, possibly due to their working environment, technical ability and stress level.