By: Gracie Rosenbach and David Spielman
December 2021
Introduction
This blog is part of a series on ‘Conducting Virtual Training during COVID-19: Experience of IFPRI Researchers’. This blog series will reflect on the experiences of IFPRI researchers in conducting virtual trainings during COVID-19—focusing on the methods and approaches, best practices, key challenges and opportunities, and the way forward.
Overview of the learning event on ‘Using Household Data for Policy Analysis’
Rwanda SSP characterizes this type of activity as a ‘learning event’ because we see it as the opportunity to exchange ideas, generate knowledge, test hypotheses, revise existing views, and develop capacity in a collaborative and dynamic manner. The aim is to support relevant skills development alongside the mutual exchange of ideas and dialogue. This type of collaboration and dialogue is essential because the domain of food policy is interconnected, multi-sectoral, dynamic, and evolving. It is crucial to be able to work together in using tools like Stata for data and policy analysis in order to successfully synthesize, identify, and understand complex issues in this domain.
The learning event was conducted using data from Integrated Household Living Conditions Survey (EICV), which is a nationally representative survey collected every few years by NISR. This survey provides information on several household level indicators including educational level, income, employment, household consumption, agricultural production, and household conditions.
Through this learning event, the participants learned how to:
- Describe the data – how can we see what percent of households live in urban areas, and of those households in urban areas, what percent are below the poverty line?
- Transform the data – how can we use the data from NISR to construct new variables of interest, such as whether or not the household is female-headed?
- Analyze the data – how can we estimate and compare household welfare, and how can we analyze what is associated with one household having higher welfare than another?
Methods of the virtual learning event
The information about the workshop was widely advertised among key stakeholder channels in Rwanda (government, donors, universities, NGOs, and other research organizations) and open to everyone. Before the workshop, participants were provided with temporary Stata licenses to download, but otherwise, all learning event activities were undertaken during the scheduled time to allow for increased participation.
During the learning event, the lecturer, Gracie Rosenbach, mainly used PowerPoint presentations and screen-sharing to demonstrate the use of Stata. The event also involved many practice sessions where participants had the opportunity to use their temporary version of Stata to try various codes and examine the resulting output.
The learning event was conducted on the Microsoft Teams platform. The organizers adjusted the audio settings so that participants could not unmute themselves or turn on their video without the organizer giving them permission. Participants were encouraged to use the ‘raise hand’ feature to request to be unmuted, but no one utilized this function. A few IFPRI colleagues were available to respond to participant questions/ queries/ comments in the chat window, of which there were many. The learning event also included many practice and question breaks where people could ask additional questions in the chat window that the trainer responded to out loud. In order to coordinate who would handle which questions and let the trainer know which questions should be addressed out loud, the IFPRI team created their own WhatsApp group for internal communication. All learning event sessions were recorded and made available to participants after course completion.
Best practices
For virtual learning events with large volume of participants and a considerable amount of material to cover, the audio and video setting adjustment discussed above was a good strategy to ensure that there was no unmuting/interjecting. Additionally, given the high volume of participants, the approach of addressing questions through the chat window ensured that questions were addressed in real time without any disruptions to the learning event, and that they could be reviewed at any time during the session by the participants.
The learning event also involved short practice and question breaks which provided participants with time to apply the skills they learned and also to interact with the trainers to get technical clarifications on the course material. Another good practice was to record all of the sessions and share them with the participants in a post-learning event e-mail for their future reference. This post-learning event e-mail was sent to all participants within twenty-four hours after the conclusion of the event which contained notes from the learning event, additional training materials, the recording links from each session, and the PowerPoint presentations. In case of additional questions raised by the participants after the event, another follow-up email could have been sent.
Constructive feedback and future opportunities
One challenge to the learning event was the varying backgrounds of each of the participants in terms of their studies, careers, and even past training experiences. When designing this learning event, it would have been useful to better understand who the participants are, their goals for the training, and their academic and professional backgrounds, including their language skills, amount of time they can dedicate to training, access to technology and internet, computer and statistical skills, and level of comfort with communication channels. Going forward, a good strategy could be to collect this information from prospective participants before the event in order to incorporate these considerations at the design stage.
Defining a learning event evaluation strategy at the event design stage was also very important for the success of this and future events. Evaluation allows for the ability to assess learners’ progress and the quality and efficacy of the course, and it helps in designing improved learning activities and content in the future. Thus, it was useful to include in the evaluation form questions asking participants whether the learning objectives were achieved, how they plan to use the knowledge and skills received, their satisfaction with the quality of support from trainers, and their perception of the logistics.
Such an evaluation questionnaire was circulated after the event which was adapted from those used in other IFPRI country program learning events. The event received good overall evaluation responses, with all participants rating the event as “very good” or “excellent” (with the other options that were not selected being “good”, “poor”, or “very poor”). The most common feedback was that the learning event was too short, and when asked how well they thought the virtual format worked for this event, one-third of participants said “very well”, two-thirds said “somewhat well”, and none said “not well at all”.
The most common feedback was that the learning event was too short. While this is valuable feedback, in trainings with high participation such as this one, it is very difficult to make it long enough that it succeeds in covering all of the material without taking too much of the participant’s time. Ultimately the decision of training length depends on the goals and objectives of the event, as well as the participant registration numbers and time availability.
Additionally, some participants found it hard to follow the learning event since it was in the virtual mode. This was anticipated given the number of participants and the technical nature of training content being covered. To address this concern, the follow-up learning events on more advanced Stata topics will likely only be offered to smaller groups of participants (e.g. only our Government of Rwanda colleagues) to allow for more collaboration and dialogue.
Finally, another piece of feedback from the participants was that they wished the recordings were shared immediately after the sessions rather than after the completion of the entire course. Going forward, sharing the recordings after each session may be considered so that participants can review what they learned in previous sessions and be better prepared to follow along in subsequent sessions.
Way forward
Going forward, the team is planning to have some follow-up learning events on more advanced Stata topics, but will likely only offer these to a smaller and more specialized group of participants such as the Ministry of Agriculture and Animal Resources analysts. If COVID-19 restrictions are relaxed, these sessions might be conducted in-person, otherwise, in smaller group virtual sessions so that everyone does not have to be muted and it can be more interactive.
For example, a smaller (10-15 people) in-person training is tentatively scheduled for November 2021 on using Stata for price analysis. Rather than opening the learning event registration to everyone, it will be for specific government analysts who are specifically working with this price data to support agriculture and nutrition policymaking. It is anticipated that this more interactive learning event will allow for better collaborative learning since we will be able to go through the practice exercises together in person and with the support of our IFPRI research staff to help participants with one-on-one support.
Gracie Rosenbach is the Rwanda Country Program Manager in IFPRI’s Development Strategy and Governance Division. David Spielman is the Rwanda Country Program Leader and Senior Research Fellow in IFPRI’s Development Strategy and Governance Division.