Most learning and teaching occurs in virtual spaces, where instruction is delivered through VLE systems such as Blackboard Ultra, either synchronously or asynchronously. As academics, we are constantly being challenged to drive up student engagement by offering a range of strategies and pedagogic practice, with various metrics for measuring their effectiveness. There are common metrics we can use, and each of these metrics can certainly tell a great deal of information about how learners are interacting with their courses. However, they don’t necessarily measure engagement and reasons for poor retention effectively.
Understanding Learner Engagement
Learner engagement on any VLE, including Blackboard Ultra is limited to how well it is being used by the module tutor and course team. What do we mean by learner engagement? Learner engagement can be defined as being a measure of the extent to which an individual is attending to or participating in specific learning tasks or activities. Learner engagement happens not only during the learning process but also after these activities are completed. It can include how much time the learner has paid attention during an activity, including the extent to which the individual can share the knowledge gained with the tutor and/or their peers. By measuring engagement, we can gain a greater understanding of the learning process and how well (or not) we are meeting the needs of our students from an academic point of view.
Challenges Of Measurement
When considering measuring and driving learning engagement, we are challenged to identify criteria and factors that are valid to use across different types of learning activities. Arguably, there has yet to be a convergence on a true metric for engagement. Many universities commonly quantify engagement with metrics, including course completions, learner satisfaction, minutes of learning per month and repeat visits per month. Most of these metrics look at the issue from a relatively transactional perspective on how a system is being used. Although useful, metrics are not necessarily effective ways to assess engagement or what learners have learned throughout the process.
Measuring Engagement
Below are some examples of measuring student engagement, without getting bogged down with analytics and metrics. All of these can be used within the context of Blackboard Ultra as a learning and teaching tool to improve and measure student engagement both synchronously or asynchronously:
- Case Studies can provide a robust method of gaining insights into specific student groups. Data can be collected from these groups which can be used to provide a rich picture of behavioural insight, interactions between students and institutions, and also contextual factors.
- Student Reports are a great way for students to write about their experiences. Students can use this method as a way of self-indicating their engagement and it can be a way of measuring the subjective thoughts and feelings of students, plus other non-observable data. For example, you could conduct a survey where they respond to predetermined questions or something less rigid.
- Work Sample Analysis is a good way to gain an insight into a student’s work methodology and how effective creating engagement within your subject is. For example, you identify and measure levels of cognitive engagement, understanding of the topic, and examples of higher-order thinking.
- Rating Scales and Checklists are simple to construct as they can be sent out in the form of surveys (Survey Monkey is a useful tool for this). Like a mini NSS, they help to show student investment in certain target behaviours such as satisfaction with course subject matter.
- Direct Observations involve creating a technique for tracking and analysing student behaviour. In this type of measurement, you can identify and measure levels of the quality of student work, their interactions with their learning material and peers, as well as how happy they are with their course. These, amongst other aims, are good ways of structuring an overall framework of measuring student engagement.
- Focus Groups are possibly one of the most successful ways of determining engagement. They allow for a detailed, individual, and considered approach to measurement. They can also help you establishment develop context surrounding any overarching issues that may be affecting multiple students.
You can read more about Effective Engagement through Pedagogic Practice by Dr. Sam Elkingtron here: https://blogs.tees.ac.uk/lteonline/2020/05/28/effectiveengagement/