Health professions students’ acceptance and readiness for learning analytics: lessons for educators | BMC Medical Education

Data is claimed to be the new oil in the twenty-first century because it provides insightful information that can help in timely identification of opportunities and potential risks for effective decision making [17]. In education, the students leave digital trails in relation to their learning. This data would be useful for identifying students’ learning needs. Although learning analytics is a new tool, this study shows that students have some understanding of its application, mainly from their previous exposure to various business intelligence tools commonly used in the retail industry.

Generally, the students were aware of and agreeable to the data gathered by the university, such as those from the learning management system (LMS) and student lifecycle information system. The survey data informed that the students perceived submission of assignment as the topmost data that the university would collect. These findings were similar to the previous study carried out in six Australian universities involving both undergraduate and postgraduate students [5]. On the other hand, data related to LMS were ranked lower in the present study compared to the study done by West et al. (2020) [8]. Learning culture and context as well as the extent of utilisation of LMS in programme delivery could have influenced the students’ decision.

As with the findings of West and colleagues (2020) [8], the present study revealed that the students were least comfortable with their data linked to the university social media group, university mobile apps usage, and location data from their mobile phone being collected. Demographic data also appeared in the lower comfortability zone. Concerns about being placed in categories, stereotyping, profiling, and segregation were likely the reasons for this [8]. It is important for institutions to ensure data transparency by clearly informing the students about how their data will be utilised in the implementation of LA [18]. They should be assured of data confidentiality and security.

The students accepted that their data which had direct association with their learning to be collected, especially those that could contribute towards improved learning experience and learning environment. They appreciated the additional recommended learning materials, services (e.g. academic writing support and library) and the tracking of their learning progression enabled by LA to support their learning. Similar findings were reported by West et al., (2020) [8]. Therefore, it can be concluded that students’ views on data collected for LA are consistent despite differences in educational context, culture and geographic locations.

While the quantitative data with health professions students was consistent with the general findings by West et al., (2020) [8], the qualitative data provided additional and new insights into the views and readiness of health professions students for LA. In the present study, the three themes emerged from the interview data were self-regulated learning, evidence-based decision making and data management. The data from the quantitative study were consistent with the themes arising from the quantitative study (Fig. 4). The students were agreeable for the relevant data to be collected for LA to promote self-regulated learning. They also expected the data to support evidence-based decision making by the education institution. Both the quantitative and qualitative studies emphasised the importance of effective data management and handling.

Demonstration of alignment of qualitative and quantitative findings

The interviewees expected that LA could help them to monitor their achievement of learning outcomes and give personalised feedback based on their performance in assessments. However, to ensure equity to all students, automated alerts from the system, additional guidance and personalised feedback should be available to all students, instead of limiting to the identified group of students [3].

The study showed that LA could help learners in all three phases of the self- regulated learning (SRL) cycle, i.e., forethought, performance, and evaluation phases [6]. The forethought phase of SRL could be facilitated by LA through reminders for deadline or automated to-do-list in accordance with the students’ daily schedule. In performance phase, LA could analyse their current level of knowledge and progress, recommend materials, and suggest learning partners. In evaluation phase, LA could provide specific feedback for further improvement [9]. Learners could further adjust their learning activities to achieve the desired outcomes based on the recommendations by LA. In the present study, the students also highlighted the usefulness of LA in their learning, as it could inform them about their ranking in the cohort and provide personalised recommendations and feedback.

The students perceived that LA could support them via resource recommendations. This was consistent with the findings in previous studies indicating that LA could recommend additional learning materials based on learning behaviours [3, 8, 9]. Intelligent tutoring systems that detect and address student knowledge gaps, together with intelligent feedback systems powered by artificial intelligence could potentially enhance the educators’ efficiency and student engagement in medical education [19]. Besides, LA could stimulate the students’ motivation by presenting data related to learners’ grades and ranking. The learner’s motivation leads to better achievement [20]. On the other hand, some students felt that they might feel pressured by continuously receiving messages from the system or from the teachers. These might interfere with their autonomous learning [3, 9]. Ten Cate et al. (2020) argued that the specialised recommendations on learning topics enabled by LA could reduce the learners’ sense of ownership over their learning, which could negatively impact their motivation and academic performance [11]. Therefore, students should be given a choice to decide whether to use LA to support their learning.

Professionalism is the core competency in health professional training. The students opined that LA could assist in tracking unprofessional behaviours such as absenteeism through attendance data, issues in time management via submission deadline data, poor teamwork, lack of self-awareness via feedback from peers and faculty, and dishonesty through data on cheating and plagiarism. When this evidence is generated in real-time by LA, interventions and corrective measures could be implemented by the education providers promptly.

In education, teaching and learning gaps exist due to diversity in the students’ educational and social backgrounds as well as their learning styles and behaviour. The findings in the current study revealed that LA could help to identify common issues related to teaching and learning strategies based on learners’ feedback. Therefore, the educators could be more specific in addressing the issues timely. The implementation of LA in education could reduce faculties’ physical and mental stress by addressing the concerns over burnout of both faculty and students [21].

Our findings showed that the students were particularly concerned about data management and data confidentiality in LA system. Similar concerns were highlighted in previous studies [3, 8, 22, 23]. They preferred to have an opt-out option for participating in LA and sharing their data with certain stakeholders. Before implementing LA, it is important to address the issue of students’ unwillingness to share their personal data, as they are critical for tracking the learning behaviours [22]. Data transparency and confidentiality need to be ensured for successful implementation of LA.

The potential danger of misuse of data from LA including selling of data and misinterpretation of data due to masking bias was highlighted by the students. Similar concerns were reported previously [24, 25]. Therefore, it is imperative that the purpose of LA should be made clear before constructing the LA system [26]. The policies and regulations, copyright and ethical concerns with regards to the use of artificial intelligence in medical education need to be addressed [27]. Educators need to be trained in data literacy including handling, analysing, and interpreting the data appropriately. Moreover, the education providers should have a clear policy and statement on data handling, data confidentiality and privacy specifically for LA.

Limitations of the study and future research recommendations

The collection of data was done only in one university among medical, dentistry and pharmacy students. This could limit the generalisability of the findings to other health professional students. Hence, the study should be extended to other institutions involving more health professions programmes in the future. Although the original plan was to conduct focus group discussion, two of the sessions were carried out as individual interview because the students were unavailable during the time of the focus group discussion. Nevertheless, it has been reported in the literature that there was no significant difference in the thematic content obtained from focus group discussion or individual interview [28]. In the current study, the information obtained from these two individual interview sessions was consistent with that from the focus group discussion. Hence, the findings were not significantly affected by this limitation. Another limitation of the study was that students who did not volunteer for the interview could have other concerns about LA which might not have been reflected in the study findings. To ensure effective and sustainable implementation of LA, the views of other stakeholders and end users of LA such as faculty, senior management members of the school and academic support staff should also be explored.