Providing insights into health data science education through artificial intelligence | BMC Medical Education

This study employed artificial intelligence methods to provide insights into health data science education by analysing an MOOC with over 3,000 enrolled learners. The findings reveal that there is not a strong difference in the frequency of visits to reading materials, video lectures, and labs, although students tended to visit labs and lectures slightly more than reading materials. Also, based on the results, students who actively engaged with practical resources, such as labs, discussions, and projects, achieved higher final grades.

Furthermore, the results indicate that the students engaged more with the Sequence Processing and Python Programming topics. However, students moved forwards and backwards more in the programming topic videos compared to other topics. One inference is that students might find this topic challenging.

To analyse the students’ learning strategies, four learning tactics were identified that are in line with educational learning theories [38, 41]. The most frequently used tactic is Elaboration, which involves learning by pausing and replaying video lectures, possibly to contemplate the taught concepts or take notes. Based on the MSLQ learning theory [38], this tactic assists learners in retaining information in their long-term memory by establishing connections between the items that need to be memorised. This tactic includes pausing and replaying a video lecture, potentially in order to rephrase or condense information into a summary, make comparisons and take notes in an active manner. This tactic supports a learner in combining and linking new information to their existing knowledge.

The second most used tactic is Programming and Problem-Solving, where students engaged with the programming labs and solved the programming assignment. Given the interdisciplinary nature of health data science, students need to develop their knowledge in programming and improve their problem-solving skills [42]. Therefore, this tactic can be effective for students to apply their theoretically acquired knowledge to solve problems by using programming.

The third tactic is Peer Learning, which involves communicating with peers in the discussion forums and solving the peer-reviewed assignment. This tactic was found to have a positive correlation with students’ final performance, which was stronger than for the other learning tactics. Existing literature confirms that peer learning is associated with high performance, especially in online courses where students do not have the opportunity to discuss the learning materials face to face [38, 39]. It could be even more useful in the health data science field because students have diverse backgrounds; therefore, they can share their ideas and perspectives towards multi-disciplinary topics to develop in-depth knowledge and reflect on different approaches.

The final tactic is Rehearsal, which involves learning mostly by going forwards and backwards in video lectures instead of watching them from beginning to end. According to the MSLQ learning theory, it is a basic tactic for learning, which involves repeating information, again and again, to memorise it instead of deeply thinking about it. This tactic is effective for simple tasks and for calling information stored in the working memory, but not for acquiring new information that will be stored in the long-term memory. This tactic is believed to impact attention and the process of encoding information, but it does not seem to help students develop relationships between the information or integrate the information with their prior knowledge [38, 39]. Based on a recent systematic review [39], one study [43] showed that this tactic has a weak positive impact on student performance; while two studies [44, 45] did not find any significant association between rehearsal and performance. In this study, we found a weak correlation between rehearsal and student final grade (the weakest correlation compared to the other learning tactics). Interestingly, the Rehearsal tactic was not used much by deep learners, who achieved a higher final grade than surface and strategic learners.

Based on the frequency of using the learning tactics, three learning strategy groups were identified: low engagement/surface, high/engagement/deep, and moderate engagement/strategic. The learning strategies detected are highly accordant with the well-recognised learning approaches introduced by Biggs [41], Marton and Säljö [37], and Entwistle [40]. These scholars have described three learning approaches named deep, strategic, and surface learning, which are not the intrinsic characteristics of students [41], rather they are selected by students based on the task type and cognitive conditions. Also, students’ motivations and intuitions, the learning environment, the way the course is delivered, and the learning contents are the key factors that influence the choice of a learning approach by students [24, 40].

The high engagement/deep learners’ group is characterised by a high level of engagement, a high frequency of employing various tactics, and a high number of quizzes and project submissions, which is consistent with a deep learning approach, by which students engage with high frequency with the course materials, they are highly engrossed in the ideas and actively try to relate them to previous knowledge [38]. Previous studies have shown that adopting the deep learning approach results in better academic performance [24, 46]. Furthermore, the students with deep learning strategy obtained the highest marks in course assessments compared to other students, which indicates their in-depth knowledge. Also, the high use of the Elaboration, Problem-Solving, and Peer Learning tactics by these students reveals that they tend to focus on course materials for a long time (these tactics include long sessions), relate learning materials to their prior knowledge, focus on programming labs to solve problems and learn from peers and solve a project, which are all aligned with the characteristics of the deep learning approach.

The surface learning approach is adopted by students whose intention is to not fail and who want to achieve a passing mark rather than gain a deep understanding of the materials or obtain high marks. Therefore, these students mainly memorise the required information that is necessary for the exams, do not focus on abstract ideas, and mostly rely on details [37]. This approach has similar characteristics to the low engagement strategy in our study because the students using this strategy only used the Elaboration and Problem-Solving learning tactics with low engagement, resulting in low performance. They also did not use the Peer Learning tactic, which had the highest impact on student performance, because the peer-reviewed assignment corresponds to 30% of the final grade. In the DSM course, the passing score is 50 out of 100, and 50% of the final grade is related to quizzes. A deeper level of knowledge is required for the project compared to the quizzes. DSM students employing a surface approach tend to concentrate primarily on quizzes (by employing the problem-solving tactic) in order to achieve a passing score without investing significant effort in the project.

The strategic or achieving learning approach is described in educational theory as a combination of the surface and the deep approaches [46]. The main motivation of students adopting this approach is to get high scores and manage their efforts to make the most of the assessments done [47]. Therefore, they try to find the demands of assessments, manage their time, study in an organised manner, and routinely make sure that they use proper materials [47]. This learning approach is similar to the moderate engagement strategy in our study. The students with this strategy had moderate efforts, moderate frequency of using different tactics, and moderate performance in comparison to the two other strategies. They also mostly used the Rehearsal tactic, which shows that students moved forwards and backwards in video lectures instead of watching them from the start until the end. This is consistent with the characteristics of strategic learners who prefer to apply timely efficient tactics to manage their learning. Therefore, they used the Rehearsal tactic more than Elaboration because the Elaboration tactic is attributed to more effort, such as pausing and replaying videos instead of only seeking videos. This is also supported by the finding that the number of learning actions per learning session was higher in the Elaboration tactic.

It is worth pointing out that students use different learning strategies in different courses [48]. The learning tactics and strategies in health data science courses may differ from those in traditional biology or data science courses due to their interdisciplinary nature. Students in health data science must engage with both domain-specific biomedicine knowledge and data science concepts, requiring distinct strategies to facilitate their learning process [9, 49]. The learning tactics and strategies identified in this study for the health data science course are unique, though they do share some similarities with the tactics and strategies reported in previous studies on biology and computer programming courses [24].

Recommendations for course design and education improvement

The identified insights about health data science students can help to design better courses and programmes in this field. Most educational design models [50,51,52] need information about students to design effective pedagogical frameworks (e.g., pedagogical strategy and tactics) and educational settings (e.g., learning tasks and organisational forms). For example, learning tactics and strategies could be defined in the form of pattern languages based on [50] for designing better educational frameworks. In other words, a key implication of our study is to provide health data science educational designers with insights about HDS students and their learning behaviours that can potentially assist them in designing better educational courses and frameworks. Our recommendations based on this study are as follows:

In the DSM course, there is a peer-reviewed project in the last week that is responsible for 30% of the final grade. Since many students were not successful in submitting the final assignment, our recommendation is to invite students to work on the assignment throughout an HDS course rather than only in the last week. This can be particularly helpful for LP students, as it can encourage them to remain engaged during all weeks [53].

We showed that students in DSM engaged with a diverse range of learning resources (lab, reading, video, quiz, and project). Previous research has shown that utilising diverse learning resources, such as reading materials, interactive video lectures, games, labs, and so on, can enhance students’ learning experiences [54]. As an example, some students may prefer to look at reading materials instead of videos, or vice versa. Therefore, the available resources should be diverse, as students are diverse in HDS courses. Additionally, previous research shows that integrating interactive resources, such as gamification tools, may increase student engagement and lead to improving their learning outcomes [55, 56].

Our findings demonstrate that student engagement with topics decreased over the course, as evidenced by higher engagement with starting topics compared to ending topics. In the DSM course, as in many MOOCs [57], students have access to all topics/weeks upon enrolling on the course, which might overwhelm students given the large volume of learning materials. This can decrease their motivation, especially if they browse materials and assessments in the final topics and find them challenging. To address this issue, a potential solution could be to provide access to course material sequentially in such a way that a student can only have access to the subsequent topics upon the successful completion of previous topics.

Our results show that students had higher interaction with video lectures in the introductory Python programming topic compared to the other course topics, in particular higher video seek, pause, and play action. There are two possible explanations here. On one hand, students proficient in programming might have found the initial topic relatively straightforward and thus, did not engage with the entire video lecture from beginning to end. On the other hand, students with no programming background might have found the topic challenging and therefore rewatched certain parts of the videos. Given this mismatch, one might wonder how to best design a health data science course that works for diverse student backgrounds, including both computational and non-computational backgrounds. Our recommendation is to still provide introductory programming topics, but make them compulsory for students with no programming experience (so as to get up to speed with programming concepts) and optional for students with advanced programming skills (so that they are not disengaged). Once this is established as a baseline, subsequent programming-related tasks in the course should be designed at a balanced level of programming difficulty [58, 59].

Based on the findings, peer learning in HDS can help students to achieve higher performance. Therefore, grouping students in such a way that each group contains students with different backgrounds and asking them to work on a project may help them not only better learn both computational and medical aspects of the course, but also help them to learn how to collaborate in an interdisciplinary community, which is essential for a career in health data science [49, 60].

Recommendations for teachers and learners

The results of this study have implications not only for educational design, but also for learners and instructors. Learners sometimes are not aware of the most effective learning strategies, and informing them can possibly improve their future learning experiences [61,62,63]. However, course design is not the whole story, and teachers’ presentation approach also plays an important role in improving students’ learning outcomes. Therefore, we also provide some recommendations for teachers that may help them teach HDS more effectively.

Recommendations for learners

Applying multiple learning tactics when interacting with a course was found to be more effective than only using one or two learning tactics. Our findings, similar to previous research [48]. For example, in order to achieve a good grade in programming and enhance one’s programming skills, simply watching video lectures about programming is not enough. Students who practised coding and used the discussion forums to ask questions and solve the peer-reviewed project were more successful.

The results indicate that successful students not only relied on required knowledge for assessments but also went beyond the syllabus [37] and even engaged with optional sections of the DSM course (e.g. case studies). Therefore, we recommend to students to not only follow the essential parts of a health data science course, but also study additional resources to get a comprehensive knowledge of each topic.

Our findings demonstrate that students who paused and replayed video lectures in order to relate the taught concepts to their prior knowledge, take notes, or think deeply about the topics were more likely to achieve high performance in DSM. Our recommendation to health data science students is, therefore, to use the Elaboration tactic along with other effective learning tactics (Peer Learning and Problem-Solving). Using the Rehearsal learning tactic without deep comprehension is not always effective.

Recommendations for teachers

Previous studies [23, 64] have shown that personalised feedback can help students to improve their learning. We recommend that instructors consider students’ learning tactics, strategies, and preferences when they are providing feedback to them.

Our results also show that although there is a relatively low number of posts in the DSM discussion forums, many students visited the discussion forums to read other students’ questions and answers. Given our finding that students who engaged in discussions more were more successful, teachers should encourage students to participate in the discussion forums. Students might be introverted or feel uncomfortable posting on discussion forums; therefore, teachers should motivate them through the use of appropriate techniques. As an example, a study showed that the active presence of teachers in discussions, through asking questions and following up with additional questions, can enhance students’ engagement [65]. Therefore, specifically for HDS courses, we recommend that teachers post a question in the discussion forums and ask students to share their opinions. Also, posting about cross-disciplinary research findings related to each topic might encourage students because it can show the application of each topic [66].

Limitations and directions for future work

The first limitation of this work is around generalisability. Given that in this study we analysed one health data science course, further research is needed to validate the generalisability of our findings. Also, given that the DSM course is a self-paced MOOC, our findings might not apply to other online courses or face-to-face classes. This is particularly important when considering the fact that students who enrol on MOOCs have different motivations [67] and it is possible that some of them did not focus on assessments because it is not part of their mandatory study programme. This limitation can impact findings related to student performance. As future work, we invite researchers to analyse the learning strategies employed by health data science students in other online or face-to-face courses. Furthermore, it is important to acknowledge the impact of user-friendly [68] and inclusive environment design [69] on students’ learning experiences in online courses. This study did not consider these factors, which may have influenced students’ learning strategies and preferences.

Another limitation of this study is to do with lack of access to temporal data (time spent to study each resource) for readings, discussions, and labs in the DSM course. Therefore, the student engagement with different topics (RQ2) was only explored based on the video lectures’ temporal data.

Our findings are limited to students’ clickstream data about the course on the Coursera platform. Since there are well-recognised survey tools, such as MSLQ [38] and self-regulation learning [70], for identifying students’ learning preferences and strategies that can uncover students’ perceptions about their learning regardless of the learning environment, it is worth collecting self-reported data and combining it with data-driven information as has been done for non-HDS courses [71], so as to strengthen results. We regard this as a fruitful avenue for future research.