e-Learning: are all users in front of the computer all the time?

ORIGINAL RESEARCH

e-Learning: are all users in front of the computer all the time?

Josephine Bigeni1, Sarah Bigeni2 and Martin Balzan3,4

1Diabetes and Endocrinology, Mater Dei Hospital, Msida, Malta
 2General Medicine, Mater Dei Hospital, Msida, Malta
 3Respiratory Medicine, Mater Dei Hospital, Msida, Malta
 4Senior Lecturer in Medicine, University of Malta, Msida, Malta

Abstract

Introduction. e-Learning has been shown to be an effective and useful tool in medical education. An e-learning project was part financed through EU funds project ESF1.19 (Malta). Objective. To assess if medical students using e-learning are in front of the screen while using the e-lectures. Method. Eighty-seven final-year medical students were encouraged to complete an online respiratory course (13 streamed lectures+13 quizzes) on a voluntary basis and were offered a reward on completion. A moodle e-learning management system collected data regarding user activity, in particular the duration of participation. A group of 14 core medical trainees who were forewarned of detection of possible irregular activity was also observed as a reference group. Results. Forty-three medical students completed all modules. Six lectures with corresponding quizzes were analysed; 65.5% (male 66.7%, female 64.1%) of the lectures were completed within the expected time frame; in 19% of lectures, users logged in for a period of 5 minutes less than the duration of the lecture, while 14.7% of users logged in for a duration that was twice as long as the total duration of the lecture. Only 16.3% of students completed all lectures within the established period; 27.9%, 20.9%, and 18.6% missed one, two, and three, lectures respectively; and 16.3% missed more than four lectures. In the other group (the 14 forewarned core medical trainees), 65 modules amongst 12 trainees showed that trainees completed a module in a mean of 113.25% (95% confidence interval 109.72–116.78) of the actual duration of a lecture. Conclusion. The study indicates that in 83.7% of the cases, there was at least one instance per user during which the user was not physically present throughout the presentation of a lecture. The e-learning management systems should be robust so as to detect this kind of behaviour before certification.

Keywords: e-learning, web-based learning, undergraduate medical education, learning management system

Received: 13 September 2013; Accepted: 30 September 2013; Published online: 23 October 2013

Journal of European CME (JECME) 2013. © 2013 Josephine Bigeni et al. This is an Open Access article distributed under the terms of the Creative Commons Attribution 3.0 Unported (CC BY 3.0) License (http://creativecommons.org/licenses/by/3.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Citation: Journal of European CME 2013, 2: 22826 - http://dx.doi.org/10.3402/jecme.v2i0.22826

Introduction

e-Learning is a very useful tool in medical education at both the undergraduate and postgraduate levels, as it can help to achieve learning objectives which doctors need before being considered competent.1 e-Learning offers several advantages which include enabling the learners to use the tool at a convenient time and place and to control the learning sequence, and it can therefore be very personalised.1 It also offers educators the opportunity to standardise and easily update content.1 There is also evidence that internet-based learning is cost effective when compared to the traditional learning methods.1,2 This, of course, depends on the cost of the outlay of the e-learning system and the possible savings on travel and accommodation. It is known that web-based learning is as effective as traditional learning in imparting knowledge to the user.1,3,4

However, even in light of these advantages, from our own experience in Malta and from other studies, if e-learning is not included in the design of a curriculum, participation in e-learning will be low.5,6 A recent pilot study done by Abendroth and colleagues7 also showed that only 30% of medical students who were invited to use e-learning participated, and they also suggested incorporating e-learning into a curriculum. Compulsory e-learning in medical education was also suggested by Critchley when only 61–70% of final-year medical students completed a series of e-learning case scenarios.8 These findings create new challenges for the educator in particular to establish whether the online training actually occurred and that educational objectives have been achieved.

The e-learning system deployed in this study used traditional slide and lecture delivery, coupled with assessments in the form of multiple-choice questions. A separate paper reported on usage patterns and user feedback for this particular system.6 As users were able to absent themselves while the presentation was running, the objective of the study was to determine indirectly to what extent users were actually in front of the screen.

Methods

The e-learning management system was the moodle version 1.9.2 which was partly financed through EU funds project ESF1.19 (Malta). Lectures given during a respiratory conference by consultants and specialists were recorded and streamed using the Articulate Presenter® version 09. Quizzes for the corresponding lecture were then uploaded using Articulate Quizmaker® version 09. The e-learning management system was accredited by the Specialist Accreditation Committee of Malta.

All 87 final-year medical students from the University of Malta were encouraged to complete an online respiratory course which included 13 streamed lectures and 13 corresponding quizzes. To improve undergraduates' compliance, upon completion of the online respiratory course, they were awarded a pulse oximeter. User participation could be accurately tracked since all e-learning software was SCORM (Sharable Content Object Reference Model) compliant. Thus, data regarding login and logout times, the number and lists of slides viewed, the total time on the module, and quiz scores could be collected from the e-learning management system.

Two arbitrary criteria (first analysis) were used to give an indication of whether users were actually in front of the computer: 1) user participation of less than 5 minutes of the actual duration of the module and, 2) more than twice the duration of the module.

In a separate assessment, 14 core medicine trainees were asked to complete a number of rheumatology modules as part of their formal curriculum requirements. The trainees were forewarned that they would be monitored for possible irregular activity. Modules performed by this group were used to determine the 95% confidence interval (CI) for the periods of time normally taken to complete a module.

Results

Six lectures with corresponding quizzes were analysed for the 43 medical students who completed all modules. These data were analysed in two different ways.

In the first analysis, 65.5% (male 66.7%, female 64.1%) of the lectures were completed within the expected time frame; 19% of the lectures were logged in for a period shorter than the duration of the lecture; in 14.7%, duration was longer than twice the total duration of the lecture; and data for 0.8% were not available (Figure 1). There was no difference between sexes. Only 16.3% of students completed all lectures within the recommended time period; 27.9%, 20.9%, and 18.6% missed one, two, or three lectures, respectively; and 16.3% missed more than four lectures (Figure 2).

Fig 1
Figure 1.  First analysis* - what percentage of lectures was completed?
*Partially completed=5 minutes shorter than the actual duration of the module. Longer than expected=more than twice the duration of the module.

Fig 2
Figure 2.  First analysis* - percentage of medical students who did not complete the lecture within the expected time frame.
*Partially completed=5 minutes shorter than the actual duration of the module. Longer than expected=more than twice the duration of the module.

For log-in shorter than the total duration, the percentage of modules completed correlated with the total duration of the lecture (R2=0.82). For log-in that was twice as long as the normal duration, the percentage of modules completed correlated inversely with the duration of the lecture (R2=0.50) (Figure 3).

Fig 3
Figure 3.  First analysis* - the lecture duration versus the percentage of lectures which were partially viewed (blue) or where the duration was longer than expected (red).
*Partially completed=5 minutes shorter than the actual duration of the module. Longer than expected=more than twice the duration of the module.

Amongst the group of 14 forewarned medical trainees, two individuals were eliminated as most lectures were either shorter by 5 minutes or longer than twice the duration. For the other 12 trainees, the average duration for 65 lecture modules was 113.25% (95% CI 109.72–116.78).

A second analysis was performed on the medical student group using the module duration of 116.78% (i.e. the upper limit of the 95% interval of the control group for the average duration of a lecture).

Of the lectures, 35.7% were completed within the expected time frame; in 19%, users logged in for a period less than the duration of the lecture; while in 44.5%, the duration was longer than expected. Data for 0.8% were not available (Figure 4). Only 2.3% of students completed all of the lectures within the recommended period; 4.7%, 18.6% and 9.3% missed one, two, or three lectures, respectively; and 65.1% missed more than four lectures (Figure 5).

Fig 4
Figure 4.  Second analysis* - what percentage of lectures was completed?
*Partially completed=5 minutes shorter than the actual duration of the module. Longer than expected=more than 116.78% of the duration of the module.

Fig 5
Figure 5.  Second analysis* - percentage of medical students who did not complete the lecture within the expected time frame.
*Partially completed=5 minutes shorter than the actual duration of the module. Longer than expected=more than 116.78% of the duration of the module.

Once again, direct and inverse correlations were noted between the percentage number of modules partially completed and those modules with a duration in excess of 16.78% of the modules (Figure 6).

Fig 6
Figure 6.  Second analysis* - the lecture duration versus the percentage of lectures which were partially viewed (blue) or where the duration was longer than expected (red).
*Partially completed=5 minutes shorter than the actual duration of the module. Longer than expected=more than 116.78% of the duration of the module.

Discussion

The e-learning system in this study used a rather traditional lecture and assessment approach, where user interaction was mainly limited to answering the online assessments. For any learning to occur, first and foremost the user must be in front of the computer. This study attempts to estimate, indirectly, the duration of usage using the log-in and log-out times of streamed modules.

A study done by Lanier on 1262 university students using an anonymous questionnaire showed that they admitted to having been less compliant when using e-learning when compared to traditional lecture courses.9 Most of the published studies10,11 rely on self-reporting questionnaires by students and do not measure online behaviour using learning management system software such as the moodle and SCORM.

The e-learning system utilised the moodle plus Articulate software, making it possible to pause lectures and complete the learning modules after a break. This possibly led some students, who were participating on a voluntary basis with a prize as an incentive, to run lecture modules without actually being present in front of the computer. Some students also discovered how to skim through the slides of the lecture modules. For this reason, excessive duration suggested the first eventuality, whereas a short duration suggested the second.

While the criterion of 5 minutes less than the actual duration was clearly indicative, the criterion of twice the actual duration was rather arbitrary. For this reason, a second group of forewarned trainees was used to establish the mean duration for pauses during the modules. This turned out to be considerably less than double, 113.25% (95% CI±3.53). When a second analysis was run, only 2.3% of students were within those criteria, and only 35.7% of modules were completed within the expected time, perhaps indicating that these criteria were probably too stringent. Irrespective of the criteria, the correlations indicated that the longer a module, the more students were likely to exit before completion.

The results of this study suggest that where e-learning is on a voluntary basis, the possibility of users not being in front of the computer is rather high. One must also take into consideration three important aspects which could have affected the student behaviour. This online conference was not part of the official curriculum. Participation was voluntary, and students were encouraged to participate because of the prize. The reference group composed of postgraduate trainees in medicine had e-learning modules incorporated in their curriculum, participation for them was compulsory, and they were clearly forewarned that irregular activity would be monitored and could adversely affect their training appraisal. Nevertheless, two out of the 14 participants in the reference group showed evidence of irregular activity. The remaining 12 were very consistent in their behaviour, giving rise to a rather narrow 95% CI of the duration of the online lecture.

Melissa Olt discussed ways in which irregular behaviour can be minimized. These include 1) log-in systems for online assessment, and distributing user names and passwords just prior to the assessment; 2) several short assessments throughout the course; 3) assignments that require group collaboration; 4) use of a plagiarism search service; 5) software that tracks log-in and log-out times; 6) making students aware that they are being monitored; 7) assessment with multiple-choice questions; 8) rotating curriculum; and 9) having a discussion about academic integrity with students.12

In a separate study by Adkins and colleagues, a questionnaire asking students if they would cheat in a web-based examination led to the following results: 42% stated that they would cheat, but this figure dropped to 14% when the same students were told that they were going to be monitored and that previous students had been caught by the system.11

Conclusion

Learning management systems and online e-learning systems must be robust in order to minimise the possibility of irregular activity, and at the same time the system should be capable of detecting any such irregularities when they occur. When e-learning is part of any type of formal education (undergraduate or postgraduate teaching or continuing medical education), consideration should be given to limit irregular activity.

Declaration of interest

Funding

Part of European Social Funds (Malta) ESF 1.19 (online e-learning management system for post-graduate medical training).

Author(s) Financial Disclosure

None.

References

  1. Ruiz JG, Mintzer MJ, Leipzig RM. The impact of e-learning in medical education. Acad Med 2006;81:207–212. Publisher Full Text
  2. Bandla H, Franco RA, Simpson D, Brennan K, McKanry J, Bragg D. Assessing learning outcomes and cost effectiveness of an online sleep curriculum for medical students. J Clin Sleep Med 2012;8:439–443.
  3. Wutoh R, Boren SA, Balas EA. eLearning: a review of internet-based continuing medical education. J Contin Educ Health Prof 2004;24:20–30. Publisher Full Text
  4. Leong S, Mc Laughlin P, O'Connor OJ, O'Flynn S, Maher MM. An assessment of the feasibility and effectiveness of an e-learning module in delivering a curriculum in radiation protection to undergraduate medical students. J Am Coll Radiol 2012;9:203–209. Publisher Full Text
  5. Palmer EJ, Devitt PG. Limitations of student-driven formative assessment in a clinical clerkship. A randomised controlled trial. BMC Med Educ 2008;8:29. Publisher Full Text
  6. Bigeni J, Bilocca D, Balzan M. Usage patterns of and feedback on online e-learning modules for ‘common trunk’ trainees in medicine. J Eur CME 2012;1:36–41.
  7. Abendroth M, Harendza S, Riemer M. Clinical decision making: a pilot e-learning study. Clin Teach 2013;10:51–55.
  8. Critchley LA, Kumta SM, Ware J, Wong JW. Web-based formative assessment case studies: role in a final year medicine two-week anaesthesia course. Anaesth Intensive Care 2009;37:637–645.
  9. Lanier MM. Academic integrity and distance learning. J Crim Justice Educ 2006;17:244–261. Publisher Full Text
  10. Şendağ S, Duran M, Robert Fraser M. Surveying the extent of involvement in online academic dishonesty (e-dishonesty) related practices among university students and the rationale students provide: one university's experience. Comput Hum Behav 2012;28:849–860.
  11. Adkins J, Kenkel C, Lim CL. Deterrents to online academic dishonesty. J Learn High Educ 2005;1:17–22.
  12. Olt M. Ethics and distance education: strategies for minimizing academic dishonesty in online assessment. Online J Dist Learn Admin 2002;5(3). [Online] Available at: http://www.westga.edu/~distance/ojdla/fall53/olt53.html

Correspondence: Josephine Bigeni, MOP2-Department of Medicine, Mater Dei Hospital, Msida, Malta. Tel: +356 99221444. josephinebigeni@gmail.com

Previous presentation
European Respiratory Society Annual Congress, Vienna, September 2012.
XII Congresso Nazionale della Pneumologia, Catania, October 2012.
VIII Malta Medical School Conference, Malta, November 2012.

About The Authors

Josephine Bigeni
Mater Dei Hospital, Malta
Malta

Medicine (Diabetes and Endocrinology), Higher Specialist Trainee.

Sarah Bigeni
Mater Dei Hospital, Malta
Malta

Medicine, Basic Specialist Trainee

Martin Balzan
Mater Dei Hospital, Malta
Malta

Medicine (Respiratory), Consultant

Article Metrics

Metrics Loading ...

Metrics powered by PLOS ALM

Related Content