Dublin City University
The School of Computer Applications at Dublin City University is part of the Faculty of Computing and Mathematical Sciences and is responsible within the University for the subjects of computing and software engineering, as well as statistics and operations research. Our undergraduate degree program in computing, the B.Sc. in Computer Applications, has the largest enrolment of any honours degree program in computing in Ireland and has by far the highest demand for places of any computing degree in the country. Undergraduate computing facilities include a network of Pentium and P6 PCs with dual boot into either Windows NT or Solaris. Access is also available to UNIX servers and SUN workstations.
Because of IrelandÆs demographic distribution we have a large population of young people and an increasing demand for College education, especially in computing. Many large multi-national companies have set up manufacturing and distribution operations in Ireland as a base for Europe and the European Community and as a result the demand for higher level qualifications in computing has increased dramatically in recent years. This has resulted in increased class sizes leading to reduced personal contact between students and lecturers which is a bad thing. At present we admit about 150 students into the first year of our program annually, small by US standards but still the largest such program in Ireland. Selection of students is based on performance in state examinations and even with the numbers we admit we must still turn away bright and eager students who would gain entry in most other European countries.
"CA309: Databases" is a module taught to about 110 full-time and 30 part-time students in computing and 15 students in the B.Sc. in Applied Mathematical Sciences. These students are in the third year of a 4-year degree program. The course covers the basics of data modelling, the entity-relationship and relational models, database design and normalisation, SQL, an overview of some database products and some details on database implementation issues such as views, security, distributed databases and the system catalog. For many years the course had been taught using the traditional lecture mode of delivery with about 35 to 40 hours of lecturing, laboratory assignments and an end-of-year written examination. The format for lectures was to use mass produced class materials and notes which were distributed to students, verbal elucidation of concepts by the lecturer using visual aids such as a blackboard as props, and animated explanation. This was backed up by two-way questioning between students and lecturer and informal discussion of the course at pre-arranged or ad hoc meetings.
Generalising, we can say that our educational format in this course had been typical of most University courses, a passive transfer of information via lectures with a re-enforcement of material in the library or laboratory using reference materials or exercises. With increasing class sizes and the pressure to take on even more students, even to double our intake, this trend away from student-lecturer contact was becoming even more pronounced. Such educational delivery has clear weaknesses in the area of personal contacts, distancing the student and lecturer because of numbers, and eroding the number and quality of one-on-one contacts and making most students reticent to approach lectures. In this paper we describe our implementation of a proposal to tackle this problem and we indicate the ways in which our proposal is being evaluated.
There are many ways in which computing technology can be used in education and in training such as the preparation of multimedia course material for distribution using CD-ROM. Indeed an industry of multimedia courseware production has grown up around this. Video conferencing using the internet is also becoming more and more common because of the cost factor and this can be used to deliver live lectures to remote locations. Although this has the advantage of being an interactive delivery of material, video delivery still has some drawbacks when it comes to users feeling involved with others via video links. It has been clearly shown that while computer-supported human-human interactions like attending lectures, mostly concentrates on supporting task-based activities, this is not complete without also incorporating a deliberate effort at replicating the social dimensions of human interactions such as chance encounters and group brainstorming [Bly et al., 1993]. This has implications for our work.
The hottest development in using technology for education, however, is the use of the WWW for delivery of course material and student-tutor interaction and there are countless examples of such initiatives world-wide. The World Lecture Hall [World,1996] is an online repository or listing of such courses; and within the "Computer Science" classification there are already pointers to 94 courses in computer science on offer in various Universities, and computer science is just one off 98 such categories ! Most of these courses deliver teaching material organised as an electronic textbook or as electronic course notes which the student can "read" at his/her own pace. Some also provide multiple routes through the material, search facilities, integration of text, image, audio, video and interaction, and question-answer tests. For example, a series of "virtual" courses in Microbiology and Immunology have been jointly developed by the University of Rochester in NY state and the University of Leicester in the UK and like most such courses have resulted in the creation of electronically-available course notes. As the internet has had an increasing influence on our lives we see online course material emerging with links to other online resources like similar courses, online case studies, databases, etc.
Besides the electronic provision of course material we can also find examples where technology is used to replace the classroom or lecture itself. The New Jersey Institute of Technology (NJIT) has an established reputation in using computing in education for over 20 years and one of their current projects is the "Virtual Classroom"„, a term on which they have a trademark. This project is to use internet technology for distance education and delivery of courses. For example, NJIT run an entire B.Sc. in Information Systems and an entire Graduate Certificate in Object Oriented Design and in both courses the delivery of course material is by internet (WWW and e-mail). The Open University in the UK is possibly the largest distance education organisation in Western Europe and already has an M.Sc. in Open and Distance Education and an M.Sc. in Computing taught using internet as a communications medium, specifically email and WWW. The University of Sheffield has a course on Information Management delivered in a similar manner. These and many other examples electronically present course material organised as a textbook rather than organised by lecture, they do not normally have tutorial backup, they do not involve audio or animation as the primary media for delivery and they have not been evaluated..
In our University, a course in database systems has been provided in hypermedia (text, image, audio) format since 1989. This started not as a set of online course notes but a collection of c.650 nodes of text/graphics information, woven together by hypertext links. Provided as a compliment rather than a replacement for lectures, the course material was indexed and a method for dynamically generating a guided tour through the material in response to a studentÆs query was developed [Guinan and Smeaton, 1992] To deliver this to students we developed our own hypermedia browser and evaluated this as a useful source of learning material during course revision using log files and questionnaires [Smeaton, 1991]. In 1993 the material was updated with more image and audio material added and it was moved to the WWW where the guided tour navigation mechanism is still available and used primarily during course revision by students in our University and elsewhere.
What is common among the vast majority of cases of using computing for presenting course material that we have come across is that it is used either as an adjunct to traditional lectures or it is used to replace the classroom, or sometimes both [Marshall and Hurley, 1996]. Furthermore, material is organised as course notes, is low on multimedia and most importantly has not been evaluated as to its effectiveness in comparison to alternative modes of delivery. Our own efforts in this field over recent years via the guided tour navigation mechanism also fall into this generalisation. While such use of technology to present material in new ways seems to be scaleable to higher number of students technologically, in practice the higher numbers of students we are experiencing introduces new problems because we use lectures as the primary delivery mechanism to these increasing class sizes. At Dublin City University we have implemented what we believe to be at least a part solution to this issue through a series of virtual lectures and the remainder of the paper outlines the philosophy of our approach, describes its implementation and an evaluation of its effectiveness.
In order to address some of the problems of increasing class sizes our proposal was to use computing resources to create virtual lectures and to deliver material to students electronically. This is not the electronic classroom but using computers to deliver the lecture, the first time presentation and explanation of material. In effect we are creating the environment of a personal lecturer, whose voice, diagrams and sketches as drawn on a blackboard, whose animated explanations of topics, and whose course texts are delivered on screen for personalized, tailored delivery. This allows progression through material in a course as fast or as slow as the student wants to go and in an order or sequence driven by the student but guided and recommended by the lecturer. Thus the normal one-to-many lecture with its problems of class size and decreased personal contact is effectively given one-to-one, repeatedly as desired by the student. In addition to following a recommended path through the material yet with the freedom to re-visit segments of previous "lectures" to clarify aspects, to remind oneself of points or to test comprehension of topics, these "virtual lectures" are blended with tutorial sessions, more than is normally made available to students at this level, where students clarify unclear topics and follow set exercises to ensure their grasp of the material.
The tutorial sessions where students meet the lecturer to discuss course material, follow set exercises and clarify course content, are an essential part of our project. It is important for us to address the social consequences of students not attending class as part of a large group and we do this with group tutorial sessions which provide an opportunity for socialising, brainstorming and chance conversations. Because the databases course is one of 6 subjects taken by students where the other subjects are delivered in traditional mode we are confident students do not feel isolated..
The obvious benefits that a series of virtual lectures have are that students take control of their own learning and do so at their own rate, albeit moderated by the lecturer in charge of the course. The passive transfer of information as conventional lectures have become in times of large class sizes is replaced by personalized delivery, with the student determining when, what, how fast, how much and how often material is covered. The re-enforcement of material in tutorials leads to a broader skills base with students working better during tutorial sessions. In our mode of delivery, tutorials assume a more important and central role than otherwise as they become the major point of contact among students and between students and lecturer.
In defining the mode of delivery we now call virtual lectures it was important for us to replicate the lecture mode of delivery as closely as possible so as to minimise the culture shock students experience in moving from group participation in one-to-many classes to individual and user-controlled participation with virtual lectures. Our own experiences with making a hypermedia version of our course available to students over the last 7 years has been mixed with only some students really embracing the technology. This was partly because it was new technology but partly because people are not always comfortable with only a hypertextual organisation of material [Smeaton, 1991] despite the supposed advantages hypertext organisation has of user control [Marshall and Hurley, 1996].
Before we present details of our implementation there were some compelling reasons why we chose the course on databases as given to third year computing students, for this work. One reason was that these students were already quite familiar with Netscape and the WWW and so had no learning curve to climb in using the delivery tools. Also, these students already had a computing culture with the use of PCs on an individual or pairwise basis being an integral part of their daily lives. This is not so true for subjects in science, business or humanities subjects at our University. Finally, the School which is responsible for presenting this course already had the networking infrastructure in place, the Pentium (and now Pentium Pro) PCs with soundcards to deliver the material, and the technical expertise from previous projects to create, update and maintain the material as well as maintain the WWW and RealAudio servers.
The contents of the database course have been re-organised into a series of segments or topic points. In traditional lecture mode of delivery, each of these would be an audio presentation synchronised with some visual props and question-answering with students. To replicate this closely, each segment is presented as a RealAudio track of between 2 and 20 minutes in length. Because we use the RealAudio 2.0 server and RealAudio 2.0 players on multiple platforms we can synchronise the presentation of material on Netscape or other WWW browsers using specification files as input into the server which detail when URLs, corresponding to visual material, are to appear in the userÆs WWW browser during audio playback. During playback users may also pause their virtual lecture to transcribe, take notes, rewind etc., using their paper copy of the screendumps which have been distributed to them. The set of segments comprising the course have been gathered into chapter-sized units for easier navigation. Each student has his/her own unique login identifier to the course material and on login are taken to the latest news of assignment deadlines, new materials online, place and time of tutorials, exam results.
Students are free to browse the material in any order using one of 4 navigation ways:
The audio we use has been digitised using Cool Edit! sampled at 22KHz in monophonic format and saved in 16 bits WAV format. It is then encoded in RealAudio 14.4 format which we use instead of 28.8 because 14.4 has been optimised for speech. The audio is made available as a stream from our RealAudio server running on a SUN Ultra Enterprise 2 server on which our WWW server for the course material also runs. The networking and computing power of this set-up would allow us to handle much more than our current maximum of 50 simultaneous users. Needless to say there is a consistent user interface for screens with feedback and comments, next, previous, home, table of contents, index and search buttons always onscreen.
To reduce disruption in computing laboratories caused by audio over speakers, each student has been issued with a set of walkman-style headphones. All user interactions are logged for subsequent analysis. We had initially planned to use Java applets for animation in the course but have found that when we want to illustrate this then multiple GIF overlays in GIF89a format is adequate for our needs.
The virtual lectures for CA309: Databases has been operational since the end of September 1996, with almost 150 students using the system as their mode of delivery for this course. Rather than being a technically-driven initiative, we include a considerable element of evaluation in our work. Students have completed a pre-course questionnaire in order to determine their familiarity with WWW and RealAudio, as well as their preferred locations for accessing the system. As each student is issued with a unique login identifier, all student interactions are logged and timestamped, on a per user basis, allowing detailed analysis of usage. Finally, students must participate in 6 or 7-person focus group interviews coupled with post-course questionnaires, both anonymous and named, in order to gain insight into what they thought of the system. The focus group meetings will last 30 minutes and will follow a hidden agenda of questions to be discussed in order to elicit studentsÆ opinions on aspects of the virtual lectures mentioned below.
When considering how to analyse our virtual lectures there are 3 aspects we must consider in parallel, namely:
The performance of students is a measure of how well they have learned from virtual vs. traditional mode of lecture delivery. The obvious metric is performance in examinations but this is inadequate because there are so many variables to be considered, the learning ability of classes from one year to another, the relative difficulty of exam papers, etc. The opinions of the external examiners, whose role in our University is to ensure quality standards in teaching and in student performance are adhered to, will be included here, but we feel that there is little more we can do for this aspect of evaluating virtual lectures.
The usability of a system can be measured based on 5 criteria, according to [Nielsen, 1993] as outlined below.
As we can see, for each of NielsenÆs usability criteria we do have some kind of evaluation mechanism in place.
The final aspect of the evaluation of our work concerns the scale of investment needed to create and maintain virtual lectures. A material investment in equipment was needed for things like the machine on which the WWW and RealAudio servers run, the RealAudio server itself as well as other software for creating graphics for example, and the purchase of headphones for students. All of these have proved to be useful for other teaching activities; the lectures for a final year undergraduate course on multimedia systems is being digitised and made available on the equipment obtained for the virtual lectures project.
The big investment in creating the virtual lectures has been the time required. One full-time research assistant worked for 9 months on this project, setting up the servers, marking up the HTML, setting up and maintaining student accounts, troubleshooting, evaluating log files, conducting and analysing questionnaires, etc. The lecturer was also required to put in more effort for the virtual lecture delivery than for a traditional mode. On the plus side, the virtual lectures are now in place and re-usable for next yearÆs teaching of the Database course, but would probably require revision and update after that.
The detailed analysis of log files, questionnaires and focus groups will be completed at the end of March 1997 (students complete the course at the end of February 1997) and at that stage we should know how worthwhile the effort was.
The conclusions from the work we have reported here remain a little bit unknown; we could say that the jury is still our on deciding whether the way in which we have implemented virtual lectures works. What we do know is that the technical challenges have been overcome, in its operation the course delivery works smoothly but it remains to be seen whether the students have learned the material and how they regard the usability and effectiveness of the system. This will be known when the evaluation is complete in March 1997.
There are some other points worth mentioning as conclusions for our work. In order for the virtual lecture to work, the course for which it would be used should not be a first year course as students would still be familiarising themselves with the computing environment. Furthermore, it should not be a final year course as these tend to be more interactive and are critical to a studentÆs final degree classification. CA309 was ideal for virtual lectures as the topics covered remain fairly static over a few years and do not require major overhaul and update on a regular basis so we are happy that we chose the right course for this.
Our virtual lectures are not what is called distance education, though the technology we use can be used for distance education. We maintain close personal contact with students who do work together in groups on campus as part of this module. Whether our mode of virtual lecture delivery could be used for distance education courses is not something we address.
Finally, would we repeat the exercise of providing virtual lectures again, for other courses ? Certainly we have fully grasped the technology for doing this and were we to repeat it we could complete the task in less time than the first time took. While the investment in technology to deliver virtual lectures is becoming less of a requirement, the manpower effort required remains considerable and is considerably more than for delivering a course traditionally. Some of this investment is paid back when multiple runnings of a module use the same virtual lectures but with the cost of updating virtual lectures and personal interaction with students still required, delivering virtual lectures the way we do is not a cheaper alternative to traditional course delivery. Our virtual lectures are useful because of the multiple ways students have to access lecture material online at times that suit the students and not because they are a cost-reduction exercise for the University.
[Bly et al., 1993] S.A. Bly, S.R. Harrison and S. Irwin, "Media Spaces: Bringing People together in a Video, Audio and Computing Environment", Comm. ACM, 36(1), 29-47, 1993.
[Guinan and Smeaton, 1992] C. Guinan and A.F. Smeaton, "Information Retrieval from Hypertext Using Dynamically Planned Guided Tours", in Proceedings of ECHT92 (European Conference on Hypertext), Milan, Italy 122-130, D. Lucarella et al. (eds.), 1992.
[Smeaton and Kelledy, 1996] A.F. Smeaton and F. Kelledy, "Using Thresholding and Accumulator Reduction in a Text Search Engine", in Proceedings of TREC-5, National Institute for Standards and Technology, Washington, DC, (to appear) 1996.
[Marshall and Hurley, 1996] A.D. Marshall and S. Hurley, "The Design, Development and Evaluation of Hypermedia Courseware for the World Wide Web", Multimedia Tools and Applications, 3, 5-31, 1996.
[Nielsen, 1993] J. Nielsen, "Usability Engineering", Academic Press, 1993.
[Smeaton, 1991] A.F. Smeaton, "Using Hypertext for Computer-Based Learning", Computers and Education, 17(3), 173-179, 1991.
[World, 1996] "World Lecture Hall", http:/www.utexas.edu/world/lecture/, 1996