USABILITY OF MULTIMEDIA/HYPERMEDIA COURSEWARE - A CASE STUDY

Thuy-Linh Nguyen, University of Canberra & University College, UNSW (ADFA), Australia.
Ph: +61-6-2688678, Fax: +61-6-2688276, Email: [email protected]
Home Page: https://www.adfa.oz.au/~tln
Courseware Home Page: https://www.adfa.oz.au/MECH
Jan Newmarch, University of Canberra, Australia.
Ph: +61-6-2012422, Fax: +61-6-2157, Email: [email protected],
Home Page: https://pandonia.canberra.edu.au
John Baird, University College, University of New South Wales (ADFA), Australia.
Ph: +61-6-2688279,Fax: +61-6-2688276, Email: [email protected]

Keywords hypermedia, multimedia, usability, statistics, courseware

 

Abstract 

The phenomenal blooming of hypermedia technology in general and the WWW in particular has opened a completely new future for the area of computer-based training. Its use as an educational tool however, is still a controversial topic. The "Cross-platform Multimedia Courseware Project" is educational research undertaken at the University of Canberra and Australian Defence Force Academy (ADFA) to investigate several issues, one of which is multimedia/hypermedia courseware usability. It is a combined study of empirical and theoretical researches, where statistical data were used in conjunction with other studies. After the release of the courseware on the WWW on 28 August, 1995, statistical data about its usability were collected and analysed. The standalone courseware was tested in the classroom environment on the 19 June, 1996. This paper reports statistical results of the study.

  1. Background 
  2. Courseware teaching Fluid Mechanics, a first-year subject taught at ADFA, was authored using Macromedia Director, Symantec C/C++, Visual Basic, MATLAB, HTML, FORM, CGI scripts, etc. The courseware was delivered as both a networked (WWW) and standalone (Mac and PC) application. The Web-based courseware consisted of explanation text (with hyperlinks), an online test, three realtime functions (Aerodynamics functions) and one dynamic document to illustrate and let learners experiment with different concepts in Fluid Mechanics. The standalone courseware extended the Web-based courseware further using animation to provide visual explanations for spatial and time-dependent concepts.

     The Web and the Visual Basic (PC) versions of the courseware were released on the Web on 28 August, 1995. An online questionaire was used to collect users feedback, and two log files kept statistics about the number of accesses to the courseware, and information about guest hosts. Data were collected from 28 August, 1995 until 8 October, 1995. The Director (Mac) version of the courseware was tested for the first time as a teaching-aid (lecture demonstration) tool in a lecture of 18 students on 19 June, 1996. A questionaire was used to collect feedback from students. 

  3. Results
    1. Networked courseware
      1. Online questionaire
      2.  The response rate was very poor, only 8 completed questionaires out of 434 accesses were returned. Most agreed that the courseware was of pratical use for teaching and learning, with only one disagreeing because of the network bandwidth limitations. Another study about network performance showed a quadratic growth of traffic volume, which had severe effect on networked courseware delivery. However 75% users, including those from the USA, still found that the response time was very good, good, or satisfactory [NGUYEN96]. Results from the questionaire are shown in figures 1 to 4.

        Figure 1 Is networked multimedia courseware of practical use ?

        Data source : Questionaire

         

        Figure 2 Why does multimedia enhance learning ?

        Data source : Questionaire

         

         

        Figure 3 Should courseware allow for both sequential access and branching ?

        Data source : Questionaire

         

        Figure 4 Do hyperlinks make it easy to learn ?

        Data source : Questionaire

         

      3. Log files 

      For naming purposes, the FORMs designed for the realtime functions are called Flow form, Kutta form, Joukowski form, and Velocity form. The first page of the tutorial is called the Introduction page. Other pages that contain hypertext and static images are called Tutorial pages.

       During 6 weeks (28/8/95 - 8/10/95), there were 217 distinct hosts visited the courseware. The longest time spent on the courseware was 5 hrs 40 minutes on the Kutta form, and the shortest was 1 second on the Introduction page. Most visits were between 10 - 30 minutes, 5 between 30 to less than 60 minutes, 6 between 60 to less than 90 minutes, 5 between 90 to less than 300 minutes, and 3 over 300 minutes. 9 hosts visited the site twice, 2 three times, 1 four times, and 1 six times. These results are shown in figures 5 and 6.

       

      Figure 5 Time spent on the courseware (28/8 - 8/10/95)

      Data source : Log file

      Figure 6 Number of visits by one particular host (28/8 - 8/10/95)

      Data source : Log file

       

      All hosts that visited the site more than once spent long time on various parts of the courseware and showed obvious interest by their thorough investigation of it. Hosts which spent between 10 - 30 minutes or more also often visited many pages and tried with the real-time functions. Interestingly but maybe not surprisingly, the host with the highest number of visits (6 times) was also the one recorded as spending longest time on it (5 hrs 40 mins). In another study about network performance, it was also recorded in other log files that there were 434 accesses to the Introduction page, 48 attempts to generate realtime plots from the Flow form, 17 from the Joukowski form, and 17 from the Kutta form [NGUYEN96].

    2. Standalone courseware

     Most students showed great enthusiasm about the use of the software. 100% agreed or strongly agreed that the coursware assisted with their learning, especially in visualising concepts. 74% found the coursware easy, pleasant and efficient to use, the rest (26%) did not give an answer. The features students found most useful were: i) The courseware demonstrated spatial and time-dependent concepts clearly (86%); ii) The courseware made learning "fun" (43%). 21% also found that the courseware could be useful as an experiment tool. These results were very much in agreement with those found from the online questionaire feedback. A point which, although common, might be worth noting is that audio effects were less appreciated than visual effects.

  4. Discussion

 With the online questionaire, as the response rate was too low, the result was not statistically significant. However based on those who did reply, the most important point that can be made here is perhaps most people (including 3 from the USA) still found the courseware on the Web of practical use, despite the time delay factor (figure 1).

 The feedback from both the online and the classroom questionaires showed that multimedia courseware was most usable for its ability to explain time-dependent and spatial concepts and, interestingly, making learning "fun". All students expressed high preference and enthusiasm about the use of the courseware in the clasroom. All online users confirmed hypertext enhanced learning, and prefered to have both sequential and branching navigation through the courseware. These results agree with findings in the theoretical research carried out in parallel with this empirical study [NGUYEN97].

 The statistics from the log file showed there were quite a few attempts to generate the real-time plots, and some showed serious interest in the courseware. An investigation of the log files also revealed that these hosts were mostly from educational institutions or NASA, who clearly had concern about computer-based training and/or the specific subject of Fluid Mechanics.

 Conclusion

 All of the above facts suggest that multimedia/hypermedia courseware is highly favourable from users’ perspective.

 References

 [NGUYEN96] T-L Nguyen, J Newmarch, J Baird. Network Performance - Impact on Networked Educational Software Delivery. Proceedings of AusWeb96, pp 345-8 

[NGUYEN97] T-L Nguyen, J Newmarch, J Baird. Hypermedia in Education. Proceedings of VTIC’97, to appear