Research on Effectiveness Maureen Schmid EMD598 Arizona State University 2nd Summer Term, 1998

Moore and Kearsley (1996) survey studies "going back more than 50 years" (p. 61) concerning the relative achievement of learners in distance education versus traditional education courses and state that the "usual finding in these comparison studies is that there are no significant differences" (p. 62). G.S. Kaeley also reports that students receiving math instruction at the post-secondary level in a New Zealand program achieved similarly, whether in a distance or traditional program (Kaeley 1988). In several recent studies cited by Moore and Kearsley (Souder, 1993; Martin & Rainey, 1993; and Chute, Baltazar, and Poston, 1989) the distance education students in fact achieved at a higher level than the traditional students did. They conclude that "what makes any course good or poor is a consequence of how well it is designed, delivered, and conducted, not whether the students are face-to-face or at a distance" (p. 65).

Therefore, researchers interested in student achievement have recently focused on course design and teaching strategies, as well as on the relationship between student characteristics and particular media choices for delivery.

S. Chinnanon, in a dissertation entitled Multi-media distance education: A study of factors affecting the educational achievement of adult participants in the radio correspondence project in Thailand (1985), used multiple regression analysis to investigate the relationship between various socio-demographic, psychological, and instructional factors and the success of students in two courses of study: Thai Language and Life Experience. With the former course, the author found that only two factors correlated with student achievement: the quality of radio reception and the previous test score. In regard to the Life Experience course, the quality of radio reception and previous test score also correlated with greater achievement, but three other factors were also found to relate: the number of years out of school, the amount of time spent studying, and the number of programs heard. Demographic and psychological factors did not relate to success.

Since Moore and Kearsey acknowledge that any distance program requires that the medium be "functioning effectively" (p.65), I assume that they would not be particularly surprised nor interested to learn that students who experienced poor radio reception did less well. Common sense also says that greater maturity and more time spent on the course ought to translate into better results, so the study does not seem to shed any particular light on whether radio is the best medium for a particular type of student. The results of this study, then, do not seem to offer much insight into Moore and Kearsey's question concerning "what types of students learn best in one environment, or from one medium, and what characterises those who learn better from the alternatives" (1996, p. 65). The study does show, however, that radio can be an effective medium for a variety of students and, in that sense, Moore and Kearsey would be interested; they want to know whether lower cost media like audioconferencing and, presumably, Radio transmission - can be as effective as more expensive methods like satellite transmission (p. 67).

In a second large study, Ostman, Wagner, and Barrowclough (1988) looked into the factors that led students to drop out of the New Zealand Technical Correspondence Institute's management courses. A survey was sent with questions on many aspects of participants' experience, one of which concerned their preferences about media used. While the abstract I consulted online did not include the results of the report, I would assume that Moore and Kearsley would be interested in learning which media were selected by those who dropped out and which by those who persevered in the program. Reported preferences do not always translate to actual results (A student might prefer satellite television delivery to the print medium, for example, but persevere and do well in a distance course under either condition). However, for certain students and here the study could provide that demographic information a program that incorporates media that they prefer might have helped them persevere. It would be interesting to look at those types of students in programs that use the preferred media to see if the choice of medium does indeed improve retention.

However, an anecdotal study related to me by my husband in the ASU Computer Science department (Aryeh Faltz, Associate Professor of Computer Science, personal communication, July 8, 1998 - faltz@asu.edu) demonstrates to me that media choices by students and outcomes are not always related in expected ways. For a number of years, he has taught at least one course each semester over the closed circuit television system at ASU. For the majority of students, the course takes place in a traditional classroom, except that TV cameras are filming while the class is going on. The TV transmission goes out to remote sites, industrial sites around the Valley, so some students taking the course have only the distance learning experience. Each televised class is also videotaped and these tapes reside in a tape library, available for students on campus or at the remote sites to check out. In (date semester) the tape library personnel decided to investigate the achievement of those who checked out videotapes versus those who did not. Faltz was asked to provide the final course grades for five randomly selected students who had checked out videotapes and five randomly selected students who had not checked out videotapes (each set drawn from both campus and remote locations). The on- or off-campus location made no difference: all students who checked out videotapes did significantly worse in the course than students who had not checked them out. We sometimes pursue studies that relate various media and student outcomes with a secret presupposition that the media choice "results in" the student success. Obviously, the videotape investigation was not a controlled research study, but the little story reminded me once again that "correlation is not causation."

One final article, from the Journal of Computer-Mediated Communication (Biocca 1997) offers no research results, but rather a proposal for modifications that could enhance the users' experience of computer media. Working in the Media Interface and Network Lab at Michigan State University, Biocca proposes research to improve users' subjective experience of "presence" and "embodiment" in virtual reality environments. Incorporating techniques such as these into future computer communications may help to counter the sense of isolation and impersonality that some students experience with distance education via this medium.

Online

Biocca, Frank. (1997, September) The cyborg's dilemma: Progressive embodiment in virtual environments. Journal of Computer-Mediated Communication 3 (2). http://jcmc.huji.ac.il/vol3/issue2/biocca2.html

Chinnanon, S. (1985). Multi-media distance education: A study of factors affecting the educational achievement of adult participants in the radio correspondence project in Thailand. Dissertation. Submitted to the Department of Educational Leadership, College of Education, Florida State University. Available UMI Dissertation Information Service, 300 No. Zeeb Road, Ann Arbor, MI 48106.
http://www-icdl.open.ac.uk/icdl/export/literatu/lit02/00002905.htm

Kaeley, G.S. (1988). A comparative study of performance in a post-secondary mathematics course of mature internal and distance students. Conference Paper. Fourteenth World Conference of the International Council for Distance Education, Oslo, Norway, August 9 16, 1988. http://www-icdl.open.ac.uk/icdl/export/literatu/lit02/00002161.htm

Moore, M.G. & G. Kearsley. (1996). Distance education. Wadsworth Publishing Company. Ostman, R.E., G.A. Wagner, & H. M. Barrowclough. (1988). Adult distance education, educational technology and dropout: The New Zealand Technical Correspondence Institute's management courses. Studies in Education, no. 48. Wellington, New Zealand: New Zealand Council for Educational Research. http://www-icdl.open.ac.uk/icdl/export/literatu/lit02/00002789.htm