Research on Effectiveness
Tiffany Winman
EMC598
Arizona State University
Summer 1998

In chapter four of Moore and Kearsley's book, Distance Education:  A Systems View, the authors discuss current research in distance education, particularly in the areas of the effectiveness of technologies, media effectiveness, effective course design, teaching strategies, cost-effectiveness, and distance education policies.  Their discussions of the effectiveness of technologies, effective course design, and teaching strategies are particularly interesting.

After summarizing the research on the effectiveness of technology, Moore and Kearsley write, "Given the evidence of research illustrated by the above studies, it seems unreasonable to ask if distance education courses can be as effective as conventional classroom instruction in terms of learner achievement measures" (65).  The authors can make such a confident statement for two main reasons:  1) because a high number of research studies on the issue show that distance education is either equivalent or superior to face-to-face instruction and 2) because they are actually problematizing the way the question is framed in the first place.  They ask their readers to question why we make face-to-face instruction the standard in terms of effectiveness.  Study after study shows that distance education is more effective or at least equally effective depending on the learner's characteristics.  They challenge us to make the learner's characteristics, purpose of pedagogy, and learning environments, to name a few, as complicating factors in education, and not just assume that one method, design, and form of education is the solution for all learners.

Indeed, current research in distance education parallels Moore and Kearsley's admonition to focus on learner characteristics.  In "Chapter V: Effects on Student Achievement" in a document titled Using Technology to Support Education Reform (http://www.ed.gov/pubs/EdReformStudies/TechReforms/chap5a.html), the author writes, "When an innovation is tried, it necessarily includes not just a given technology medium (i.e., computers or television or books) but also particular instructional content and methods (as represented in the software or programming or text). These variables will interact with features of the context within which the innovation is used and with the characteristics of the particular students involved. When differences are found, there is no defensible logic to attributing them to a technology medium rather than to instructional content or method, instructor or student characteristics, or some interaction among these variables."  The author argues for future studies, explaining, "The studies summarized above provide examples of the kind of encouraging results that are being observed in individual projects. There is a need for many more such projects, however, to provide the data needed for the kind of theory-based research synthesis the field needs (Herman 1992)."

Teaching with Technologies at KU Medical Center: Report of the Distance Education Strategies and Technologies Planning Team (http://www.kumc.edu/de_strategies/final_report.html#comparing) lists different types of technologies and mediums that distance education teachers may use today for their pedagogy.  It also explains the appropriateness of when to use each medium.  For example, discussing e-mail, the authors write, "Electronic mail is not an important technology for delivering educational content. It is an ideal means, however, for communication between and among students, faculty, and others involved in course delivery. E-mail is an excellent mechanism for instructors and faculty advisers to communicate with remote students. The great strength of e-mail, aside from its extremely modest cost, is the fact that it is not time-constrained; unlike the telephone, e-mail will sit in an e-mail in box' until the recipient is prepared to read it."  The authors find this type of research important to helping them know when best to use certain technologies and mediums and in developing a quality and cost-effective course.

Although I understand Moore and Kearsley's reasoning behind their statement that we no longer need to ask if distance education courses can be as effective as conventional classroom instruction in terms of learner achievement measures and that we should begin comparing different technologies, I must say that when I first read this statement, I disagreed even after I read the summary of research they provided in support of the effectiveness of distance education courses.  First, I am skeptical of Moore and Kearsley's report because I automatically would suspect that they are biased since they did not include even one report where studies showed negative results for distance education courses.  Now Moore and Kearsley may be absolutely correct, but experience has shown me that there is bound to be one, even if it's out of a thousand, study that reports negative findings.  For instance, the author of Using Technology to Support Education Reform reports that although many studies claim a significant advantage for computer-assisted instruction, when one researcher, "Clark (1985), reexamined samples of the studies included in earlier meta-analyses, however, he found that effect sizes were much smaller when the same teacher provided instruction in both treatment and comparison groups and were absent when instructional method was controlled (such that the study measured the effect of instructional delivery medium only).  Effects were larger in shorter-term studies, suggesting that novelty effects boost performance with new technologies in the short term but tend to wear off over time."  I wanted to know oabout any negative findings of distance education, even if I were to disagree with or problematize the results.  By omitting this type of information, Moore and Kearsley encourage me even more to go out and research the issue (which is a good thing, one could say) to find out this information on my own.

Second, studies can always be improved upon.  Even Moore and Kearsley admit that most studies on distance education are anecdotal, which is good because it is a starting point to help isolate important variables for future studies.  However, if we are really going to take a systems view of distance education, then it will be critical to keep many variables constant to study one, and then to continue further studies that show the complexity and dynamism of the systems view, reflecting the interdependency of all of the variables.

Third, I would argue that even if there are a wealth of studies reporting positive findings, there are always skeptics who will doubt the studies, especially past studies.  For example, the writing program administrator and much of the English Department here at ASU are quite skeptical of distance education, especially since many of the faculty are humanists, not to mention the fact that distance education threatens to put many of us out of a job.  My administrator will ask me to write a proposal to include distance education courses in the curriculum, a request that he is justified in requesting since a distance education component in the program would entail a lot, only one point of which is the high initial cost.  If I submit a proposal with outdated research, my credibility will be weak.  Therefore, it is important to continue researching already proven points for currency's sake, especially since even face-to-face pedagogy is constantly changing.