Home

The scope of this literature review has been limited to articles and studies completed in the last five years. Until recently, many believed that reliable and valid research studies regarding the criteria used in distance education courses were rare. This paper concentrates primarily on an evaluation of the original research—including experimental, descriptive, correlational, and case studies. While this review of original research does not encompass every study published since 1995, it does capture the most important and salient of these works, which are few.

Over the past ten-years, distance education has created quite a stir among educators. Going outside the traditional classroom approach to teaching/learning, distance education has been an experimental design that has not had the support of established educational heads. Each year the debate has risen to a higher level: Is distance education as good as the traditional approach to education? If so, what criteria (benchmarks) characterize those distance education courses?

The most difficult part of this research was finding quality and reliable case studies regarding distance education courses. There appears to be three primary shortcomings in research studies which test and measure the effectiveness of distance education courses (What's the Difference?, 1999, pp. 3,4):

Following are two case studies where it appears all three of these controls are followed. The first has a narrow focus (one course) involving 33 students, while the second one has a larger focus (nine courses) involving about 4,000 students.

An experimental design was carried out during the Fall 1996, in which 33 students in a Social Statistics course at California State University, Northridge, were randomly divided into two groups, one taught in a traditional classroom (control group) and the other taught virtually on the World Wide Web (experimental group). The null hypothesis was that face-to-face interaction makes no difference in student test performance. The research hypothesis asserted it did, arguing that such face-to-face interaction with the professor is fundamental to the learning process and that without it, students suffer (Schutte, 1997).

All students were given identical pre-term questionnaires, midterm tests, and final tests. The tests consisted of four parts: 1) matching; 2) objective; 3) definitions; and 4) problems. No significant difference appeared in any of the demographic or experiential variables. The same instructor was used for the control group and the experimental group, although researchers were unable to control or measure the teaching effort across the two sections.

Contrary to the proposed hypotheses, quantitative results demonstrated the virtual class scored an average of 20% higher than the traditional class on both the 100-point midterm and the final examination. Further, post-test results indicate the virtual class had significantly higher perceived peer contact, spent more time on class work, and had a perception of more flexibility, understanding of the material and greater affect toward math at semester’s end than did the traditional class. Thus, the null hypothesis was rejected. The probability of error was less than one percent.

An authentic control group was used for this study as was true randomization. The midterm and final examinations were quantitatively measured. Therefore, it appears this was a reliable research study. However, it only tested one class at one campus, so it does not represent the broader picture for distance education in higher education.

Between 1996 and 1998, the Sloan Center for Asynchronous Learning Environments (SCALE) at the University of Illinois at Urbana-Champaign, performed nine research studies which were entitled, "Efficiency Projects" (Arvan, September, 1998). The nine Efficiency Projects were specifically aimed at using ALN (Asynchronous Learning Network) to achieve higher student/faculty ratios without sacrificing instructional quality.

The nine ALN courses studied under the Efficiency Projects were only a small number of the courses supported by SCALE (there were 80 ALN courses per-semester offered in the 1997-98 academic year). However, the nine ALN courses did account for about half of the 8,000 students enrolled in the ALN courses. The following table shows the criteria used for this study:

ALN Courses Studied	Common Criteria	Other Criteria
1. General Chemistry	¨ All large undergraduate classes ¨ Automatic, web-based grading software ¨ All online developers were early adopters of technology	¨ Range of ALN experience of instructors: none to prior ¨ Help for students: asynchronous, synchronous (text-based chat), face-to-face ¨ Delivery methods: large variation in authoring, presentations, coaching, and traditional lecture format ¨ Environment of campus: culture of daily life permeated with technology
2. Advanced Organic Chemistry
3. Circuit Analysis (Intro)
4. Differential Equations
5. Microeconomics
6. Intermediate Microeconomics
7. Microbiology (Intro)
8. Spanish (Intermediate Grammar)
9. Statistics (Introductory for non-technical students)

From the results obtained from SCALE's Efficiency Projects, the study supports the view that when a sensible pedagogic approach is embraced that affords the students with avenues to communicate about their learning, Asynchronous Learning Networks (ALN) can produce real efficiency gains in courses without sacrificing the quality of instruction.

In early 1999, Tom Russell developed a web site with the title, No Significant Difference Phenomenon (Russell, 1999). Here he collected extracts from 355 research reports, summaries, and papers—a comprehensive annotated bibliography on technology for distance education. With documents dating back as far as 1928, Russell relates that there is no significant difference between teaching/learning with different media.

Russell's No Significant Difference claim set off a fire for many in the educational community. They retorted that most of the studies reporting positive results in distance education have gaps in their research methods and thus are not reliable or verifiable. To substantiate this point with sound research, last April the Institute for Higher Education Policy released the paper, What’s the Difference? A Review of Contemporary Research on the Effectiveness of Distance Education in Higher Education (What's the Difference, 1999). The objective of this case study was to examine the research on distance education more closely so that public policy may be better informed: "Assessing the quality of the original research requires a determination that the studies adhere to commonly accepted principles of good research . . . If a study does not abide by these principles, the results can be erroneous or misleading, and therefore lead to conclusions that result in poor public policy" (p. 9).

What was the conclusion of this research paper? "There is a relative paucity of true, original research dedicated to explaining or predicting phenomena related to distance education. The most significant problem is that the overall quality of the original research is questionable and thereby renders many of the findings inconclusive" (p. 2). The study found that much of the research made on distance education did not use control groups or randomization in their studies (p. 9). Furthermore, their instruments used to measure student outcomes and attitudes were questionable (p. 10).

The report's final comments: the key question still needs to be asked, "What is the best way to teach students?" (p. 38) The implication for the reader of the report—the jury is still out regarding the effectiveness of distance education.

As a follow up to the What's the Difference case study, the NEA and Blackboard, Inc., last month released the findings of a first-of-its-kind case study regarding distance education (Quality on the Line, March, 2000). "The results of the study revealed, for the most part, the benchmarks for quality Internet-based distance education were considered important and, in general, the institutions strove to incorporate them into their policies, practices, and procedures" (p. 2).

The case study process consisted of three sequential phases. In the first phase, a survey was sent out to several organizations to find out what benchmarks [criteria] were being used in distance education courses. This search resulted in 45 benchmarks being identified, which were then grouped into seven categories (p. 9). As the study progressed, the list of 45 was reduced to 21 as being the most relevant to distance education courses, and three additional benchmarks were added to bring the total to 24.

In the second phase, institutions that had substantial experience in providing distance education courses were identified for the study. This list was narrowed down to six institutions including a community college, a comprehensive institution, a research institution, and a virtual institution (pp. 9, 10). What was missing in this study was a control group, such as institutions which offered distance education courses, but had little experience in the field.

For phase three, between September 1999 and January 2000 the staff from the Institute of Higher Education Policy (NEA) visited each of the six institutions being studied. In addition to conducting in-depth interviews with faculty, administrators, and students, a survey using a Likert Scale was administered to each interviewee. The survey also was administered to distance education students who were not able to participate in the interview process primarily because they did not reside near the institution. In all, 27 faculty, 62 administrators, 16 individuals who were both a faculty member and an administrator, and 42 students were interviewed and/or completed a survey, for a total of 147 respondents (p. 10). Following is a table showing the high/low range of scores calculated for all the surveys completed on the Likert questionnaires:

45 Benchmarks	Mean	SD	Count
Range - Low	2.8	.0520	72
Range - High	6.1	2.083	141

Although several good outcomes were produced from this study, it also had its weaknesses. For instance, out of thousands of higher educational institutions offering distance education courses, why were only six chosen for the study? Were proper randomization procedures followed for this selection? Also, out of six institutions, why were only 147 respondents surveyed when it takes closer to 1200 to obtain statistical numbers for reliable and trusted research? Accordingly, more comprehensive studies need to be made to test the null hypothesis that distance education courses that use specific benchmarks are as effective as traditional classroom courses.

In order to provide teachers with the simplest of transitions when setting up distance education courses, new companies like Blackboard.com and eCollege.com have designed templates to greatly reduce the time it takes to get courses online. This is perhaps the most important criterion for getting distance education courses online.

The virtual campus for the University of Colorado (CU) uses the delivery system (interface design) of eCollege.com as shown below (see: http://www.ecollege.com or http://www.cuonline.edu):

Instructors delivering their courses through CU Online make use of cutting-edge technology for presenting course content such as streaming audio, video, and multimedia slide shows. A number of technologies allow students to interact with the instructor and their peers—these include threaded discussions in a bulletin board-type area, live discussions in an online classroom, email, and collaborative workspaces.

Blackboard.com has as their motto, "You do the teaching, we handle the technology." Founded only in 1997, their vision is to transform the Internet and other online networks into powerful environments for teaching and learning (Blackboard Profile). Without any knowledge of HTML, teachers can literally add their instructional material to Blackboard's template in five easy steps and immediately begin to teach their online class. Based on flexible, open technology, Blackboard.com incorporates numerous criteria to enrich the online learning experience (Blackboard Features and Benefits):

The most popular criteria concerning Blackboard is—it is a free service to those who do not charge for their online courses; otherwise, there is a small fee. They also offer more advanced services at different levels of costs for the more serious online instructors. Because of the popularity of their free online course templates, they have more than 3,300 institutions in every state and more than 70 countries that are using their services (Blackboard Overview).

Some companies offering online courses and training have 24/7 online mentors as a valuable complement in enhancing the learning process. Even though this idea is still in the testing stages, it may turn out to be one of the more important criteria used in distance education courses. Microsoft first began to experiment with this idea back in 1998 when implementing training for their Microsoft Certified Solution Providers (Scholars.com, 1998). Since then, SmartForce has taken over the training for Microsoft, but continues to offer the online mentors' immediate help and support for all their students (SmartForce News, 1999).

Western Governors University is another new cyber university. Their motto is, "Education without boundaries." The distance education criteria for their claim to fame is also online mentors; however, they are not available 24/7. Upon enrollment, every student is provided with their own qualified mentor who is an expert in their field of study. This mentor will help the student create and plan a calendar for completing their degree program and will work with their assigned student until they graduate (Western Governors University).

Until recently, no cyber university was accredited. But in March 1999, Jones International University (formerly Jones College Connection) became the first fully online accredited institution of higher learning when North Central Association of Colleges and Schools bestowed upon them this honor (Jones International University, 1999). This is a most important criterion for the future of distance education programs, which will allow them to compete head-to-head with traditional teaching/learning systems.

One of the complaints from traditional educators is the lack of "authentic" and "reliable" research sites available for distance education students. This, too, seems to be another roadblock that is coming down. On April 3, 2000, six of the world's leading educational and cultural institutions announced that they would create Fathom, a company that intends to launch the premier site or knowledge and education on the web. Fathom will present the best public content and courses of universities, libraries, and museums on a wide variety of professional, cultural, and academic subjects. The consortium's website, fathom.com, will introduce the first home for authenticated knowledge on the Internet, serving a worldwide audience of business and individual users (Fathom 2000).

As is shown in this literature review, it is possible to develop quality distance education courses that capitalize on the opportunities provided by emerging technologies. Every day the boundaries for distance education courses are pushed further out. It is hoped that the criteria (benchmarks) set forth in this research paper will provide trusted and reliable resources for any instructor or trainer who wish to put their course(s) online.