Online Learning
Below is a collection of some recent key documents on online learning in reverse chronological order (references to earlier studies can be found in these papers, in particular, in the U.S. Department of Education study). The main focus of this list is the pedagogial and education effects of online learning as well as the faculty perspective on online learning. Suggestions for additions of thorough recent studies, important discussion pieces, or key earlier studies (not quoted in studies listed here) for inclusion on this list are welcome. Email teachers@lmu.edu with comments or suggestions.
Jenkins, R., Online Classes and College Completion, The Chronicle, 2012.
Perhaps what the most at-risk students really need, instead of being herded into online courses, is the "personal touch." Maybe they need more face-to-face interaction with instructors and other students; more conferences in their professors' offices; more private, one-on-one tutoring sessions; more hanging out with their peers in the student center between classes.
Maybe allowing those students to sit at home, alone in front of their computers, with little in the way of emotional support—not to mention, in many cases, educational support—is actually a bad idea. Maybe instead of doing everything we can to encourage them to take as many of classes as possible online, we should be welcoming them to our campuses and into our classrooms.
If future funds for higher education really do follow completion rates, rather than just enrollment figures, then this approach could even end up being more cost-effective for colleges in the long run.
Ragan, L., 10 Principles of Effective Online Teaching: Best Practices in Distance Education, Faculty Focus, Special Report, 2012.
In the traditional college classroom today, faculty and students arrive with a certain set of expectations, shaped largely by past experiences. And although students may need the occasional(or perhaps frequent) reminder of what’s required of them, there’s usually something very familiar about the experience for both faculty and students alike.
In the online classroom, an entirely new set of variables enters the equation. It’s a little like trying to drive in a foreign country. You know how to drive, just like you know how to teach, but it sure is hard to get the hang of driving on the left side of the road, you’re not quite sure how far a kilometer is, and darn it if those road signs aren’t all in Japanese.
Xu, D., Smith Jaggars, S., The Effectiveness of Distance Education Across Virginia’s Community Colleges: Evidence From Introductory College-Level Math and English Courses, Educational Evaluation and Policy Analysis, 2011.
Although online learning is rapidly expanding in the community college setting, there is little evidence regarding its effectiveness among community college students. In the current study, the authors used a statewide administrative data set to estimate the effects of taking one’s first college level math or English course online rather than face to face, in terms of both course retention and course performance. Several empirical strategies were used to minimize the effects of student self selection, including multilevel propensity score. The findings indicate a robust negative impact of online course taking for both subjects. Furthermore, by comparing the results of two matching methods, the authors conclude that within-school matching on the basis of a multilevel model addresses concerns regarding selection issues more effectively than does traditional propensity score matching across schools.
Euzent, P.J., Martin, T.L., Moskal, P., Moskal, P., Teaching Principles to the Masses: Assessing Student Performance in Lecture Capture vs. Face-to-Face Course Delivery, SSRN, 2011.
Public universities are facing fiscal constraints due to declining state tax revenues and a resulting funding decrease. Improvements in technology have made possible the use of on-line lecture capture (LC) instruction. The question arises as to how LC delivery affects the effectiveness of instruction. The research literature on distance learning over the years has typically found no significant differences in student performance for the various instructional delivery modalities. With recent computer and video advances, high quality courses, streamed over the Internet, are now becoming common. However, little research to date has been published that specifically examines student performance or perceptions of courses that employ lecture capture. In this case, lecture capture refers to storing videos of actual course lectures on our college’s computer server, and making them available to students via their course websites, during the semester in which the course is offered. Students may view these videos at their convenience, as often as they wish, and without the need to download the videos onto their own computers. This study examined student performance and student perceptions in two large sections (N>300) of an introductory Economics course. One section employed traditional face-to-face instruction, and the other employed LC over the Internet. This study took place over two consecutive semesters. Students selected their course section (delivery format) during course registration. The instructional methods, exams, and instructor were the same for each section over both semesters. Students who agreed to participate allowed the authors to use their exam and homework scores, final grade, and some demographic data to compare student performance across the two delivery approaches. At the end of each semester, participating students also were asked to complete an online survey about their perceptions of their course section. The results indicated that there were no significant differences in student performance across the two delivery formats. Our results did show a higher withdrawal rate in the LC sections compared to the face-to-face sections (5.1% to 1.9%). LC courses require more discipline because students must complete coursework on their own, and it is likely that freshman and sophomores, in particular, may find this medium more difficult. Finally, student perceptions of LC were quite positive. Seventy-two percent perceived that they had more control over their learning than in a traditional face-to-face class. Forty-three percent also responded that they felt LC enhanced their performance in the course, while only 28% thought that it did not. Eighty percent indicated that LC was as good as or better than the traditional large lecture class experience, and 73% reported that they would choose to take another LC course. The flexibility and convenience of LC were what students liked the most about taking the course this way. Thus, it does appear that, indeed, we are “doing no harm” with LC course delivery, and that many students prefer this medium.
Online Learning 2010: Virtual Education Goes Mainstream, The Chronicle, October 2011.
Will the classroom of the future be a virtual one, a face-to-face one, or a hybrid of the two? Yes to all of those.
Milliron, M.D., Online Education vs. Traditional Learning: Time to End the Family Feud, The Chronicle, October 2011.
Online learning tools and techniques including fully online courses, blended learning, mobile learning, game-based learning, and social networking¬are some of the newest and rowdiest children in the family of higher-education resources. They hold the promise of expanding, improving, and deepening learning for our students. A quick exploration of Carnegie Mellon's Open Learning Initiative, or the Monterey Institute for Technology and Education's National Repository of Online Courses, or Florida Virtual School's Conspiracy Code(a history course in a game) gives you sense of what's possible and what's coming.
Gose, B., Learning the Art of Virtual Instruction: Traditional colleges offer training, along with incentives, for wary professors, The Chronicle, 2010.
Everywhere Carol Maxson looked, colleges were delivering courses online. Everywhere, it seemed, except at Trevecca Nazarene University, where Ms. Maxson is associate provost and dean of academic affairs.
Trevecca's faculty members were not eager to teach online, and many doubted that such courses could create the same sort of community found in classes on the Nashville campus. Ms. Maxson knew she had to do something. "The train had passed us by, and I was hoping that we could grab on to the caboose," she says. So last fall, Ms. Maxson signed up for "Introduction to Online Teaching," a six-week virtual course offered by Nazarene Bible College, in Colorado Springs. Since then, about 30 Trevecca faculty members have taken the course, galvanizing a grass-roots movement at Trevecca toward online education. In October, Trevecca offered its first online course, for its first online-degree programa master's in organizational leadership. While some for-profit institutions, like the University of Phoenix, have a finely honed process for training professors to teach online, at nonprofit institutions skeptical faculty members drive much of the academic decision making, and the process isn't nearly so neat.
Online Learning by the Numbers The Chronicle, 2010.
Figlio, D. N., Rush, M., Yin, L., Is it Live or is it Internet? Experimental Estimates of the Effects of Online Instruction on Student Learning, NBER Working Paper 16089, June 2010.
This paper presents the first experimental evidence on the effects of live versus internet media of instruction. Students in a large introductory microeconomics course at a major research university were randomly assigned to live lectures versus watching these same lectures in an internet setting, where all other factors (e.g., instruction, supplemental materials) were the same. Counter to the conclusions drawn by a recent U.S. Department of Education meta-analysis of non-experimental analyses of internet instruction in higher education, we find modest evidence that live-only instruction dominates internet instruction. These results are particularly strong for Hispanic students, male students, and lower-achieving students. We also provide suggestions for future experimentation in other settings.
"In summary, none of the studies cited in the widely-publicized meta-analysis released by the U.S. Department of Education included randomly-assigned students taking a full-term course, with live versus online delivery mechanisms, in settings that could be directly compared (i.e.,similar instructional materials delivered by the same instructor.) The evidence base on the relative benefits of live versus online education is therefore tenuous at best. From a public and university policy standpoint, the current state of this research is dismaying: More students are being exposed to internet classes yet there is no satisfactory research demonstrating whether such changes help, hinder, or have no effect on student learning. This paper aims to fill this important gap by reporting on an experiment in which students were randomly assigned to either an online or a live section of a course taught by one instructor and for which the ancillaries for the class, such as the web page, problem sets and TA support, as well as the exams, were identical between the sections.The only difference between these sections is the method of delivery of the lectures: Some students viewed the lectures live, as would be the case in traditional classes, while other students viewed the lectures on the internet. Thus we are able to determine how online delivery of lectures compares with live delivery." (p 4-5)
Smith Jaggars, S., Bailey, T., Effectiveness of Fully Online Courses for College Students: Response to a Department of Education Meta-Analysis, Community College Research Center, July 2010.
Proponents of postsecondary online education were recently buoyed by a meta-analysis sponsored by the U.S. Department of Education suggesting that, in many cases, student learning outcomes in online courses are superior to those in traditional face-to-face courses. This finding does not hold, however, for the studies included in the meta-analysis that pertain to fully online, semester-length college courses; among these studies, there is no trend in favor of the online course mode. What is more, these studies consider courses that were taken by relatively well-prepared university students, so their results may not generalize to traditionally underserved populations. Therefore, while advocates argue that online learning is a promising means to increase access to college and to improve student progression through higher education programs, the Department of Education report does not present evidence that fully online delivery produces superior learning outcomes for typical college courses, particularly among low-income and academically underprepared students. Indeed some evidence beyond the meta-analysis suggests that, without additional supports, online learning may even undercut progression among low-income and academically underprepared students.
In keeping with the notion of improved student access as a strongly emphasized rationale for online learning, we first narrowed our focus to the 28 studies included in the Department of Education meta-analysis that compared fully online courses to face-to-face courses. [...] When considering only those studies conducted with undergraduate or graduate students in semester-long online courses, the set of 28 studies is reduced to 7. [...] Lack of consistent differences in outcomes between online and face-to-face. [...] More than half of the studies targeted courses that explicitly taught technology or electronic communication concepts, perhaps because these topics were thought to be particularly well suited for online teaching and learning. [...] All seven studies were conducted at mid-sized or large universities, with five rated as 'selective' or 'highly selective' by U.S. News and World Report, and all seemed to involve relatively well-prepared students. [...] Most institutions place a strong value on increasing access for underserved students. [...]Does online learning meet these goals? For well-prepared and motivated students, perhaps it does [...] For low-income and underprepared students, however, an expansion of online education may not substantially improve access and may undercut academic success and progression through school." (p 3, 4, 7, 8, 11)
U.S. Department of Education, Office of Planning, Evaluation, and Policy Development, Evaluation of Evidence-Based Practices in Online Learning: A Meta-Analysis and Review of Online LearningStudies, Washington, D.C., (revised version) 2010.
A systematic search of the research literature from 1996 through July 2008 identified more than a thousand empirical studies of online learning. Analysts screened these studies to find those that (a) contrasted an online to a face-to-face condition, (b) measured student learning outcomes, (c) used a rigorous research design, and (d) provided adequate information to calculate an effect size. As a result of this screening, 50 independent effects were identified that could be subjected to meta-analysis. The meta-analysis found that, on average, students in online learning conditions performed modestly better than those receiving face-to-face instruction. The difference between student outcomes for online and face-to-face classes—measured as the difference between treatment and control means, divided by the pooled standard deviation—was larger in those studies contrasting conditions that blended elements of online and face-to-face instruction with conditions taught entirely face-to-face. Analysts noted that these blended conditions often included additional learning time and instructional elements not received by students in control conditions. This finding suggests that the positive effects associated with blended learning should not be attributed to the media, per se. An unexpected finding was the small number of rigorous published studies contrasting online and face-to-face learning conditions for K–12 students. In light of this small corpus, caution is required in generalizing to the K–12 population because the results are derived for the most part from studies in other settings (e.g., medical training, higher education).
APLU - Sloan National Commission on Online Learning, Volume II: The Paradox of Faculty Voices: Views and Experiences with Online Learning, Results of a National Faculty Survey, Part of the Online Education Benchmarking Study, 2009.
Recognizing that faculty represent a critical constituency in building quality online learning programs, the APLU-Sloan National Commission included a comprehensive survey of faculty experiences, attitudes, and beliefs toward online learning as a component of its Online LearningBenchmarking Study. This survey was conducted in fall 2008 and winter 2009. The findings contained in this volume are based on responses from more than 10,700 faculty from 69 colleges and universities across the country. Participation varied somewhat for a few institutions, but in most cases the survey was targeted to all faculty members—full-time and part time, irrespective of their level of knowledge and/or experience with online learning. There are a number of divisions apparent among the Faculty Survey responses. First, faculty are not uniform in their opinions toward online learning. Faculty with experience developing or teaching online courses have a much more positive view towards online instruction than those without such experience. Faculty with no online experience remain relatively negative about online learning outcomes. Faculty with online development or teaching experience are not restricted to a particular class of faculty (such as part-time, non-tenure track, or those just beginning their teaching careers), but are well represented among all types of faculty. While there are some differences in participation rates, the overall conclusion is that full- and part-time faculty, those at every stage of their career, and those on the tenure track, as well as those outside of the tenure track, are all involved in online instruction. It is also clear that, in addition to differences of opinion between different groups of faculty members, there are a number of paradoxes apparent among the faculty views. Driving faculty concerns is the pervasive belief that teaching or developing an online course requires more time and effort than for a comparable face-to-face offering. Faculty rate this issue as the most important barrier to teaching and developing online programs. Faculty also report that they have serious reservations about the quality of online learning outcomes, and they believe that their institutions are below average in providing support and incentives. Approximately one-third of all faculty have taught an online course, with around one-quarter currently teaching online. When asked why they teach online, faculty consistently provide student centered reasons. The survey results show that, even with their reservations about online learning, a majority of faculty members have recommended online courses to students, a rate that jumps to well over 80 percent among faculty with experience developing or teaching an online course. The views of the faculty suggest that significant challenges must be resolved before online learning is universally accepted across the academy. However, the paradoxes evidenced by the survey results also suggest considerable opportunity for campus leaders to engage the faculty in constructive dialogue about the quality, support, and overall role of online at their respective institutions.
Webinar: Selected Faculty Survey Result Slides.