[LINK] 'Online Education Beats Classroom'

Mon Aug 24 08:39:16 AEST 2009

At 14:47 +0000 23/8/09, stephen at melbpc.org.au wrote:
>US Department of Education: Online Education Beats Classroom
>By Chris Crum - Thu, 08/20/2009 - 14:07
><http://archive.webpronews.com/2009a/0823.html>
>Department Says Students in Online Learning Conditions Perform Better

Research of this kind is horrendously difficult to design.

Three variables that present major challenges are the novelty effect, 
the fishbowl effect, and non-random sampling of the population.

The novelty effect arises where the difference in performance may 
transpire to be because the participants are experiencing something 
different from their day-to-day activities (which become a drudge, 
nomatter how hard you try), and basically 'perk up' for a while.  Any 
short study will have great difficulty in controlling for this 
variable, because there's a need to continue long enough for the 
effect to wear off, and then measure the gains made during the period 
*after* the effect wears off.  That's expensive (not to mention 
boring for the researcher), and usually impracticable.

The fishbowl (or 'Hawthorne') effect arises where the participants 
alter their behaviour because they know they're being watched.  It's 
possible for the change in behaviour to be rejection of the 
intervention that's being studied ('bolshie').  But it's usually 
better-than-normal performance, because of the implicit recognition 
that the participants are important.  It's not easy to avoid 
participants knowing that they're being observed, partly for 
practical reasons (because the act of measurement is generally 
visible), but also for ethical reasons.  Allowing for the effect, or 
waiting until it wears off, is also challenging and expensive.

The choice of who's studied is also awkward.  Commonly, convenience 
samples are used, i.e. groups who have been funded to have the 
intervention (as distinct from groups chosen by the researchers). 
These groups may be sufficiently different from a random sample that 
the results are unrepresentative and hence generalisation is fraught 
with danger (aka the study has 'low external validity').  One factor 
is that expensive technologies are commonly invested in first by the 
well-off;  and the well-off may have different performance 
characteristics from other groups.  Another factor is that investment 
may be made in technology for 'underprivileged' groups, which again 
may have different performance characteristics from the other groups.

Added to those complexities is the fact that this wasn't a 
purpose-designed empirical research project, but a meta-analysis, 
i.e. a review of the 'secondary data' from multiple empirical 
projects, selected from a larger set.  That's challenging to get 
right, in a couple of ways at least.  One is the criteria whereby 
some projects are included and some omitted.  Another is how the 
analysts infer key information about the research design and conduct 
(Reports on research frequently leave out vital details.  Trust me, I 
get asked to review far too many mediocre articles on empirical 
studies).

Undertaking good research is really difficult.  I'd sooner be what I 
am - a consultant and dilettante researcher.  (:-)}

</pontification>

________________________________________________________________________

>An interesting study from none other than the US Department of Education
><http://www.ed.gov/> says that students using online education actually
>perform better than those who just learn in the classroom. The findings
>are all the more intriguing, considering the source.
>
>The study examined a number of other studies on the subject to reach its
>conclusions. Online education programs should be pleased with the results.
>
>Some of the key findings from the study:
>
>- Students who took all or part of their class online performed better,
>on average, than those taking the same course through traditional face-to
>face instruction.
>
>- Instruction combining online and face-to-face elements had a larger
>advantage relative to purely face-to-face instruction than did purely
>online instruction.
>
>- Studies in which learners in the online condition spent more time on
>task than students in the face-to-face condition found a greater benefit
>for online learning.
>
>- Most of the variations in the way in which different studies
>implemented online learning did not affect student learning outcomes
>significantly.
>
>- The effectiveness of online learning approaches appears quite broad
>across different content and learner types.
>
>- Blended and purely online learning conditions implemented within a
>single study generally result in similar student learning outcomes.
>
>- Elements such as video or online quizzes do not appear to influence the
>amount that students learn in online classes.
>
>- Online learning can be enhanced by giving learners control of their
>interactions with media and prompting learner reflection.
>
>"The ability to share ideas with anyone in the world is a major benefit
>of online education," says Dr. Sal Arria, CEO/Co-Founder of the
>International Sports Sciences Association (ISSA).
>
>"Once people experience the fun, ease, and power of online education,
>they won't want to stop learning," adds Arria, "Online education allows
>people all over the world to expand their knowledge base, at their own
>pace, and in the way that works best for their schedule and lives."
>
>You have to wonder if the findings will lead to an increased presence of
>online training programs in classrooms across the US. You have to also
>wonder how that might affect teaching jobs, many of which are already in
>peril, due to the state of the economy.
>
>Here is a look at the Department of Education's full report (pdf):
>
><http://www.ed.gov/rschstat/eval/tech/evidence-based-
>practices/finalreport.pdf>
>
>Abstract
>
>A systematic search of the research literature from 1996 through July
>2008 identified more than a thousand empirical studies of online
>learning.
>
>Analysts screened these studies to find those that (a) contrasted an
>online to a face-to-face condition, (b) measured student learning
>outcomes, (c) used a rigorous research design, and (d) provided adequate
>information to calculate an effect size. As a result of this screening,
>51 independent effects were identified that could be subjected to meta
>analysis.
>
>The meta-analysis found that, on average, students in online learning
>conditions performed better than those receiving face-to-face instruction.
>
>The difference between student outcomes for online and face-to-face
>classesómeasured as the difference between treatment and control means,
>divided by the pooled standard deviationówas larger in those studies
>contrasting conditions that blended elements of online and face-to-face
>instruction with conditions taught entirely face-to-face.
>
>Analysts noted that these blended conditions often included additional
>learning time and instructional elements not received by students in
>control conditions.
>
>This finding suggests that the positive effects associated with blended
>learning should not be attributed to the media, per se. An unexpected
>finding was the small number of rigorous published studies contrasting
>online and face-to-face learning conditions for Kñ12 students. In light
>of this small corpus, caution is required in generalizing to the Kñ12
>population because the results are derived for the most part from studies
>in other settings (e.g., medical training, higher education).
>
>--
>
>Cheers people
>Stephen Loosley
>Member, Victorian
>Institute of Teaching
>
>_______________________________________________
>Link mailing list
>Link at mailman.anu.edu.au
>http://mailman.anu.edu.au/mailman/listinfo/link

-- 
Roger Clarke                                 http://www.rogerclarke.com/

Xamax Consultancy Pty Ltd      78 Sidaway St, Chapman ACT 2611 AUSTRALIA
                    Tel: +61 2 6288 1472, and 6288 6916
mailto:Roger.Clarke at xamax.com.au                http://www.xamax.com.au/

Visiting Professor in Info Science & Eng  Australian National University
Visiting Professor in the eCommerce Program      University of Hong Kong
Visiting Professor in the Cyberspace Law & Policy Centre      Uni of NSW