Summary: Assessing Dynamics in Computer-Based Instruction
Title: “Assessing Dynamics in Computer-Based Instruction”
Name: Peter Ash
Date: 12/1/02

Reference: Pane, J. F., Corbett, A. T., & John, B. E. (1996, April 17). Assessing Dynamics in Computer-Based Instruction. CHI’96 Electronic Proceedings. Retrieved December 1, 2002 from http://www.acm.org/sigchi/chi96/proceedings/papers/Pane/jfp_txt.htm

Problem: Since producing computer-based movies and simulations require substantial effort, we need to recognize when they add value. Earlier research mostly compared animation-based lessons with no lessons, or lessons where students just read text or examined still graphics. This paper attempts to compare equivalent lessons where one uses animations (movies and simulations) and the other uses still graphics. The subject of the lessons is time-varying biological processes.

Context: The subjects were 34 students chosen from an upper-level undergraduate course in developmental biology at Carnegie Mellon University. Students were rank-ordered on the basis of prior performance with two successive students forming a pair. In each pair, one student was randomly assigned to group A; the other to B.

All students took two test lessons. Group A students took the dynamic version of the first lesson and the static (control) version of the second. Group B students took the static version of the first lesson and the dynamic version of the second.

Participants used Macintosh IIsi computers with 9 MB memory and 13-inch 8-bit color monitors.

Data was collected by a student survey, records of usage patterns and problems, and post-tests. Tests were graded by a graduate student, blind to condition. Data was subject to statistical analysis.

Findings: Students made far less use of the movies and simulations than anticipated. For the first lesson, students ran each movie 1.4 times and each simulation 1.3 times, on average. The second lesson was modified by adding encouragement to run the simulations, and the average number of runs increased to 2.0, still low.

There was no significant difference reported in student satisfaction between static and dynamic lessons, but students complained that the simulations were slow.

Students took somewhat longer to complete the dynamic lessons.

Students in the experimental and control groups scored better on test questions that covered material in the Lesson 1 simulations, but scored about equally on test questions in all other categories including material covered in movies and in Lesson 2 simulations.

Recommendations: Based on the authors’ findings that “dynamic presentations are not a panacea for instructional difficulties” and “even motivated students cannot be relied on to take full advantage of exploratory opportunities”, I recommend educational technologists carefully consider the use of movies and simulations. The decision to use dynamics should depend on the sophistication of the audience; more advanced students may have less need for dynamic presentations.

Simulations must supported by sufficiently fast hardware.

Incentives must be provided that will get the students to run simulations enough times and with enough different inputs to achieve educational goals.

This paper presents some fascinating insights. More research is needed to determine under what conditions dynamics in computer based instruction can improve learning.