Monday, November 12, 2007

Kirschner, Sweller, Clark (2006) - Summary

Note: I'm posting this as an example of a rigorous examination and deconstruction of an academic paper. these are my notes on Kirschner, Sweller and Clark's paper Why Minimal Guidance During Instruction Does Not Work: An Analysis of the Failure of Constructivist,
Discovery, Problem-Based, Experiential, and Inquiry-Based Teaching.

Through this analysis, the weaknesses of the paper become very clear (I couldn't resist a few leading comments in the text) though I will provide the full account of these in a later post.

Overview of the Debate

The debate:

On one side of this argument are those advocating the hypothesis that people learn best in an unguided or minimally guided environment, generally defined as one in which learners, rather than being presented with essential information, must discover or construct essential information for themselves(e.g., Bruner, 1961; Papert, 1980; Steffe& Gale,1995).

On the other side are those suggesting that novice learners should be provided with direct instructional guidance on the concepts and procedures required by a particular discipline and should not be left to discover those procedures by themselves.

The 'minimally guided approach' has been called:
- discovery learning (Anthony 1973, Bruner 1961)
- problem-based learning (Barrows & Tamblyn 1980, Schmidt 1983)
- inquiry learning (Papert 1980, Rutherford 1964)
- experiental learning (Kolb & Fry 1975, Boud, Keogh & Walker 1895)
- constructivist (Jonassen 1991, Steffe & Gale 1995)

Two main assumptions underlying these theories:

1. "They challenge students to solve "authentic" problems or acquire knowledge in information-rich settings based on the assumption that having learners construct their own solutions leads to the most effective learning experience."

2. "They appear to assume that knowledge can best be acquired through experience based on the procedures of the discipline."

- minimal guidence is provided on process- or task-relevant information if students choose to use it
- "imply that instructional guidance... interferes with the natural processes by with learners draw on their unique prior experience and learning styles to construct new situated knowledge to achieve their goals..."

Human Cognitive Architecture - The Main Argument

(a) the argument

"Any instructional procedure that ignores the structures that constitute human cognitive architecture is not likely to be effective."

"Minimally guided instruction appears to proceed with no reference to the characateristics of [human cognitive architecture]"
- characateristics of [human cognitive architecture] = characteristics of working memory, long-term memory and the relations between them

Minimally guided instruction is unlikely to result in effective learning

(b) the explanation

The result is a series of recommendations educators find impossible to implement, are reluctant to implement, because inimally guided instruction is unlikely to result in effective learning.

Human Cognitive Architecture - The Theory

- concerned with the manner in how our cognitive structures are organized
- based on Atkison & Shiffrin 1968:
- sensory, short-term (STM) and long-term memory (LTM)
- senory memory not relevant here
- relations between STM and LTM are critical to the argument

- not just passive, discrete fragments that we repeat, nor does it have only a peripheral role in thinking, problem-solving
- central role in cognition - everything we see, hear and thinks about depends on it

The nature of LTM:
- "our LTM incorporates a massive knowledge base"
- "we are skillful in an area because our long-term memory contains huge amounts of information concerning the area"
---> evidence: DeGroot 1965 work on chess expertise - experts better able to reproduce board configurations
---> experts derive their skill by (a) drawing on LTM (b) selecting and applying best procedure

Consequences of LTM:
- the architecture of LTM provides us with the "ultimate justification for instruction"
- "The aim of all instruction is to alter long-term memory"
- "If nothing has been changed in LTM, nothing has been learned"
- "any instructional that:
- does not or cannot specify what has been changed in long-term meory,
- or does not increase the efficiency with which information is stored in or retrieved from LTM
is likely to be ineffective."

Working Memory

- Working Memory (STM) is the cognitive structure in which conscious procssing occurs, ie., we are only conscious of information being processed in STM

Two well-known characteristics:

1. Limited duration and capacity
- Peterson and Peterson 1959 - information is lost in 30 seconds

2. Restricted to a small number of element
- Miller 1956 - working memory restricted to 7 elements

Interactions with LTM may be more important than processing restructions
- Sweller 2003, 2004 - these limitations may apply only to new, unlearned information that has not yet been stored in LTM
- when dealing with previously learned info, the limitations disappear
- no known limits to how much such info can be transferred from STM to LTM

Origins of Constructivism,

Steffe & Gale 1995:
Knowledge is constructed by learners, and thus
(a) learners need to have the opportuniy to construct
- by being presented with goals, minimal insruction
(b) learning is idiosyncratic
- so a common instructive strategy is unlikely to be effective

---> This is accuate but the instructional consequences do not follow.

- when given adequate information, most learners can construct knowledge
- no evidence that when given partial information that this ability *improves*
- indeed, the reverse appears to be true

Learners must construct a representation whether or not given
complete or partial information
- complete information will result in a more accurate rep.

Thus, constructivism does not lead to a prescriptive instructional
design theory.

---> Shift of emphasis away form teahing a discipline as a body of knowledge

- instead, an emphasis on learning a discipline by experiencing the process and procedures of the discipline
- assumption that knowledge is best or only learned through experience primarily based on the procedures of the discipline
- leads to a committment to extensive pratcical or project work
- rejection of instruction "based on the facts, l;aws, principles and theories that make up a discipline's content" and discovery and inquiry methods of instruction

"fundamental error to assume that:
- pedagogical content of the learning experience
- methods and processes (ie., epistemology) of a discipline
are identical."

---> Shulman 1986 :

"Content knowledge"
= "the amount and organization of the knowldge per se in the mind of the teacher"
"Pedagogical Content Knowledge"
= knowledge "which goes beyond knowledge of subject matter per se to the dimension of subject knowledge for teaching"
"Curricular Knowledge"
= "the pharmacopoeia from which the teacher draws those tools of teaching or exemplifying particular content"

Despite this clear distinction between learning a discipline and practicing a discipline, many seem to confuse:
- the teaching of a discipline *as* enquiry
(ie., curricular empasis on the research process in a science)
- the teaching of a discipline *by* enquiry
(ie., using the reasearch procss o a discipline as a pedagoy for learning)

Basis for this may be Hurd 1969 - the "rationale of the scientist"
- instruction in a science "should be a mirror image of a science discipline, with regard to both its conceptual structure and its patterns of enquiry"
- this rationale assumes "the attainment of certain attitudes, the fostering of interest in science, the acquisition of laboratory skills..." will be accomplished through the methodology of science (ie., inductively)
- the *fallacy* of this rationale that it makes no distinction between:
a) behaviours and methods of a researcher who is an expert, and
b) behaviours and methods of a student who is a novice
- Kyle 1980 - science is a performance activity after a person has acquired a broad range of skills, and not to be equated with the investigative methods of science-tecahing
- Novak 1988 - science learning as inquiry failed as a result of obsolete epistemology

Comparing Guided and Unguided Instruction

"Controlled experiments almost uniformly indicate that when dealing with novel information, learners should be explicitly shown what to do and how to do it."

- empirical studies - solid research base against the use of minimally guided instruction - see Mayer 2004 for a review
- he suggests that each decade, after research shows the approach won't work, the same approach, under a new name, is proposed
- each new set of advocates unaware or uninterested in previous results

(SD: if you are telling people what to do, then the best approach is to tell them what to do.)

Research Supporting Direct Guidance

Because students learn so little from constructivist approaches, most teachers turn to direct guidance.
- see. eg. Aulls 2002 - describes 'scaffolding' as teachers implement constructivism in the classroom
(a) by modeling procedures for identifying and checking important info
(b) showing students how to reduce info to paraphrases
(c) having students use notes to construct
(d) promoting collaborative dialogue
- other studies showing that when students learn science with pure discovery methods, they become lost and frustrated
- Moreno 2004 - students learn more deeply from strongly guided instruction than discovery
- Klahr and Nigam - quality of leaning better with direct instruction

A. Cognitive Load

"Cognitive load theory suggests that the free exploration of a highly complex environment may generate a heavy working memory load that is detrimental to learning."
- especially in the case of novice learners
- more knowledgabl learners did not experience a negative effect

Mayer 2001 - "guided instruction not only produced more immediate recall of facts than unguided approaches, but also longer term transfer and problem-solving skills."

B. Worked Examples

A worked example = "the epitome of strongly guided instruction"
"Discovering the solution to a problem in an information-rich environment" = "the epitome of minimally guided discovery learning."

"The worked example effect"
- based on cognitive load theory
- occurs when learners perform worse than learners who study the worked problems

see Sweller & Cooper 1985, Cooper & Sweller 1987
- algebra problems

Why? Explained by cognitive load theory:
- solving a problm requirs a problem-solving search
- search must occur using STM
- problem-solving search is inefficient because its function is to find a solution, not to alter LTM
- problem-olving search can occur with no learning at all
- problem-solving serach "overburdens limited working memory resourcs to be used for activities that are unrelated to learning"

By contrast, studying a worked example reduced memory load
- because no search is required
- attention is directedx to learning "the essential relations between problem-solving moves"
"Students learn to recognize which moves are required for particular problems, the basis for the acquisition of problem-solving schemas"

Conditions under with the 'Worked Example Effect' is not obtainable:
1. when worked examples themselves produce a heaviy cognitive load
2. as learners' expertise increases - the "expertise reversal effect"

C. Process Worksheets

---> "description of the phases one should go through when solving a problem as well as hints or rules of thumb..."

- Nadolski, Kirschner, and van Merriënboer 2005, with law students

Research on Minimal Guidance

A. Experiential learning at Work - Kolb 1971, Kolb & Fry 1975
i. person carrys out action, sees effect
ii. understand those effects, to anticipate what would follow
iii. understand the general principle
--> they also hypothesized learning styles

Attempts to validate experiential learning and learning styles "appear not to have been completely successful"
- Iliff 1994 - correlations were low, effect sizes were weak
- Ruble and Stout 1993 - low test-retest reliability
- Roblyer 1996 and Perkins 1991 - guidance necessary

B. Individual Differences
- "apptitude treatment interaction" (ATI) studies
- "Much of this work provides a clear antecedent to the expertise reversal effect"
- ATI findings (Cronbach and Snow 1977)
: strong treatment benefited less able learners
- the now-recognized "scaffolding effect"
- "strong treatments" = "explicit organization of information and learning support were provided"

C. Knowing Less After Instruction
- Clark 1989 on ATI studies - lower aptitude students with unguided instruction got significantly lower scores on posttest
- "Failure to provide strong learning support for less experienced or less able students could actually produce a measurable loss of learning"
- "even worse", students "tend to like the experience"
- higher apptitude students "didn't suffer" from guidance
- "These (guided) strategies require explicit, attention-driven effort on the pafrt of learners and so tend not to be liked"

D. Science Learning from Unguided Instruction
- Klahr and Nigam - advantages of direct instruction in the sciences
- "There is a wealth of such evidence" - various studies cited
- "These reports were prepared, in part, because of the poor state of science and mathematics education in the United States"
(SD: this would be the same environment where teachers 'cheat' on constructivism and offer direct instriuction, hm?)
- tecahers have found discovery learning is successful "only when students have prerequiite knowledge and undergo some prior structured experiences."

E. Medical Problem-Based Research
"A lack of clarity about the different between learning a discipline and research in a discipline coupled with the priority afforded to unbiased observation in the best inductivist and empiricist tradition has led many educators to advocate a problem-based method..."
- meshed with ideas in the philosophy of science
- fit well with progressive learer-centered views
cawthron and Rowell 1978 - "It all seemed to fit. The logic of knowledge and the psychology of knowledge coalesced under the umbrella term discovery." - why should educators look further than the traditional inductivist and empirical explanation?
- 60 medical schools have adopted PBL in the last two decades
- best-known survy comparing PBL with conventional: Albanese and Mitchell 1993 - lower basic scienc exam scores, no difference in residency selections, more study hours each day - "Although PBL students receive better scores for their clinical performance, they also order significantly more unnecessary tests at a much higher cost per patient with less benefit"
- Colliver 2000 - "PBL studies show no statistical on the performance of medical students... during the first 2 years of medical school"
- PBL is not more effective, but more costly

Two major components of PBL:
a) explicit teaching of problem-solving strategies in the form of the HD-Method (Barrows & Tamblyn 1980)
b) teaching of basic content in the context of a specific case or instance

- proponents argue that students taught this way learn more meaningfully
- "it is assumed" that
i. because students are exposed to the problems from the beginning, they have more opportunity to practice those skills
ii. and that by explicitly applying the HD method they larn to analyze problms and search for explanations
- But: Patel and Groen - the HD method may not be the most appropriate way to solve problems

Patel, Groen, Norman: when you teach basic science in a clinical setting, students have difficulty separating it from the particular clinical problems into which it has been integrated (ie., students generalize badly) - this appears to persist after training

"Experts use schema-based pattern recognition to determine the cause of a patient's illness."
- Elstein 1994 - "knowledge organization and schema acquisition are more important for the development of expertise than the use of particular methods of problem solving."
- "cognitive research has shown that to achieve expertise in a domain, learners must acquire the necessary schemata..."

Arocha and Patel (1995) concluded that the negative results can be accounted for by the splitting of attention resources and the high working memory load on schema acquisition during problem solving"
- multiple hypothesis occupy working memory
- the failure based on use of HD reasoning

"The practice of a profession is not the same as learning to practice the profession."

(SD: you say that the learning fails because it uses the HD method... but hyou just say that the experts use schemas... in other words, the learning was DIFFERENT from the practise... AND THAT IS WHY IT FAILED!!!)


- after a half-century of advocacy of minimal guided learning, it appears there is no body of research supporting the technique

- insofar as there is any evidence, "it almost uniformly supports direct, strong instructional guidance rather than constructivist-based minimal guidance"

- not only is unguided instruction less effective, it may produce negative results

Origins for support found in post-Sputnik curriculum reforms
- educators shifted away from teaching a discipline as a body of knowledge
- toward the assumption that knowledge can best or only be learned through experience (that is based on the procedures of the discipline)
- has led to unguided practical or project work, rejection of instruction based "on the facts, laws, principls, and theories gthat make up a discipline's content"

However "it may be an error to assume that the pedagogical conten t of the learning experience is identical to the methods and procdesses (ie., the epistenology) of the discipline being studied..."

"... and a mistake to assume that instruction should exclusively focus on application."

"Why do outstanding scientists who demand rigorous proof for scientific assertions in their research continue to use... teaching methods that are not the most effective?" - handelsman, 2004

Agree with Mayer's 2004 recommendation that we "move educational reform efforts from the fuzzy and unproductive world of ideology..."



    looks like working replacement for the dead link at the top of the post, Stephen.


  2. Bruner says, "How one conceives of education, we have finally come to recognize, is a function of how one conceives of culture and its aims, professed and otherwise."

    The conceptualization of education implicit in the KSC article appears to me to be very very narrow.

    Bruner also says, "The teaching and learning of structure, rather than simply the mastery of facts and techniques, is at the center of the classic problem of transfer... If earlier learning is to render later learning easier, it must do so by providing a general picture in terms of which the relations between things encountered earlier and later are made as clear as possible."

    The implications of this single sentence, most notably that there is a role for "teaching," appears to elude the summary if not the understanding presented by KSC.

    These two quotes illustrate that K,S, & C have deeply misunderstood Bruner. Similarly, they also fail to adequately and fairly summarize the others they have conflated into a single category and then further reduced into their category of "minimal guidance." In simple terms, "strategic guidance" or "contextualized guidance" might have been more appropriate characterizations of some aspects of the theorists and researchers they have clumped into straw men. Their characterization of learning is similarly constrained. Their review of the information processing literature they rely on is also reductive.

    I would venture to say that in their own learning, they have been , well, misguided.

    There is no real debate here to sink one's teeth into.

  3. Strawmen indeed! Perhaps the most contentious aspect is on the first page, where discovery learning, PBL, inquiry learning, experiential learning and constructivist learning are lumped gaily together under the term "minimally guided". Leaving aside the fact that constructivist learning, as they call it, is of a different order than the others (and they seem to have left out a few -!!- of the seminal constructivist authors, whose work they could address), the arguments for the similarity between these approaches are highly selective, and differences between them are conveniently ignored. It is questionable to argue that any of these approaches involves "minimal" guidance. (The conflation of "instruction" and "guidance" in their terminology is itself "interesting")
    Shulman is quoted, briefly, (elsewhere he emphasises the importance of avoiding an "overly technical image of teaching, a scientific exercise that has lost its soul" !!). The concept of pedagogic content knowledge is glossed over in order to allow it to support their thesis; more detailed discussion would reveal that it is arguably a concept that would support a diversity of pedagogical approaches. Such as for example the diversity these authors collapse into one in their second paragraph!!!

  4. Stephen, my thought is that constructivism and instructivism are different "world views" that will never be persuaded by each other's data.

    Each side perceives the other side as dodging the question.

    An example might be students exploring a museum freely and discovering their own learnings.

    If an instructivist said, "On a test at the end, those students did worse than students who were given direct instruction," a constructivist would be unimpressed.

    Conversely, if a constructivist said, "In a discussion at the end, those students' comments indicated appreciation, application, and synthesis," an instructivist would probably be skeptical that two independent observers would code the comments the same way.

    Tom Werner

  5. Tom, you are still buying into a definition of constructivism that is not accurate. KS&C have conflated informal learning with constructivism. There is no reason that the theory of constructivism cannot be applied to formal learning in a very rigorously "guided" and "tested" environment--though I think you are correct that a deeper understanding of the theory would recognize and respond to the extreme limits of formal testing as a useful way to measure learning. But a constructivist is also likely to advocate very mindful ways to structure a learner's experience cognizant of those learners' background knowledge.

  6. PBL is NOT an example of minimally guided instruction. Be sure to read Schmidt, Loyens, van Gog and Paas (2007) and Hmelo-Silver, Duncan and Chinn (2007) for a deconstruction of Kirschner et al.

  7. Gary, good point that I sloppily described constructivism as 'students wandering freely around a museum and creating their own learning.'

    But my point is that I do not think that a constructivist would be persuaded to drop constructivism and adopt instructivism if an experiment comparing the two approaches showed better test results for the instructivist approach.

    That's because these are philosophies or world-views.

    In practice a constructivist and an instructivist would design and test a visit to a museum differently because they fundamentally believe different things about learning.


Your comments will be moderated. Sorry, but it's not a nice world out there.