A PDF version of our grant proposal from 2002 (including a full research narrative and references) can be downloaded here: IDDEASproposal.pdf


Desirable Difficulties Quick Reference
This area should serve as a quick (and likely oversimplified) introduction or reference to three desirable difficulties we are working with.

Interleaving (a.k.a Contextual Interference) - Imagine two similar items, A and B, must be learned, and that the learner will study the items several times each. Prior research has shown the following pattern for simple materials learned in the laboratory:

Presentation Order Example Performance on Immediate Test (relative to alternate presentation order) Performance on Delayed Test (relative to alternate presentation order)
Blocked A,A,A,A,B,B,B,B better worse
Interleaved A,B,A,A,B,B,A,B worse better

- Imagine that a single item, A, must be learned, and that the learner will study the item several times. Prior research has shown the following pattern for simple materials learned in the laboratory:

Presentation Schedule Example Performance on Immediate Test (relative to alternate presentation schedule) Performance on Delayed Test (relative to alternate presentation schedule)
Massed A,A,A,A better worse
Spaced A,,,,,A,,,,,A,,,,,A worse better


Generation - Imagine that a learner must learn paired associates. For example, s/he would study "king:queen" and then would be shown just "king" on a test and would have to produce "queen." Prior research has shown the following pattern for simple materials learned in the laboratory:

Study Condition 1st Presentation (same in both conditions) 2nd Presentation Performance on Immediate Test (relative to alternate study condition) Performance on Delayed Test (relative to alternate study condition)
Read "king:queen" "king:queen" (learner simply reads the pair again) better worse
Generate "king:queen" "king:q____" (learner must generate "queen" on own) worse better

National Significance
Today, in this nation, we have both the will and the opportunity to upgrade education significantly. The will stems from a national consensus, and a constructive response by government agencies to that consensus. Political and business leaders, concerned parents, and typical citizens agree that America needs to improve education and enhance student achievement. Our educational system has significant shortcomings as shown in results from cross-national comparisons (e.g., TIMSS, 1998; Stigler & Heibert, 1999) and by studies of workplace competence (e. g., SCANS, 1991). Performance in mathematics and science and among students diagnosed as having disabilities has caused widespread alarm. There is also broad agreement that education is the future—not only for our children, but also for our nation as a whole.

The opportunity to upgrade education comes from extensive progress in the last several decades on understanding the cognitive processes that underlie learning. Basic research on learning and memory now provides a foundation for improving educational practices, potentially in revolutionary ways. Recent research, for example, questions the common view that student performance during instruction indexes learning and validly distinguishes among instructional practices. Work by Bjork and other researchers has established that conditions of practice that appear optimal during instruction can fail to support long-term retention and transfer of knowledge; whereas, and remarkably, conditions that introduce difficulties for the learner—slowing the apparent rate of the learning—can enhance long-term retention and transfer. (Such "desirable difficulties" (Bjork, 1994, 1999) include spacing rather than massing study sessions; interleaving rather than blocking practice on separate topics or tasks; varying how instructional materials are presented or illustrated; reducing feedback; and using tests rather than presentations as learning events.)

The opportunity to upgrade instruction also benefits from technology-enhanced learning environments that enable researchers to test the impact of these new research findings by consistently varying the conditions of instruction (e.g. Anderson, et al., 1996; 1997).

Enabled by the OERI Cognition and Student Learning program, we propose Introducing Desirable Difficulties for Educational Applications in Science (IDDEAS) to build bridges from the science of cognition to educational practices. IDDEAS will identify the laboratory-based principles and phenomena that do and do not generalize to educational settings and test mechanisms for implementing the principles and phenomena that do generalize in actual classrooms using technology-based instruction. IDDEAS requires a partnership of collaborating cognitive researchers, educational researchers, and classroom teachers who jointly design and carry out experiments in progressively more complex educational settings. We propose to form a sustainable partnership that can build a strong bridge linking the science of cognition, effective classroom practices, and powerful learning technologies. If successful, IDDEAS will develop theory-based principles to guide future instructional designers working in new contexts.


Students’ performance during instruction is commonly viewed as a measure of learning and a basis for evaluating and selecting instructional practices. Basic-research findings question that view: Conditions of practice that appear optimal during instruction can fail to support long-term retention and transfer of knowledge and, remarkably, conditions that introduce difficulties for the learner—and appear to slow the rate of the learning—can enhance long-term retention and transfer. Such "desirable difficulties" (Bjork, 1994, 1999) include spacing rather than massing study sessions; interleaving rather than blocking practice on separate topics; varying how to-be-learned material is presented; reducing feedback; and using tests as learning events.

The benefits of desirable difficulties found using simple laboratory tasks and short retention intervals not only raise concerns about prevailing educational practices, but also suggest unintuitive ways to enhance instruction. The present study focuses on whether such results generalize to realistic educational materials and contexts. In controlled experiments involving middle-school and college students, the effectiveness of standard Web-Based Inquiry Science Environment (WISE, http://wise.berkeley.edu) projects—on topics such as light propagation, thermal equilibrium, and science and treatment of malaria—will be contrasted with the effectiveness of experimental versions that incorporate selected desirable difficulties.

As a tool for teachers and students, WISE projects can enhance science education. If successful, this investigation can bridge the science of cognition and education and provide theoretically based principles that designers can use to create new materials.


Relevant References

Anderson, M.C., Bjork, R.A., & Bjork, E.L. (1994). Remembering can cause forgetting: Retrieval dynamics in long-term memory. Journal of Experimental Psychology: Learning, Memory, and Cognition, 20, 1063-1087.

Anderson, M. C., & Spellman, B. A. (1995). On the status of inhibitory mechanisms in cognition: Memory retrieval as a model case. Psychological Review, 102, 68-100.

Baddeley, A.D., & Longman, D.J.A. (1978). The influence of length and frequency of training session on the rate of learning to type. Ergonomics, 21, 627-635.

Bahrick, H.P. (1979). Maintenance of knowledge: Questions about memory we forgot to ask. Journal of Experimental Psychology: General, 108, 296-308.

Battig, W.F. (1956). Facilitation and interference. In E.A. Bilodeau (Ed.), Acquisition of skill (pp. 215-244). New York: Academic Press.

Battig, W. F. (1979). The flexibility of human memory. In L. S. Cermak & F.I.M. Craik (Eds.), Levels of processing in human memory (pp. 23-44). Hillsdale, N. J.: Erlbaum.

Becker, Henry Jay. (February 1999). "Internet Use by Teachers: Conditions of Professional Use and Teacher-Directed Student Use." Center for Research on Information Technology and Organizations, The University of California, Irvine and University of Minnesota.

Becker, H., Ravitz, J. & Wong, Y. (1999). Teacher and Teacher-directed Student Use of Computers and Software. Report #3. Teaching Learning and Computing: 1998 National Survey. Irvine, CA: University of California at Irvine, Center for Research in Information Technology in Organization.

Bjork, R.A. (1994). Memory and metamemory considerations in the training of human beings. In J. Metcalfe and A. Shimamura (Eds.), Metacognition: Knowing about knowing. (pp.185-205). Cambridge, MA: MIT Press.

Bjork, R. A. (1999). Assessing our own competence: Heuristics and illusions. In D. Gopher and A. Koriat (Eds.), Attention and peformance XVII. Cognitive regulation of performance: Interaction of theory and application (pp. 435-459). Cambridge, MA: MIT Press.

Bransford, J. D., Brown, A. L., & Cocking, R. R. (Eds.). (1999). How people learn: Brain, Mind, Experience, and School. Washington, DC: National Research Council

Bransford, J.D., Franks, J.J., Morris, C.D., & Stein, B.S. (1979). Some general constraints on learning and memory research. In L.S. Cermack & F.I.M. Craik (Eds.), Levels of processing in human memory (pp. 331-354). Hillsdale, NJ: Erlbaum.

Bransford, J.D., Sherwood, R.D., Hasselbring, T.S., Kinzer, C.K. & Williams, S.M. (1990). Anchored instruction: Why we need it and how technology can help. In Nix, D. & Spiro, R. (Eds.), Cognition, education, and multimedia. Hillsdale: Erlbaum.

Carson, L. M., & Wiegand, R. L. (1979). Motor schema formation and retention in young children: A test of Schmidt's schema theory. Journal of Motor Behavior, 11, 247-251.

Catalano, J.F., & Kleiner, B.M. (1984). Distant transfer in coincident timing as a function of variability of practice. Perceptual & Motor Skills, 58, 851-856.

Christina, R. W., & Bjork, R. A. (1991). Optimizing long-term retention and transfer. In D. Druckman & R. A. Bjork (Eds.), In the mind’s eye: Enhancing human performance (pp. 23-56). Washington, DC: National Academy Press.

Davis, E.A. (1998). Scaffolding students’ reflection for science learning. Unpublished doctoral dissertation, University of California at Berkeley, CA.

Dempster, F. N. (1990). The spacing effect: A case study in the failure to apply the results of psychological research. American Psychologist, 43, 627-634.

Dempster, F. N. (1996). Distributing and managing the conditions of encoding and practice. In E. L. Bjork and R. A. Bjork (Eds.), Memory: Vol. 10. Handbook of Perception and Cognition. (pp. 317-344). New York: Academic Press.

diSessa, A. (2000). Changing minds: Computers, learning, and literacy. Cambridge, MA, MIT Press.

Driver, R. (1985). Changing perspectives on science lessons. In N. Bennett, & C. Desforges (Eds.), Recent advances in classroom research. British Journal of Psychology Monograph.

Driver, R., Leach, J., Millar, R., & Scott, P. (1996). Young People's Images of Science. Buckingham, UK: Open University Press.

Edelson, D. C., Gordin, D. N., & Pea, R. D. (1999). Addressing the challenges of inquiry-based learning through technology and curriculum design. Journal of the Learning Sciences, 8(3/4), 391-450.

Eylon, B. S. and M. C. Linn (1988). Learning and instruction: An examination of four research perspectives in science education. Review of Educational Research 58(3): 251–301.

Farr, M. J. (1987). The Long-term Retention of Knowledge and Skills: A        Cognitive and Instructional Perspective. New York: Springer-Verlag.

Feurzeig, W., & Roberts, N. (Eds.). (1999). Modeling and simulation in science and mathematics education. New York: Springer.

Gates, A. I. (1917). Archives of Psychology #40

Ghodsian, D., Bjork, R. A., & Benjamin, A. S. (1997). Evaluating training during training: Obstacles and Opportunities. In M. A. Quinones and A. Dutta (Eds.), Training for a rapidly changing workplace: Applications of Psychological Research (pp. 63-88). Washington DC: American Psychological Association.

Gick, M. L., & Holyoak, K. J. (1980). Analogical problem solving. Cognitive Psychology, 12, 306-355.

Glenberg, A. M. (1992). Distributed practice effects. In L. R. Squire (Ed.), Encyclopedia of learning and memory (pp. 138-142). New York: Macmillan.

Hall, K. G., Domingues, E. & Cavazos, R. (1992). The effects of contextual interference on extra batting practice. Unpublished paper.

Hogan, R.M., & Kintsch, W. (1971). Differential effects of study and test trials on long-term recognition and recall. Journal of Verbal Learning and Verbal Behavior, 10, 562-567

Homa, D., & Cultice, J. (1984). Role of feedback, category size, and stimulus distortion on the acquisition and untilization of ill-defined categories. Journal of Experimental Psychology: Learning, Memory, and Cognition, 10, 83-94.

Inhelder, B. & Piaget, J. (1958/1972). The growth of logical thinking from childhood to adolescence. New York: Basic Books.Izawa, 1970).

Izawa, C. (1970.) Optimal potentiating effects and forgetting-prevention effects of tests in paired-associate learning. Journal of Experimental Psychology, 83, 340-344.

Jacoby, L. L., Bjork, R. A., & Kelley, C. M. (1994). Illusions of comprehensions and competence. In D. Druckman and R. A. Bjork (Eds.), Learning, remembering, believing: Enhancing human performance. (pp.57-80) Washington, DC: National Academy Press.

Kafai, Y. B., & Resnick, M. (Eds.). (1996). Constructionism in practice: Designing, thinking, and learning in a digital world. Hillsdale, NJ: Lawrence Erlbaum Associates.

Kerr, R., & Booth, B. (1978). Specific and varied practice of a motor skill. Perceptual and Motor Skills, 46, 395-401.

Landauer, T. K., & Bjork, R. A. (1978). Optimal rehearsal patterns and name learning. In M. M. Gruneberg, P. E. Morris, & R. N. Sykes (Eds.), Practical aspects of memory pp. 625-632). London: Academic Press.

Langley, D. J., & Zelaznik, H. N. (1984). The acquisition of time properties associated with a sequential motor skill. Journal of Motor Behavior, 16, 275-301.

Lee, T. D., & Genovese, E. D. (1988). Distribution of practice in motor skill acquisition: Learning and performance effects reconsidered. Research Quarterly for Exercise and Sport, 59, 277-287.

Linn, M. C. (1992). The Computer as Learning Partner: Can Computer Tools Teach Science? In K. Sheingold, L. G. Roberts, and S. M. Malcolm (Eds.), Technology for Teaching and Learning. Washington, DC: American Association for the Advancement of Science.

Linn, M. C. (in press). WISE design for lifelong learning—Pivotal Cases. In GĢrdenfors, P. & Johansson, P. (Eds.), Cognition, Education and Communication Technology. Mahwah, New Jersey, Lawrence Erlbaum Associates.

Linn, M. C., & Hsi, S. (2000). Computers, Teachers, Peers: Science Learning Partners. Mahwah: Erlbaum.

Linn, M. C. (2000). "Designing the Knowledge Integration Environment: The partnership inquiry process." International Journal of Science Education, Special Issue (22): 781-796.

Linn, M.C. & Songer, N.B. (1991). Teaching thermodynamics to middle school students: What are appropriate cognitive demands? Journal of Research in Science Teaching, 28, 885-918.

Mannes S.M., & Kintsch, W. (1987). Knowledge organization and text organization. Cognition and Instruction, 4, 91-115.

Morris, C.D., Bransford, J.D., & Franks, J.J. (1977). Levels of processing versus transfer appropriate processing. Journal of Verbal Learning and Verbal Behavior, 16, 519-533.

National Research Countil (1999). High Stakes: Testing for Tracking, Promotion and Graduation. Washington, D.C.

Pea, R. D. & Gomez, L. (1993). Distributed multimedia learning environments: The Collaborative Visualization Project. Communications of the ACM, 36(5), 60-63.

Piaget, J. (1971). Structuralism. New York: Harper and Row.

Rea, C.P., & Modigliani, V. (1985). The effect of expanded versus massed practice on the retention of multiplication facts and spelling lists. Human Learning, 4, 11-18.

Reder, L. M., Charney, D. H. & Morgan, K. I. (1986). The role of elaborations in learning a skill from an instructional text. Memory and Cognition, 14, 64-78.

Reder, L. M., & Klatzky, R. L. (1994). Transfer: Training for performance. In D. Druckman and R. A. Bjork (Eds.), Learning, remembering, believing:        Enhancing human performance (pp. 25-56). Washington, DC: National Academy Press.

Reiser, B.J., Kimberg, D.Y., Lovett, M.C., & Ranney, M. (1992). Knowledge representation and explanation in GIL, an intelligent tutor for programming. In J.H. Larkin & R.W. Chabay, Computer-assisted instruction and intelligent tutoring systems: Shared goals and complimentary approaches. Hillsdale: Erlbaum.

Scardamalia, M., & Bereiter, C. (1991). Higher levels of agency for children in knowledge building: A challenge for the design of new knowledge media. Journal of the Learning Sciences, 1(1), 37-68.

Schmidt, R.A. (1991). Frequent augmented feedback can degrade learning: Evidence and interpretations. In G.E. Stelmach & J. Requin (Eds.), Tutorials in motor neuroscience (pp. 59-75). Dordrecht: Kluwer.

Schmidt, R. A., & Bjork, R. A. (1992). New conceptualizations of practice: Common principles in three paradigms suggest new concepts for training. Psychological Science, 3, 207-217.

Schmidt, R.A., Young, D.E., Swinnen, S., & Shapiro, D.C. (1989). Summary knowledge of results for skill acquisition: Support for the guidance hypothesis. Journal of Experimental Psychology: Learning, Memory, and Cognition, 15, 352-359.

Shea, J.B., & Morgan, R.L. (1979) Contextual interference effects on the acquisition, retention, and transfer of a motor skill. Journal of Experimental Psychology: Human Learning and Memory, 5, 179-187.

Simon, D. A., & Bjork, R. A. (2001). Metacognition in motor learning. Journal of Experimental Psychology: Learning, Memory, and Cognition, 27, 907-912.

Slotta, J.D., Chi, M.T.H. & Joram, E. (1995). Assessing the ontological nature of conceptual physics: A contrast of experts and novices. Cognition and Instruction.

Smith, S. M, Glenberg, A. M., & Bjork, R. A. (1978). Environmental context and human memory. Memory & Cognition, 6, 342-353. (

Smith, S. M., & Rothkopf, E. Z. (1984). Contextual enrichment and distribution of practice in the classroom. Cognition and Instruction, 1(3), 341-358.

Songer, N. B. (1996). Exploring learning opportunities in coordinated network-enhanced classrooms: A case of kids as global scientists. The Journal of the Learning Sciences, 5, 297-327.

Slotta, J. & Linn, M.C. (2000). The Knowledge Integration Environment: Helping students use the Internet effectively. In M.J. Jacobson & R.Kozma (Eds.), Innovations in Science and Mathematics Education: Advanced Designs for Technologies of Learning (pp. 193-226). Mahwah: Erlbaum.

Vanderbilt, Cognition and Technology Group. (1997). The Jasper Project: Lessons in curriculum, instruction, assessment, and professional development: Lawrence Erlbaum Associates.

Vygotsky, L. S. (1962). Thought and language. Cambridge, MA, MIT Press.

White, B.Y. & Frederiksen, J.R. (1998). Inquiry, modeling, and metacognition: Making science accessible to students. Cognition and Instruction, 16, 3-118.

Winstein, C.J., & Schmidt, R.A. (1990). Reduced frequency of knowledge of results enhances motor skill learning. Journal of Experimental Psychology: Learning, Memory, and Cognition, 16, 677-691.