Here we see students doing what all biology educators who use multiple-choice assessment fear, scanning the options for one that looks right based on limited knowledge. As researchers have discovered, lower-order problems, not higher-order problems, are the type most often found in college biology courses Momsen et al.
Second, it is striking that domain-specific procedures are more prevalent among higher-order problems than lower-order problems. These data suggest that higher-order problems promote strong content usage by students. As others have argued, higher-order problems should be used in class and on exams more frequently Crowe et al.
Although it is interesting in and of itself to learn the procedures used by students during multiple-choice assessment, the description of these categories of procedures begs the question: does the type of procedure used by students make any difference in their ability to choose a correct answer?
As explained in the Introduction , the strongest problem-solving approaches stem from a relatively complete and well-organized knowledge base within a domain Chase and Simon, ; Chi et al. Thus, we hypothesized that use of domain-specific procedures would be associated with solving problems correctly, but use of domain-general procedures would not.
Indeed, our data support this hypothesis. While limited use of domain-general procedures was associated with improved probability of success in solving multiple-choice problems, students who practiced extensive domain-specific procedures almost guaranteed themselves success in multiple-choice problem solving.
In addition, as students used more domain-specific procedures, there was a weak but positive increase in the course performance, while use of domain-general procedures showed no correlation to performance.
These data reiterate the conclusions of prior research that successful problem solvers connect information provided within the problem to their relatively strong domain-specific knowledge Smith and Good, ; Pressley et al.
In contrast, unsuccessful problem solvers heavily depend on relatively weak domain-specific knowledge Smith and Good, ; Smith, General problem-solving procedures can be used to make some progress in reaching a solution to domain-specific problems, but a problem solver can get only so far with this type of thinking. In solving domain-specific problems, at some point, the solver has to understand the particulars of a domain to reach a legitimate solution reviewed in Pressley et al.
Likewise, problem solvers who misunderstand key conceptual pieces or cannot identify the deep, salient features of a problem will generate inadequate, incomplete, or faulty solutions Chi et al. Our findings strengthen the conclusions of previous work in two important ways. First, we studied problems from a wider range of biology topics.
Second, we studied a larger population of students, which allowed us to use both qualitative and quantitative methods. Think-aloud protocols typically take place in an interview setting in which students verbally articulate their thought processes while solving a problem.
When students are silent, the interviewer is there to prompt them to continue thinking aloud. We modified this protocol and taught students how to write out their procedures. However, one limitation of this study and all think-aloud studies is that it is not possible to analyze what students may have been thinking but did not state. Despite this limitation, we were able to identify a range of problem-solving procedures and errors that inform teaching and learning.
However, problem solving is not intuitive to students, and these skills typically are not explicitly taught in the classroom Nehm, ; Hoskinson et al.
One reason for this misalignment between faculty values and their teaching practice is that biology problem-solving procedures have not been clearly defined. Our research presents a categorization of problem-solving procedures that faculty can use in their teaching. Instructors can use these well-defined problem-solving procedures to help students manage their knowledge of biology; students can be taught when and how to apply knowledge and how to restructure it. This gives students the tools to become more independent problem solvers Nehm, We envision at least three ways that faculty can encourage students to become independent problem solvers.
First, faculty can model the use of problem-solving procedures described in this paper and have students write out their procedures, which makes them explicit to both the students and instructor. Second, models should focus on domain-specific procedures, because these steps improve performance. Explicit modeling of domain-specific procedures would be eye-opening for students, who tend to think that studying for recognition is sufficient, particularly for multiple-choice assessment.
However, our data and those of other researchers Stanger-Hall, suggest that studying for and working through problems using strong domain-specific knowledge can improve performance, even on multiple-choice tests. Third, faculty should shift from the current predominant use of lower-order problems Momsen et al. Our data show that lower-order problems prompt for domain-general problem solving, while higher-order problems prompt for domain-specific problem solving.
We took what we learned from the investigation reported here and applied it to develop an online tutorial called SOLVEIT for undergraduate biology students Kim et al. The problems focus on species concepts and ecological relationships. In brief, SOLVEIT asks students to provide an initial solution to each problem, and then it guides students through the problem in a step-by-step manner that encourages them to practice several of the problem-solving procedures reported here, such as Recalling, Checking, Analyzing Visual Representations, and Correcting.
Thus, research to uncover the difficulties of students during problem solving can be directly applied to improve student learning. We also thank the Biology Education Research Group at UGA, who improved the quality of this work with critical feedback on the manuscript. Finally, we thank the reviewers, whose feedback greatly improved the manuscript. Prevost and P. This article is distributed by The American Society for Cell Biology under license from the author s. It is available to the public under an Attribution—Noncommercial—Share Alike 3.
Luanna B. Paula P. Add to favorites Download Citations Track Citations. View article. Abstract This study uses the theoretical framework of domain-specific problem solving to explore the procedures students use to solve multiple-choice problems about biology concepts. TABLE 1. Clarifying a Restated or paraphrased the problem stem or one of the multiple-choice options.
Comparing Language of Options a Detected similarities and differences in the language of two multiple-choice options. Correcting b Pointed out that they had been thinking incorrectly about the problem earlier in the written think-aloud and now see the correct way to think about the problem. Delaying b Considered one of the multiple-choice options and decided that it should not be eliminated.
Rather, the quality of that option should be evaluated later, after the other multiple-choice options are considered. Hybrid procedures Comparing Correctness of Options a Detected similarities and differences in two multiple-choice options, often based on a superficial evaluation of the content of the options e.
Recognizing a Noted that a multiple-choice option is correct or incorrect without any rationale. Domain-specific procedures Adding Information b Provided more information about one of the multiple-choice options, such as additional facts that were omitted or corrections to incorrect statements i. Analyzing Domain-Specific Visual Representation a,b For a visual representation that is unique to biology e.
Asking a Question c Asked a question about the problem stem or multiple-choice options. Checking a Explained why an option is correct or incorrect by comparing the option with their knowledge or with the data provided in the problem.
Predicting a,c As an early step in the written think-aloud, predicted what they expected the answer to be i. Recalling a Retrieved basic facts or concepts from class, notes, or the textbook i.
C says that are they same based on the biological species concept. The data that proves there are hybrids proves this to be true. I mark it. E could also make sense but I think there is enough information to make a decision. Disregarding Evidence Did not use some or all of the data provided in the problem. Incorrect answer—the data does not represent morphological characteristics, so cannot conclude this answer.
Move on. Misreading Read the question prompt or answer options incorrectly B is incorrect because Atlantic eels should show some resistance since the Atlantic eel have developed in the presence of krait toxin.
Opinion-Based Judgment Gave an opinion and did not use biology content knowledge. E may be right, but I feel confident with C.
Domain-specific errors Making Incorrect Assumptions Stated that the graph or other visual representation provides no useful information. Examine graph. Hybrids are not seeming to live not viable. Misunderstanding Content Showed incorrect understanding of content knowledge. Frequency of each problem-solving procedure for lower-order and higher-order problems Procedures are presented from left to right in alphabetical order.
Frequency of errors for lower-order and higher-order problems Categories of errors are presented from left to right in alphabetical order. The interaction of domain-specific and strategic knowledge and academic performance.
Rev Educ Res 58 , Domain-specific and strategic knowledge as predictors of expository text comprehension. J Reading Behav 23 , Learning from physics text: a synthesis of recent research.
J Res Sci Teach 31 , J Res Sci Teach 27 , Google Scholar Belland BR Portraits of middle school students constructing evidence-based arguments during problem-based learning: the impact of computer-based scaffolds. Educ Technol Res Dev 58 , A new method for assessing critical thinking in the classroom. BioScience 56 , Google Scholar Bloom BS R Glaser , Knowing when, where, and how to remember: a problem of metacognition In: Advances in Instructional Psychology, vol.
How students think about experimental design: novel conceptions revealed by in-class activities. BioScience 64 , Enhancing chemistry problem-solving achievement using problem categorization. J Res Sci Teach 28 , Non-mathematical problem solving in organic chemistry. J Res Sci Teach 47 , Categorization and representation of physics problems by experts and novices. Cogn Sci 5 , Understanding Cognitive Science, 1st ed. On problem-solving.
Psychol Monogr 58 , i Protocol Analysis: Verbal Reports as Data, rev. Applied Longitudinal Analysis, 2nd ed. BioScience 61 , How can we improve problem solving in undergraduate biology? Applying lessons from 30 years of physics education research. Educ Psychol 22 , Google Scholar Jonassen D Google Scholar Jonassen DH Toward a design theory of problem solving.
Educ Technol Res Dev 48 , Google Scholar Keys CW Investigating the thinking processes of eighth grade writers during the composition of a scientific laboratory report. J Res Sci Teach 37 , J Comput Assist Learn 31 , Patterns of multiple representation use by experts and novices during physics problem solving. Google Scholar Lavoie DR The development, theory, and application of a cognitive-network model of prediction problem solving in biology.
J Res Sci Teach 30 , Supporting learning of variable control in a computer-based biology environment: effects of prompting college students to reflect on their own thinking. J Res Sci Teach 36 , Google Scholar Martinez ME What is problem solving. Phi Delta Kappan 79 , Using assessments to investigate and compare the nature of learning in undergraduate science courses.
Just the facts? Introductory undergraduate biology courses focus on low-level cognitive skills. Google Scholar Nehm RH J Microbiol Biol Educ 11 , BioScience 57 , Evol Educ Outreach 4 , Elements of a theory of human problem solving. Psychol Rev 65 , Google Scholar Patton MQ Google Scholar Polya G Cognitive strategies: good strategy users coordinate metacognition and knowledge.
Ann Child Dev 4 , The challenges of classroom strategy instruction. Elem Sch J 89 , Cognition and creativity. Educ Psychol Rev 7 , Metacognitive theories. Developing problem-solving skills of students taking introductory physics via Web-based tutorials.
J Coll Sci Teach 39 , Development of the biology card sorting task to measure conceptual expertise in biology. Successful and unsuccessful problem solving in classical genetic pedigrees. Other co-authors are Michael T. The pink bollworm, a global pest of cotton, has evolved resistance to genetically modified cotton in India, but not in Arizona, where farmers have planted refuges of conventional cotton to reduce selection for resistance.
Study: evolutionary biology must be used to overcome global agriculture challenges. Findings from the study will appear Sept. Similar examples abound in other domains of biology Table 1B and in physics Table 1C. Alternatively, consider the kinds of problems that scientifically literate citizens should be able to solve Table 1, D—F.
In these problems and many others, the understanding of the system is vague or may not be shared among the problem solvers. This can lead to multiple, potentially competing ideas of what constitutes a solution or answer Nicolson et al.
The representations of both problem elements and relationships among the elements are poor Jonassen, ; Fischer and Greiff, As the number of elements and relationships increases, their interdependence makes simpler problem representations and paper-and-pencil solutions by single individuals less feasible.
Consequently, cooperation among multiple people—population ecologists, environmental chemists, economists, politicians, and citizens—may be required to adequately represent and solve the problem Table 1, D and E. The need for communication among experts and nonexperts may also be featured Table 1, D and E. To solve these authentic, complex problems, participants engage in a wider variety of scientific practices than when solving simple exercises. These kinds of problems are infrequently presented in most of undergraduate biology curricula; thus, there remains a significant gap between the kind of problems students can solve and the kind of problems they should be able to solve.
Over the past 30 years, physics education researchers have investigated how students learn to solve problems in physics courses through a variety of perspectives McDermott and Redish, ; Hsu et al.
This work has lead to a substantial number of research-based curricular and pedagogical tools that improve student learning in physics Meltzer and Thornton, Despite significant strides made by PER, many researchers and instructors are just now beginning to make use of authentic, complex problems. There is still much work to be done in PER to describe what authentic and complex physics problems look like, how students engage in scientific practices when solving such problems, and how students who learn to solve authentic and complex physics problems perform on more traditional assessments of problem solving.
Early work in PER highlighted the substantial differences between how physics students and experts think about the nature of science, including how their knowledge of physics principles is organized and used and what practices they employ to solve physics problems Chi et al.
Representative problems were typical exercises. This research illuminated how students could successfully solve exercises without conceptual knowledge of the underlying physics.
Unlike novice students, physics experts rarely begin solving problems by using mathematical equations. Just as in the biological sciences, physics experts often think of a few governing principles and heuristics and then construct models to make sense of physical phenomena. Often, they begin with their conceptual knowledge of the problem and then develop it to include mathematical representations.
Further refinement and mathematical manipulations lead to appropriate expressions of the problem Reif, Novices, on the other hand, often think of physics as a loose collection of ideas and equations with few or no connections among them Chi et al.
When solving exercises, students rarely employ their conceptual knowledge, preferring instead to hunt for equations that contain all the elements e. Traditional exams tend to reward finding a single solution to an exercise answer making rather than demonstrating a deep understanding of principles and methods sense making; McDermott and Redish, ; Meltzer and Thornton, Early work in PER demonstrated that, in addition to applying conceptual knowledge to solve exercises, students need to learn to construct a variety of representations of physical systems, to coordinate between those representations, and to execute the necessary mathematics to successfully solve problems in physics.
While this early work did not seek to define the authenticity or complexity of problems per se, it did provide the necessary foundation for researchers and instructors to develop transformative teaching methods. By emphasizing a number of scientific practices that were not present in traditional lecture courses, teaching methods emphasized authentic problems.
For example, when a standard exercise is reframed as a design activity Table 1F , students must confront how a problem is defined, how a model can be constructed, and how variables can be reduced, so the problem can be solved using the elementary physics and mathematics they are learning. Building on the work of Chi, McDermott, Reif, and others, Heller and colleagues formalized the tasks needed for solving typical exercises into a coherent framework Heller and Heller, In one instantiation of this framework, Heller and colleagues developed a suite of authentic problems not typically represented in physics curricula.
While these problems were not necessarily complex they still resolved to a single solution , they were authentic context-rich in Heller and Heller [ ], or real-world scenarios, often with personal relevance. To solve these problems, students used a stepwise framework: 1 focus on the problem, 2 describe the physics, 3 plan the solution, 4 execute the plan, and 5 evaluate the answer.
This framework makes explicit use of conceptual knowledge in steps 1 and 2 and connects that knowledge directly to representing the problem mathematically in step 3 Heller and Heller, Explicit teaching of problem-solving practices in this way e.
The structure and content of such problems were not particularly authentic e. Students attended to multiple elements in a single scenario e. These tasks have been broadened to include a variety of alternative problem types and to engage students in more authentic scientific practice. Many of the newer activities require students to utilize their conceptual knowledge to explain their solutions to peers or to critique the solution offered by others Hieggelke et al. Others have leveraged alternative representations verbal descriptions, diagrams, graphs, and equations to focus novice problem solvers on performing a conceptual analysis by first coordinating between the different representations Van Heuvelen and Maloney, ; Van Heuvelen and Zou, More recently, PER has begun work at the upper-division undergraduate level, in which course goals and the activities pursuing those goals are significantly more complex e.
For typical upper-division problems, students often grapple with complex systems in addition to considering how multiple elements and their relationships facilitate a solution e.
Moreover, students in upper-division physics become acculturated to the scientific practices of professional physicists e. Despite the increasing complexity of these problems, many upper-division physics problems are still inauthentic idealized models of systems with limited personal relevance , and upper-division students struggle to solve these problems Pepper et al.
To uncover why students struggle, Caballero and colleagues developed a framework that has been used to analyze how students solve problems in upper-division physics courses Caballero et al.
Their work investigates how students blend their conceptual knowledge with problem-solving practices to achieve solutions. In physics, scientific practices include constructing and evaluating models, designing and executing experiments, and engaging in argumentation based on evidence.
Scientific practices underpin what it means to engage in complex problem solving; in fact, these are the practices that professional scientists use to solve challenging problems in their own work NRC, a. Through this lens, recent reforms in introductory physics have broadened the traditional definition of problem solving to include engaging in the practices of professional science.
Consider again the design problem posed in Table 1F ; such a problem engages students in the practice of science and thus can be characterized as an authentic and complex problem.
There are a number of curricular examples that emphasize scientific practices, and, hence, this broader definition of problem solving, such as Workshop Physics Laws, ; Etkina and Van Heuvelen, Modeling Instruction Hestenes, is another approach gaining wider acceptance both in PER and the broader physics community.
Modeling Instruction is worth describing in some detail because it has been implemented in both high school Hestenes et al. Modeling Instruction uses a theoretical framework the modeling cycle around which student activities are organized. Students engage in open-ended experimental and theoretical procedures while making real-world observations and then propose possible measurements that can help describe observed patterns.
From these measurements, students observe trends and patterns that help to inform their development of representative models. Students subsequently repeat this observation—development—evaluation cycle for new phenomena. Through this cycle, students discover the necessary elements and their relationships that describe the observed system. While the physics and mathematics are not particularly complex, students utilize several scientific practices during a single cycle: developing and using models, finding patterns, testing hypotheses.
Physics courses that emphasize scientific practices will likely serve students well in their future course work and beyond. While many teaching methods have been developed by the PER community, effectiveness is often evaluated using end-of-course conceptual assessments. Concept inventories assessments measure whether students can show evidence of deeper learning from particular instructional strategies Hestenes et al.
Through measuring student-learning gains, these assessments have demonstrated the benefits of using active-engagement scientific practices in teaching Hake, ; Hestenes, ; Pollock and Finkelstein, Concept inventories have also been used to quantify the outcomes of introductory and upper-division physics courses Kohlmyer et al. Despite their value, most physics concept inventories do not directly measure problem solving—even as narrowly defined.
Thus, they may or may not serve well as predictors of such skills in students. However, this instrument cannot directly evaluate authentic scientific practice skills, because the MBT's multiple-choice format is not amenable to investigating how students employ scientific practices.
Future work in PER is needed to assess how students employ scientific practices. Theoretically, biologists should be able to apply the approaches that physics education researchers have found successful in improving instruction and student learning, especially because some of the same scientific practices have already been identified as critical in biology e.
However, biology education researchers are still working to define what it looks like for students to solve complex problems in biology. We focus below on ways in which the findings on problem solving in PER can specifically impact curriculum development and research on problem solving in biology.
New curricula for biology students should include ways for students to use scientific practices and process skills and to engage in complex problem solving, as described above. Currently, many problems that biology students are asked to solve are exercises.
Consider an exercise that a biology student might encounter in an introductory genetics course: calculating inheritance probabilities of sex-linked traits. A problem on this topic could be presented as follows: Suppose that hemophilia is an X-linked recessive trait.
If a mother is a carrier for hemophilia, and the father is normal, what is the chance that their son will have hemophilia? To make the problem more complex and engage students in more authentic problem-solving practices, the prompt can emphasize the construction design of a possible pedigree, such as: Generate a possible pedigree for three generations showing unaffected, affected, and carrier individuals in a pedigree for hemophilia.
Share your pedigree with your neighbor. How are the two pedigrees different? Which is more likely to occur, given the history of hemophilia? This approach as well as the other examples included in Table 1 transforms a typical exercise into a complex problem, inviting students to generate possible scenarios by applying their knowledge of a generalized system chromosomal inheritance to a specific application.
Such problems could also be presented to students as in-class concept clicker questions. In the peer instruction model originally proposed by Mazur , multiple-choice in-class questions can be used to foster discussion among students, engaging them in a community of problem solvers. Clickers work well for implementing this active-learning technique in both biology and physics, because they are easy to incorporate, especially in large classes, and they can provide immediate feedback to both students and instructor Wood, Some may view multiple-choice clicker questions as limited, because the potential answers are defined; however, it is possible to write multiple-choice questions that require higher-order thinking and that require students to engage in complex problem solving Crowe et al.
Curricula that encourage the use of complex problem solving in class, as practices or as part of formative assessment, have the potential to foster complex problem-solving abilities in students DeHaan, ; Maskiewicz et al. The observation—development—evaluation cycle of Modeling Instruction could also be adapted to biological problems and implemented in a variety of classroom settings.
Some biology resources already exist that could be adapted for this purpose, such as case studies National Center for Case Study Teaching in Science, ; Ecological Society of America, and problem-based scenarios Norman and Schmidt, Then, students develop experiments to test hypotheses, to explore the sensitivity of a model to changes in variables or parameters, or to suggest and justify what further measurements they would make authentic scientific practice.
The final step in case-based or problem-based curricula is to evaluate evidence against observations and to construct arguments for a resolution to a problem or dilemma. This kind of curricular path, then, builds in higher-order process skills that support student problem solving.
Following the work of PER scholars who introduced the importance of measuring conceptual understanding and sense-making with concept inventories e. Not all concept inventories have been developed with attention to evidence of validity and reliability Huffman and Heller, , and few directly measure problem-solving or critical-thinking skills Smith and Tanner, , as also described above.
In addition, several recently developed assessments are devoted at least in part to specifically measuring problem solving i. Student performance on these kinds of tools, coupled with the results of formative assessments devoted to problem solving, may be useful in further informing curricular change.
Though gaps between theory and practice remain, these gaps also suggest rich opportunities for research. For instance, as described in Undergraduate Biology and Complex Problems: A Course Work—Practice Gap , some BER scholars have begun to investigate how student conceptions of biological principles and processes differ from those of experts. One approach to solidifying our understanding of biological problem solving would be to ask what process skills novices tend to employ when problem solving, and compared those with skills that experts typically employ.
Similar investigations were helpful in framing the development of research-based instructional materials for introductory physics courses McDermott and Shaffer, An understanding of the processes that expert biologists use to solve problems could be used to help teach the scaffolding of both content knowledge and process skills to students.
Process skills may also be an important mechanism linking problem-solving activity and conceptual knowledge. Although there exist ways of measuring some aspects of problem solving, and other methods to assess conceptual knowledge, there have been few investigations of ways to measure conceptual knowledge using the processes of problem solving Nehm, DQCs Parker et al.
Work that explores the kinds of links between conceptual knowledge and sense making with actual problem solving competence should also focus on developing assessments that are validated and easy to deploy. One important difference between physics and biology lies in how problems represent systems behavior. Physics problems tend to emphasize quantitative representations. Qualitative, conceptual, and pictorial representations are used extensively in physics, but, ultimately, the goal of many problems is to connect the physics to the mathematics to make predictions.
Whether simple or complex, such problems require the problem solver to use quantitative representations, such as equations, graphs, or predictive models.
While some biological problems are quantitative, many others rely upon different representations, including diagrammatic or pictorial representations, such as those used to represent signaling pathways, biological cycles, or relationships among cycles i. Understanding and coordinating between representations is but one distinguishing feature and promising area of research into complex problem solving in biology.
Other process skills may be just as necessary and important for biology students to learn. Metacognition is widely recognized as important for learning Tanner, , and it may be especially important in CPS for progress checking and for reconciling feedback among potential solutions, representations, and mental models. Decision making as a process skill may also be fundamental to CPS Nicolson et al.
Although we have identified several structures and processes common to complex problem solving, this is by no means a comprehensive list or an operational model for designing problems or assessing problem solving. The development of operational models of CPS processes would help target the behaviors and skills necessary for students to engage in solving authentic, complex problems in the classroom and in their lives.
We live at a time when complex biological problems are not just the realm or responsibility of highly trained scientists. The people who are currently our students will need to engage along with us as citizens once they leave college.
0コメント