"Problem-based Learning: helping your students gain the most from PBL" 3rd edition, March 1996

Instructor's Guide for "Problem-based Learning: how to gain the most from PBL" ISBN 0-9698725-0-X

© copyright 1996, Donald R. Woods

4. Issues about setting up small group, self-directed, self-assessed PBL

Table 4-1 compares conventional and PBL courses so that the some key issues raised by Albanese and Mitchell (1993) and Dolmans et al. (1993) can be brought into sharp focus.

Table 4-1 Some extreme characterizations of the conventional and PBL approaches.

Table 4-1 purposely paints a very black and white picture. In truth, I think both programs are somewhere between the extreme descriptors. However, I haveshown it this way so that we can try to create a PBL environment that best can minimize the difficulties listed.

Here are nine things to do before your first class.

1. Decide how to start.

2.Visualize the timing and the meetings.

3.Organize the student groups.

4. Create the resources.

5.Create the environment for learning the subject knowledge.

6.Create the environment for the process skills.

7. Create the environment to develop "expertise".

8.Assess the student's performance.

9.Evaluate the program's effectiveness.

Let's consider each in turn.

4.1 How to start

Start simply. Do a pilot project. Convert part of your course.

•Select the course, series of topics, subject content. Example, 4 "lectures" on process safety or 16 "lectures" on engineering economics.

•Convert these into desired objectives and outcomes for your students.

• List the fundamental subject concepts you want the students to learn. Don't be afraid to integrate subject material across disciplines.

•Pose the problem that will drive the learning. Look at typical exams you have used in the past. Select a set of a dozen good questions. Analyze these and see which issues each question asks the student to demonstrate knowledge about. Cross compare these with your list of outcomes generated above. Pose some trial questions and ask colleagues to list the issues that your question triggers. Check to see that all the key issues were included. If not, revise the problem statement. Example, for the 16 lectures, I listed about 27 issues or "learning objectives." I divided these into four different clusters, and these posed four different questions. The questions were short, similar to exam questions. Each "problem" then becomes the focus for the sequence of goals, teach, feedback meetings. Thus, the 16 lectures became 5 weeks of PBL addressing four "problems."

4.2 Visualize the timing and the "meetings"

The next issues are:

How much total amount of time should I allow?

How many different types of "meetings" to arrange?

What is the duration and what are the characteristics of each meeting?

• How much total time? From issue #1 in Table 4-1, experience with PBL suggests that it takes about 20% more time to "cover" the same amount of subject material. As a start, you might look at "covering 50% of the material" in the amount of time you used to spend lecturing. Why so drastic a reduction? Much of the contact time will be spent establishing and developing the process skills. As mentioned often, putting people in groups doesn't mean that immediately they become the perfect PBL process group. As groups develop their expertise, more subject content can be "covered."

• Visualize the types of meetings you plan. Some options are given in Table 4-2. The basic two meetings are a Goals meeting followed by a Teach meeting. Another variation to start with is the three meeting sequence of Goals, Teach and Feedback (as illustrated on the videotape, Woods, 1993). The Feedback meeting helps the students realize the quantity and quality of their learning. As your program evolves, you will probably want to later try the four meeting sequence of Goals, Consolidate, Teach and Feedback. In the consolidate meeting, people from different groups who are learning the same subject material, get together to consolidate their understanding of the material. To address issue #5 in Table 4-1 and as discussed later, you might follow a five meeting sequence of Goals, Consolidate, Teach, Feedback and Elaborate. However, to start with I suggest you use the three meeting format.

• Allow about 50 minutes minimum for the Goals meeting. The Teach meeting can be handled in about 50 minutes although the students often hold additional meetings. Probably, try 90 minutes for the first pass through. The Feedback meeting can usually be completed in 60 minutes.

4.3 Organize the student groups

•Create the groups. My feeling is that the optimum size is 4 to 6. Many options are possible. If the groups are to be tutorless, one can:

Use the commitment charting (HTGTM Chapter 1) and cluster students together who have the same degree of commitment. This helps to overcome the complaints that "not everyone is pulling their weight in my group."

Use "marks" to provide students groups with the "same" potential. I have used this by numbering the students sequentially from a ordered list based on marks. With 50 students to make 10 groups of 5, Group 1 has student 1, 11, 21, 31, 41; etc. I modify this after I see the initial distribution to ensure that there is equity, balanceand no known conflicting personalities.

Let them choose. Students are responsible for the decisions they made about group membership.

I have used all three methods; they all seem to work OK.

•Decide on the "life" of a group. Select something in the order of 6 to 10 weeks. If you change groups every couple of weeks, the members do not put in the effort to cope with conflicts nor do they try to make the groups work well. They "put up with" it because it only lasts a couple of weeks and then "they'll be with a good group."

4.4 Create the resources

An example of the resources for the first problem in economics is given in Resources Section D.1.

Consider the material you used to make your "lecture notes." Make a five-star rating of the different resources you used. Make copies of the key materials and put them on reserve in the library. Also put on reserve copies of your lecture notes and transparencies that you used to use.

4.5 Create the environment for learning the subject knowledge

The students need to feel comfortable with their learning environment. They need to feel "in control" and that they will succeed... or at least can learn how to succeed. Since assessment is the most powerful driving force for learning, the assessment must be consistent with the learning objectives selected by the group in the Goals Meeting. It is unacceptable to let the students create their objectives and have the faculty assess the students based on the faculty's objectives. However, recall issue #2 in Table 4-1: Dolmans et al. (1993) suggested that students generate about 60% of the objectives intended by the creators of the course. They found that students tend to generate about 3 to 4 objectives per case; I usually expect my students to generate at least 5 to 6 per problem. Thus, somehow we need to ensure that the students generate the correct and complete objectives. Here are some of the options:

Option 1: You create the learning objectives and give them to the group. This is where you might start if you are implementing PBL for the first time.

Option 2: They create the learning objectives. This causes problems with assessment and breeds uneasiness in the students. They are unsure as to whether they are learning the right stuff. This is supported by the findings of Albanese and Mitchell (1993). To overcome this,provide some form of monitoring by the tutor. Thus, do not use this approach; rather consider Option 3.

Option 3: They create the objectives which are then validated by the tutor.

Option 4: Tapered. Start with Option 1 and move toward Option 3 as the students mature in their skill. This is the approach used at Alverno College.

I would avoid Option 2. If you are implementing PBL for the first time, I recommend Option 4.

4.6 Create an environment for the process skills

To me, this is the most important step. This is one where many err in their early attempts. Start by drawing on the experience of others to identify most of the things that might go wrong. Then think of options of how you want to handle this and write this into the environment before you encounter it. What if...

someone doesn't attend the meetings?

someone continually disrupts the group?

someone promises to research and teach a topic and then pleads "can't do it, I have too many other commitments." ?

someone really hasn't learned anything from the teach meeting but expects to pass?

Empowering student groups is linked with accountability. Students must be accountable for their actions. How you set the course up at the beginning dictates how well you will be able to handle empowerment-accountability throughout the program.

Since small group, self-directed, self-assessed PBL requires that students use problem solving, interpersonal-group, self-directed, critical thinking skills, why not:

1. Include those in your course objectives.

2. Develop objectives (that you can keep as reference, standard objectives) for the process skills and for the subject knowledge.

3.Legitimize and help students develop confidence that they have acquired those skills by assessing the skills.

4.Spell out the consequences of faulty performance right at the beginning.

5.Try to make the implicit explicit so that all can monitor and see the performance (or lack thereof). I use feedback forms that people must fill out about a performance. These forms are given to the "performer" who then must write an objective analysis of the feedback he/she has received.

6.create a mindset that suggests "without positive and negative feedback we stagnate. Positive and negative feedback are needed for personal growth." These are consistent with Level 5 attitude on Perry's scale, HTGTM Chapter 1, p. 1-6. However, students still have many misconceptions and misgivings about assessment and feedback.

7.Run empowerment-accountability workshops to develop process skills.

We create the environment through our course outline, by what we elect to assess, and how clearly we spell out the empowerment-accountability conditions. We create the environment by how we anticipate and address contentious issues. We create the environment by what we say and the types of empowerment-accountability workshops we run.

Resources provides enrichment and practical examples of most of the issues discussed here.

1. Section D.2 gives example course objectives.

2. Section D.3 gives example objectives for specific process skills.

3. Section D.3 lists examination questions for specific process skills; while Section E addresses the issues of student assessment and program evaluation.

4. Section D.4 presents an example letter written to a delinquent student. Other ideas are given by Woods et al. (1995).

7. Section B.6 gives timing sheets and example transparencies for empowerment-accountability workshops to develop process skills.

4.7 Create an environment to develop "expertise"

When we encounter a problem in a subject area where we know little, the problem is a problem. We read and reread the problem statement. We identify the goal. We explore the problem to try to discover what is really pertinent, what is important and what can be neglected. We work backwards by focusing on the "goal" and trying to see what we need to do to connect the goal to the given information. We try many different things. We make mistakes... many mistakes. This describes "problem solving".. the process most of us will use when we encounter the problem in PBL.

Once we solve the problem, however, when we encounter a "similar" problem in the future, it is no longer a problem; it is an exercise. Here we use patternrecognition. We scan the problem statement, recognize a pattern that is similar to what we have experienced (and solved) before. We then identify this as an exercise. We recall familiar procedures. We work forwards from the given information toward the goal.

When we start into a new field, we encounter many problems, work our way through them and build up our repertoire of sample solutions and solved problems. The more our expertise develops, the more we encounter exercises, and the less we encounter problems.

Figure 4-1 illustrates this phenomena. During university, students experience a mixture of problems and exercises. Gradually, as they "solve problems" and build up the number of problems solved and conscientiously build up the context and process specificity so that they can "recognize problems as being exercises," then we encounter fewer and fewer problems. Norman (1984) suggests that of the situations that an experienced doctor encounters, only about 5% are "problems;" the rest are "exercises." But, just solving problems does not automatically mean that the next time we encounter a similar problem we will be able to recognize it as an exercise. Schoenfeld and Herrmann (1982) asked novices and experts to group together those problems that are similar mathematically in that they would be solved in the same way. They were also asked to indicate the extent to which the same mathematical principles would be used. Thus, 100% would mean that the two problems would be solved with exactly the same method. Some of the results are shown in Figure 4-2. Novices saw problems 1, 11 and 3 to be very similar and completely dissimilar from 2 and 9. Experts saw problems 1, 2 (9 and 11) to be very similar and completely dissimilar from 3. In using pattern recognition novices tended to use "similar wording" whereas experts were able to consider the underlying "similar fundamentals." Hence, it is not a trivial task to use pattern recognition. Norman and Regehr (1995) emphasize the importance of the context in which the problem is solved and the type and extent of elaboration.

This work helps me to interpret issues #3, 4 and 5 in Table 4-1. Two of the major difficulties with PBL are issues #4 and #5 in Table 4-1. In the conventional program, even though we dislike it, the student's approach tend to be to collect sample solutions. They then use pattern recognition to try to match memorized past solutions to the new situations. Such an approach

Figure 4-1: Gradually problems become "exercises" as expertise develops

Figure 4-2 The results of Pattern recognition of novices versus expertsis characterized by "working forwards." This is discussed in detail by Albanese and Mitchell (1993) and in the problem solving literature related to "exercise" versus "problem" solving. The students learning strategy is, therefore, to collect many sample solutions. Over a one to two week period of time, students will often "collect" about a dozen sample solutions. Hence, when faced with a new "problem" their strategy will be one of pattern recognition to try to match the new situation with the extensive collection of sample solutions.

In PBL, students use primarily one problem to drive the learning for a one to two week period of time. They use a problem solving approach. Unless we explicitly add activities, they see few additional sample solutions. Their strategy is to approach each new problem and each new situation as a problem.

Hence, in PBL to develop an environment to help develop expertise, we want students to spend time elaborating on those problems that they solve. One idea, suggested in Table 4-2, is to add the "elaborate" meeting.

4.8 Assess the student's performance

Legitimize the student's effort in developing lifetime learning skills, problem solving and the other process skills by assessing them. The key is, before you have your first meeting with the class, sort out the details of what and how you plan to assess them. Then, make your plans explicit and public.

Issues related to assessment are described in Chapters 8 and 9 of companion book HTGTM.

Details of what you might do are given in Resources Sections E and F.

Chapter 5 of this book gives other insight.

4.9 Evaluate the program's effectiveness

Before your first class, think of ways that you can evaluate the effectiveness of your shift to PBL.

First, decide on your purpose for evaluation. Is it toconvince others that PBL is better than the conventional approach to learning? Is it to monitor the student's acquisition /learning of skills and thus set benchmarks for further development? Is it to ensure that some standard of performance is maintained? Are you evaluating the program to help the students see their own personal progress?

Second, select the elements that you plan to evaluate. Do you want to evaluate the program's effectiveness in developing lifetime learning skills? in facilitating the student's learning of subject knowledge? developing the student's problem solving skills? in changing the student's attitudes toward learning so that they take charge of their own learning? or in developing the student's critical thinking skills?

The instruments to consider selecting should be:

pertinent to the skills you are interested in;

validated so that their results make sense;

ones where we expect a change in the results because of the PBL experience.

Evaluating the program's effectiveness provides you with useful feedback to the students about their growth and development.

Some of possible instruments are given in Resources Section E. Some of these instruments [*] can be used to help the students understand themselves and improve their personal progress and can be used to evaluate the program. Some can be used primarily for evaluation of the program; these are given as [@]. The results are not given to the students.

Other instruments, not included in Resources Section E, are available to help students understand themselves. These include such instruments as Jungian Typology, FIRO-B. In general, throughout one's lifetime, there is little shift in scores.

4.10 Your first classes

Help your students understand your new approach. If you are using PBL, then take time to help them to understand the new expectations. No matter how

To introduce students to the new approach, I recommend the mini-workshop outlined in Resources Section B.1 .

To help students cope with the change - not matter what it is- consider starting your course with the workshop on Coping with Change, Resources Section B.2.

enthusiastic you are about what you are doing, assume that the students will feel threatened by the change.

4.11 Nice touches you can add

Each student group will be researching similar learning issues although for any group often each person in the group is researching a different topic. However, across the different groups, one person from each group is undoubtedly researching the same topic.

Jigsaw, Scramble or Consolidate (from Table 4-2) are names given to an intervention that brings together people from different groups that are all researching and preparing to teach the same knowledge. More about this can be gathered from Luis Branda, Wendy Crebbin, George Ambury, Philip Specht, Philip Doust and Luisa Fertitta.

This additional Consolidate meeting is very helpful to defuse any tendency for students to deface or hoard the library resources.

4.12 Summary

•Research highlights the difference between conventional and PBL programs. We use that research to help design more effective PBL programs.

•Start simply. Do not be too ambitious.

•Visualize the timing and the student meetings. For any problem, students could have between two and six different types of meetings. I recommend starting with the three meeting format: goals, teach and feedback meetings.

•Organize the student groups. Any way of distributing seems to work OK.

•Create the learning resources. As a start use an annotated list of references.

•Create the environment for learning the subject knowledge. In particular, ensure that the student's learning objectives resemble your objectives for the problem.

•Create the environment to develop the process skills. Make explicit the implicit process.

•Create the environment to develop expertise. Consider adding an "Elaboration meeting" for each problem.

•Assess the student's performance: on both the subject knowledge and the process skills.

•Anticipate how to evaluate the program's effectiveness. Select instruments and approaches.

Then, you are ready for your first class. Enjoy!

Incidently, an additional Consolidate meeting is a nice touch you can add.

4.13 References

Albanese, M.A. and S. Mitchell (1993) "Problem-based Learning: a review of literature on its outcomes and implementation issues," Academic Medicine, 68, 52-81.

Alverno College (1985) "Assessment at Alverno College," Alverno College, 3401 S. 39th St., Milwaukee, WI.

Ambury, G.G. (1995) Queen's University, Kingston, On., e-mail contribution to PBL-LIST, Feb 16. <amburyg@qucdn.queensU.ca>

Branda, Luis, Department of Biochemistry, McMaster University, Hamilton ON L8N 3Z5

Crebbin, Wendy (1995) e-mail on PBL-LIST, Feb 16 <wc1@fs3.ballarat.edu.au>

Dolmans, D.H.J.M., W.H. Gijselaers, H.G. Schmidt and S.B. Van der Meer (1993) "Problem Effectiveness in a Course Using Problem-based Learning," Academic Medicine, 68, 207-213.

Doust, Philip, Tufts Medical School, Tufts University, Medford, MA 02155

Fertitta, Luisa, Tufts Medical School, Tufts University, Medford, MA., 02155.

Norman, G.R. (1984) McMaster Medical school, personal communication.

Norman, G.R. and G. Regehr (1995) "Contemporary Issues in Cognitive Psychology: implications for professional education," McMaster University, Hamilton, The Toronto Hospital, Toronto.

Schoenfeld, A.H. and Herrmann (1982) "Problem Perception and knowledge structure in expert and novice mathematical problem solvers," J Experimental Psychology: learning, memory and cognition, 8, 5, 484-494.

Specht, Philip C., UP School of Medicine, San Juan, Puerto Rico, e-mail Feb 15, 1995 on PBL-List <p_specht@rcmaca.upr.clu.edu>

Woods, D.R. (1985) "Problem-based Learning and Problem Solving" Chapter 2 in "Problem-based Learning in Education for the Professions," D. Boud, ed., Higher Education Research and Development Society of Australasia, Kensington, NSW, Australia, 19-42.

Woods, D.R. (1993) "The MPS SDL program," 24 min videotape, Chemical Engineering Department, McMaster University.

Woods, D.R., W. Duncan-Hewitt, F. Hall, C. Eyles and A.N. Hrymak (1995) "Tutored versus tutorless groups in PBL," Chemical Engineering Department, McMaster University, Hamilton ON L8S 4L7.

Back to PBL