"Problem-based Learning: helping your students gain the most from PBL" 3rd edition, March 1996

© copyright 1996, Donald R. Woods

Postface: about the MPS program.

Throughout this book, and other books in the series, are frequent references to the McMaster Problem Solving, MPS, program. In these books, we cite workshops within the MPS program. We infer the effectiveness of these materials and approaches. What is the MPS program? How does the MPS program relate to PBL? How do we know that the MPS and PBL approaches are effective?

P.1. What is the MPS program?

Higher order thinking, problem solving, team work and lifetime learning skills - these are some process skills that students expect to get from their university programs (Boud and Lublin, 1983; Bradford, 1984). Several reports have suggested that current undergraduates and graduates both need but do not possess these abilities (Rush et al., 1985; Sparkes, 1989; Resnick, 1987; Woods and Crowe, 1984). Over the past 20 years, we defined these process skills, identified effective methods for developing student's process skills, implemented a series of four, required courses to develop the skills and evaluated the effectiveness of the program. Through four research projects we identified which teaching methods failed to develop process skills and which methods were successful in developing the skills (Woods et al., 1975; Woods et al., 1979 and 1985; Woods, 1993a,b,c). We identified 57 general component skills and focus on the development of 37 of these in the time we have available. Table P-1 lists the units. We use 120 hours of workshops spread over four required courses to develop the skills. Each skill is built (using content-independent activities), bridged (to apply the skill in the content-specific domain - such as Chemical Engineering) and extended (to use the skill in other contexts and contents and in everyday life). Tests and examinations of process skills, TEPS, were developed to assess the degree to which the students can apply the skills. We call this program of 120 hours of integrated workshops the McMaster Problem Solving (MPS) program.

Table P-2 lists the process skill, the names of the major MPS units that develop that skill (plus names of pertinent units whose prime goal is to develop otherskills) and the course sequence. The amount of time required depends on the class. Some skills are honed concurrently in several units so that the number of in-class workshop time and the hours in the courses are not consistent.

Details about the program are reported elsewhere (Woods et al., 1984; Woods, 1987; Woods, 1992).

P.2 How does the MPS program relate to PBL?

We find the problem-based learning environment to be one of the most effective media to develop our student's skill in lifetime learning. In addition, we believe that small group, self-directed, self-assessed PBL promotes better learning and retention of the subject knowledge. So, small group, self-directed PBL workshops are required parts of courses #3 and 4 when the students are learning the chemical engineering subjects "process safety and engineering economics." We wish we could use this learning environment for other subjects in the curriculum. What is delaying us is:

•We feel that our students need to have a high degree of process skill development before we use PBL (more specifically, we need about 80 hours of workshops first before using the PBL format)

•Our faculty resources are limited so that we cannot provide one tutor for each group of five to six students. Hence, we work with tutorless groups so that one instructor can manage five to ten groups of students simultaneously (Woods, 1991; 1996; Woods et al., 1996).

Hence, in the MPS program we use small group, self-directed PBL

1. To develop the process skill of "lifetime learning", and

2. So that students learn "more effectively" some chemical engineering subject knowledge.

We do not use small group, self-directed PBL until after a basic core of "process skills" have been developed through the MPS program so that we can then work with tutorless groups.

P.3 How do we know that the MPS and PBL approaches are effective?

We have evaluated the MPS program from seven different perspectives: 1) control studies showing marks improvement, 2) student acceptance of the learning environment, 3) student's confidence in their process skills, 4) student's skill in problem solving and other process skills, 5) their attitude toward lifetime learning, 6) the development of self-assessment skill and 7) alumni, recruiter and employer response.

P.3-1 Control studies

In our program, Chemical Engineering and Applied Chemistry students take a required, 4-credit course on the "principles of chemical engineering" in the second semester of the second level. We set up the new curriculum in the fall of 1982. Chemical Engineering students were required to take course #1 in problem solving concurrently with a convention, subject-based "principles" course. The Applied Chemistry students did not take course #1 and therefore served as the control group. We used marks for both groups before 1982 (before the treatment) and the pooled marks for 1982-83 and 1983-84 when the chemical engineering students received the problem solving courses and the control group did not. For both groups we measured the difference between the marks in "principles" course after 1982 and before 1982. The null hypothesis was that the difference in Chemical Engineers marks before and after 1982 would be the same as the difference in Applied Chemists marks before and after 1982. In other words, the problem solving course would have no significant effect on the students' marks in the "principles" course. Based on a t-test, the probability that the marks between the two groups differ by chance alone is 5.9%. Although not conclusive, this is a good indication that an improvement in marks in the "principles" course occurred because the students took the concurrent course in problem solving.

P.3-2 Learning environment. The students' assessment of the learning environment, as measured by the Course Perceptions Questionnaire (Knapper, 1994; Ramsden, 1983), is d= +1 (where d is the response of the target group minus the response of the control group divided by the standard deviaation of the control group) more positive than the responses from a control group of engineering students in a conventional program (N=47).

P.3-3 Confidence with problem solving skills. Based on Heppner's PSI inventory (Heppner and Peterson, 1982), our student's scores changed positively by d = +1 to +2. This has occurred in each of the eight years since the program's inception. Students completed the inventory when they first entered the program; the same inventory was used as a post-test for the same students leaving the program. The total PSI value changed from 92 to 71. A small value is wanted for all measures. The control groups' PSI scores changed negligibly over the three or four year period: 91.3 to 90.4.

P.3-4 Process skills In each of the three years, students wrote two or three-hour written examinations, TEPS, assessing their processing skills. Class average marks were 60% to 87% over the past 10 years. In addition, the Billings-Moos Coping Responses Inventory (Billings and Moos, 1981) test showed improvements in scores on both avoidance and problem solving in most successive years (with d = 0.43 and 1.07 between entry and exit of the program). In this test, Billings and Moos reported that problem avoidance and problem solving were the significant factors affecting one's ability to cope. For avoidance, a small value was wanted; for problem solving, a large value.

P.3-5 Attitude toward lifetime learning. The Perry inventory (see Woods, 1994, Chapter 1) changed from an average of about 3.5 in third year to an average of about 4.6 in the final year as measured by either the Gainen or Moore-Fitch inventories. In another study, done within our own group of students, we created problem-based learning, PBL, groups based on grade point so that each group had approximately the same grade point average. After six weeks of PBL activity, we measured the degree to each group was comfortable with self-directed learning (as measured by peer and self assessment). We can define effective groups as those where all five members work as interdependent self-directed learners . Ineffective groups might be those where only two of the five members work as interdependent self-directed learners. Our research showed that effective groups scored 10 marks higher on subject knowledge examinations than did ineffective groups (Woods, 1996).

P.3-6 Self assessment skill development. In two studies, we compared the student's self-assessment with their complete performance in two courses and with their performance on written two or three-hour examinations. (The performance mark differs from thefinal examination mark because students contract for different weighting for the term work. That is, some may elect to have the term work count 40% with the final examination worth 60%.) In the one study (N=49), the self-assessment marks were 1.3% lower than their marks for complete performance and were 6.3% higher than their marks on the final examination. In the second study (N=50), the self-assessment marks were 1.9% lower than their marks for complete performance and were 1.6% lower than their marks on the final examination.

P.3-7 Alumni and recruiter response. We completed a blind survey of graduates one to five years from the program (N=48). We asked them to identify "The courses that were the most important for their current professional progress." The results were that 58% of the alumni cited the process skills and PBL courses in our curriculum. The other courses cited were "engineering fundamentals" (25%); project work (10%) and the remainder identified individual courses, such as the environmental course, or the statistics course. On the blind survey, here are typical comments, "If you learn nothing else in Chemical Engineering, remember everything you learn in the process skills courses #1, 3 and 4." "The problem solving that is developed from Day 1 in the Chemical Engineering program is one of the tools that puts the McMaster graduate above engineers from other schools." "The processing courses give me a bit of an edge."

Alumni and students have written articles (Lieske, 1983; Moore et al., 1979; Liebold, B.G. et al., 1976; Chornenko, et al., 1979; and Bouchard, 1996) and have written directly to us about their undergraduate experience. "I consider my experience as an undergraduate in the MPS program invaluable; I simply could not do what I do without having developed critical problem solving skills. My career demands that I am constantly up-to-date on technology and that I always learn new ways to apply fundamentals to the pharmaceutical industry. While much of the base knowledge is technical, a large portion involves using fundamentals to solve difficult, open-ended problems. This type of work takes much more than a knowledge of "type" problems (where the problem is essentially solved by combining past-solved exercises). One of the most important points addressed is that of the transfer of skills from one problem-solving environment to another. I believe the activity that helped me the most was bridging the problem solving skills (which Ideveloped during the workshops) to different worlds, technical and everyday life. I think it was writing the reflective reports that underlined this. The report made me focus on applying the learned skills." Many of our alumni now run workshops based on the MPS materials in industry.

Recruiters wished to remain anonymous. However, here are the reactions we have received:

employer X used to recruit on 15 campuses across Canada, then on 5 and now on 3. McMaster is one of the three.

employer Y hired a series of our graduates, each of whom he said could "think for themselves and solve problems upon graduation." He also hired from two other ChE schools in Canada, and noted that they had to spend "1 to 1 1/2 years" training the new hires before they could "think for themselves."

employer Z requested that another university should set up identified parts of our MPS problem solving program before they would recruit from their campus;

employers O, P, Q, R, S and T who hired us or graduates of our program to give in-house MPS workshops on problem solving.

employer M comments "graduates of the McMaster ChE program are able to clearly communicate ideas. I see vast and immediate differences between graduates of the McMaster program and other university science programs."

The alumni, recruiter and employer responses are difficult to interpret because our program includes many elements. Yet, employers identify the problem solving and group process skill as a clearly identifiable attributes that they see our graduates possess. Identifying which components in our whole program created this shift, and the role of PBL, is impossible for us to discern.

P.4 Summary

The MPS program is a series of four, required, workshop-style courses to develop process skills and to use small group, self-directed PBL in tutorless groups. The target skills being developed include self-confidence, problem solving, interpersonal and group, self-assessment, change management and lifetime learning. Seven measures were used to evaluate the effectiveness of the program.

Table P-1: List of the MPS Units

Core Units for course #1

1. Awareness

2. What is Problem Solving?

3. Self-assessment

4. Strategies

5. I want to and I can: Stress Management

6. Analysis: classification

7. Creativity

8. Introduction to visual thinking: translation

9. Define the stated problem

10. Getting Unstuck

11. Identifying Personal Preference and Implications

12. Learning Skills

13. Analysis: Consistency

14. Creating the Look Back and Extending Experiences

15. Exploring the Situation to Identify the Real Problem

16. Tactics:

17. Time Management for Individuals

18. Evaluation and Stress Management.

Core Units for course #3

19. More on Visual Thinking: Reading P&IDs

20. Asking Questions

21. Analysis: Sequences and Series

22. Broadening Perspectives.

23. Obtaining Criteria.

24. Decision making.

23-24a. Criteria and Decision making in the context of career Counselling and Guidance.

25. Time Management for groups and projects.

26. Listening and Responding:

a) Attending and following

b) Body language.

c) Reflecting

27. Group Skills.

28. Group Evaluation.

Core Units for course #4

29. Being an Effective Chairperson

30. Analysis: Reasoning and Drawing Conclusions

31. Defining Real Problems

32. Implementing

33. Coping with Ambiguity:

34. Trouble Shooting

35. Heuristics or Rules-of-thumb for Problem Solving:

36. Self-Directed Learning: or Problem-based Learning

37. Simplifying and Generalizing:

38. Consolidating the Knowledge Structure:

38a. Consolidating the Knowledge Structure in Chemical Engineering:

39. Creating Tacit Information or Experience Knowledge:

39a. Creating Tacit Information or Experience Knowledge in Chemical Engineering:

40. Successive Approximation and Optimum Sloppiness:

Other Units

41. Finding Opportunities

42. Procrastination and other Attitudes:

43. Giving and Receiving Feedback

44. Assertiveness

45. Coping Creatively with Conflict

46. Coping with Difficult Behaviours

47. Accentuating the Negative

48. Communication:

49. Coping with Change:

50. Being a Change Agent

51. Managing Change

52. Fundamentals of Interpersonal skills

53. Effective Teams and Team building

54. Goals, Mission and Vision

55. Roles and Responsibilities in Teams

56. Networking: How to enrich your Life and Get Things Done

57. Convincing Others: Getting a Buy-in

Table P-2: Details of the MPS units, their sequence and themes

P.5 References

Billings, A.G., and R.H. Moos (1981) "The Role of Coping Responses and Social Resources in Attenuating the Stress of Life Events," J. Behavioral Medicine, 4, 2, 139-157. Available as the Coping Responses Inventory CRI from Psychological Assessment Resources, PO Box 998, Odessa FL 33556.

Bouchard, C.G. Kyle (1996) "Developing problem solving and team skills," Toronto Institute of Pharmaceutical Technology, Toronto, ON

Boud, D., and J. Lublin (1983) "Self-assessment in professional education: a report to the commonwealth education research and development committee," Tertiary Education Research Centre, The University of New South Wales, Kensington, NSW, Australia.

Bradford School of Technical Management (1984) "Managerial Skills and Expertise Used by Samples of Engineers in Britain, Australia, Western Canada, Japan, the Netherlands and Norway," University of Bradford, Technical Report TMR 152.

Chornenko, D.M., et al. (1979) "What is Problem Solving?" Chemical Engineering Education, 13, 3, 132-137.

Heppner, P.P and C.H. Petersen (1982) "The Development and Implications of a Personal Problem-solving Inventory," J. Counselling Psychology, 29 1, 66-75; Heppner, P.P. (1986) "The PSI Manual," 210 McAlester Hall, University of Missouri-Columbia, Columbia, MO 65211; Available as Problem solving inventory from CPP, PO Box 10096, Palo Alto CA 94303-0979

Knapper, C. (1994) Instructional Development Center, Queen's University, personal communication of the short CPQ version used in the paper D. Bertrand and C. Knapper (1993) "Contextual Influences on Student's Approaches to Learning in Three Academic Departments," Queens University, Kingston ON.

Liebold, B.G. et al., (1976) "Problem Solving: a freshman experience," Engineering Education, 67, 2, 172-176.

Lieske, S. (1983) "Solving Problems in Industry," in "Problem Solving," J.T. Sears et al., eds., AIChESymposium Series 79, No. 228

Moore, R.F., et al. (1979) "Developing Style in Solving Problems," Engineering Education, 69, 7, 713-717.

Ramsden, P. (1983) "The Lancaster Approaches to Studying and Course Perceptions Questionnaires: Lecturer's Handbook," Educational Methods Unit, Oxford Polytechnic, Oxford, OX3 0BP

Resnick, L. (1987) "Education and Learning to Think" National Academy Press, Washington , DC. (1987).

Rush, J.C., J.A. Krmpotic and F.T. Evers (1985) "Making the Match" Corporate Higher-education Forum, Montreal plus Phase II Reports.

Sparkes, J.J (1989) "Quality in Engineering Education," Engineering Professor's Conference, Occasional paper #1, Department of Mechanical Engineering, University of Surrey, Guilford, Surrey (1989).

Woods, D.R., J.D. Wright, T.W. Hoffman, R.K. Swartman and I.D. Doig (1975) "Teaching Problem Solving Skills," Annals of Engineering Education, 1, 1, 238-243.

Woods, D.R. et al. (1979) "Major Challenges to Teaching Problem Solving" Annals of Engineering Education, 70, No. 3 p. 277 to 284, 1979 and "56 Challenges to Teaching Problem Solving" CHEM 13 News no. 155 (1985).

Woods, D.R. and C.M. Crowe (1984) "Characteristics of Engineering Students in Their First Two Years," Engineering Education, Feb., 280-295.

Woods, D.R. (1993a) "Problem solving - where are we now?" J. College Science Teaching, 22, 312-314.

Woods, D.R. (1993b) "Problem solving - what doesn't seem to work," J. College Science Teaching, 23, 57-58.

Woods, D.R. (1993c) "New Approaches for developing problem solving skills," J. College Science Teaching, 23, 157-158.

Woods, D.R. (1994) "Problem-based Learning: how to gain the most from PBL," D.R. Woods, Waterdown, distributed by McMaster University Bookstore, Hamilton, ON, Canada.

Woods, D.R., W. Duncan-Hewitt, F. Hall, C. Eyles and A.N. Hrymak (1995) "Tutored and Tutorless Groups in PBL," McMaster University, Hamilton, ON.

Woods, D.R. (1996) "Assessing Learning in Small Group Problem-based Learning," Chemical Engineering Department, McMaster University, Hamilton ON.