Volume 6 Number 1
Boulder, Colorado
Fall 1991
_Barbara Reilly, Co-Chair. Forty surveys were mailed and sixteen were returned for a 40% response rate. We got a very representative sampling of individuals and some very valuable input_Thank you for responding. I have summarized the comments by question.
I. Please check (4) the Brunswik meetings you have attended. 6 Brunswik 1 Boston 8 Brunswik 2 New Orleans 5 Brunswik 3 Seattle 7 Brunswik 4 Chicago 13 Brunswik 5 Atlanta 11 Brunswik 6 New Orleans 1 Never Attended Of the 16 respondents, 1 had never attended a meeting, 1 had been to one meeting, 1 had been to two meetings, 3 had been to three meetings, 4 had been to 4 meetings, 5 had been to five meetings, and 1 (guess who) had been to six meetings.
II. The meeting I got the most out of was: The responses, in rank order, were Atlanta (5), Chicago (4), Seattle (3), and Boston and New Orleans (1) vote each. Reasons given for these choices included: short informal reports, work in-progress reports, spontaneous, informal, new ideas, good discussion, able to hear from lots of people.
III. The meeting I got the least out of was: Most people said none or couldn't decide. Of those who indicated a least favorite, New Orleans (5), Boston (1). Reasons given included: too formal, papers too long, wanted to hear from more people, too much Brunswikian philosophy, not enough data.
IV. The ideal format for a Brunswik meeting is in my opinion (fill in): informal discussions, a few key notes, combination of show and tell and advances, many short presentations organized by area, fewer talks, more give and take, 20-minute presentations and 10-minute discussions, more time for discussion.
V. Please list the researchers you are most interested in hearing from. Virtually everyone was mentioned by someone_we like ourselves!!
VI. Given that the meeting this year will take place in the Bay Area, are there any local individuals that you would like to see give an invited address? Kahneman or Tversky, No keynote, Gordon Bower, Lew Petrinovich, Barbara Mellers. VII. Any additional comments: Thanks for the survey, alternate speakers by year, need thoughtful critiques of well-presented papers, include more people. In the next few weeks, Roy and I will be working out a format that we hope will meet with most everyone's satisfaction. If you think of anything else in the meantime give one of us a call.
William Balzer and Michael Doherty, Department of Psychology, Bowling Green University
During the past year we have been involved in three major activities:
CFB presentation format
The CFB research group at BGSU (Bill Balzer and a large number of graduate students) has completed a study which examined whether CFB presentation format moderated the CFB_performance relationship. Two CFB presentation format characteristics were investigated: the way the CFB was displayed (graphically or alphanumerically) and the degree of elaboration of the CFB (basic CFB or CFB with analogies and rules-of-thumb to aid judges' interpretation and use of the CFB). 323 students were randomly assigned to one experimental condition of a 4 (CFB components: TI, TI+CI, CI, or TI+CI+FVI) x 2 (CFB display) x 2 (CFB elaboration) design. The results indicated practically and statistically significant effects for CFB components; all conditions that received TI (i.e., TI, TI+CI, TI+CI+FVI) outperformed the CI feedback only condition. There were very few main or interactive effects for presentation format, and those effects that were statistically significant were relatively small in magnitude. Overall, our findings indicate that CFB presentation format, at least as we manipulated it, does not appear to moderate the CFB_performance relationship. The paper has been submitted to OBHDP.
Policy capturing study
Mike Doherty, Pat Raymark, Karen Warren, and Bill Balzer are collaborating with Bob Wigton and Tom Tape at the University of Nebraska Medical School on a policy capturing study investigating what factors people use when deciding to provide or withhold medical treatment for themselves given a number of possible medical situations they could confront at an older age. 53 undergraduates from BGSU and the University of Toledo rated 100 profiles in which we manipulated a number of factors thought to be important (e.g., the amount of pain, predicted level of future mental or physical functioning, financial considerations, etc.). Our preliminary results indicate that in addition to assigning high weightings to a number of individual factors (such as mental functioning), a "level of mental functioning" by "level of physical functioning" interaction contributed to their decision to withhold/provide medical treatment. We were surprised to see such consistency among judges in their interactive use of mental and physical functioning. We are in the process of writing this paper for publication.
Dependent measures in SJT
Mike Doherty, Pat Raymark, Jim Sullivan, and Bill Balzer (all of BGSU) are writing a review paper on dependent measures used in SJT research. The goals of the paper are to identify what measures have been used, identify the strengths and weaknesses of these various measures, and suggest new measures for possible inclusion in future research. For example, we will argue for using the Intraclass Correlation Coefficient (ICC) to index the agreement between a judge's predictions and the outcome in the environment (or another judge in an interpersonal learning task). We are at the very early stages of thinking on this project; any insights, comments, citations, etc. from our Brunswikian colleagues would be greatly appreciated.
Barbara Reilly, Department of Psychology, Clemson University
As you know, I have been working on a line of research that involves self-insight. After the last Brunswik meeting I had a decision to make regarding further studies. I could either continue the research at a more micro level, that is, "How much can you alter a policy and still have adequate recognition," "Under what specific conditions does recognition fail," etc., or I could take a more global perspective. I have chosen the latter for now. Specifically, I'm interested in the benefit of self-insight_that is to say, "Are people with better self-insight somehow helped or aided by this knowledge?" The answer, I believe, is yes. I will be presenting a paper on this topic at J/DM and have summarized it here.
Research in judgment and decision making has shown fairly consistently that people have a difficult time articulating why they choose a particular course of action or make a particular decision. This inability to articulate reasons for a decision, or lack of self-insight, has been shown in a wide variety of decision settings including: stock purchases, home purchases, clinical diagnoses, employee selection, job choice, and other nontrivial decisions.
Recently, research has been conducted which shows that people have far better self-insight than previously believed possible (Reilly & Doherty, 1989, and Reilly & Doherty, in press). When subjects are asked to choose their statistical decision policy from a group of their peers, it has been shown that they can identify their own policy at a probability that far exceeds chance or random guessing. This indicates that subjects know much more about their decision behavior, but that they just have a difficult time expressing this knowledge. This research has been replicated in several different settings and using different types of subjects. Note, although called the recognition method of assessing self-insight, the process has very little to do with recognition from a memory perspective.
Given that these findings are so unusual and so new, there are many unanswered questions including: (a) Under what situations can subjects identify their policies? (b) Is the ability to identify one's policy related to other important behavior, for example, a better ability to articulate? (c) Is self-insight related to better insight in general, for example, insight into how other people feel or make decisions? (d) Do individuals that possess high levels of insight (self and other) have less conflict in a relationship that depends on shared knowledge. These questions were addressed by the current study.
Subjects
Subjects were 50 pairs of undergraduate females (Total N=100) who had lived together for at least one semester. The subjects were making decisions about a domain that they were familiar with, that is, what makes a suitable roommate.
Procedure
The subjects took part in two research sessions, approximately 6 weeks apart.
Session 1. In the first research session subjects made judgments of the suitability of potential roommates that varied along 12 attributes (cleanliness, thoughtfulness, etc.). Note, that the stimulus materials have been constructed and tested through prior research. Each subject judged 100 potential roommates. Additional information was collected at this time regarding the relationship with the current roommate and the subjects' self-reported decision making strategy. The self report consisted of distributing 100 points among the 12 attributes. Subjects were also asked to estimate their roommate's decision strategy via the same method.
Between Sessions 1 and 2, regressions were conducted to develop a mathematical model for each student. The variance accounted for by each attribute was presented in standardized UI format.
Session 2. After an introduction to the types of analyses used to determine their policies and a brief overview of Session 1, the subjects were given a table consisting of a matrix of decision policies of 19 randomly selected policies, included with their own policy. Subjects were asked to indicate which policy of the 20 was their own. After this was completed, subjects were given a different table consisting of a matrix of decision policies of 19 randomly selected policies, included with their roommate's policy. Subjects were then asked to choose which policy of the 20 was their roommate's policy.
Results
Analyses were conducted to answer the following research questions:
1. Under what situations can subjects identify their policies? Prior findings were replicated, that is, subjects demonstrated the expected levels of self-insight.
2. Is the ability to identify one's policy related to other important behaviors, for example, a better ability to articulate? Yes. It appears that subjects who have a higher correlation between their statistical and self-reported policies are more likely to demonstrate self-insight via the recognition procedure (r=.38, p<.01).
3. Is self-insight related to better insight in general, for example, insight into how other people feel or make decisions? Yes. Subjects who identified themselves were more likely to identify their roommate's policy as well.
4. Do individuals who possess high levels of insight (self and other) have less conflict in a relationship that depends on shared knowledge? Yes. Roommates who could identify each other's policies reported less conflict than those who were unable to identify the policies. This was shown in spite of the fact that the conflict scale had a very restricted range (no one had an awful roommate). These findings suggest both the presence of self-insight via the recognition procedure and the importance of self-insight.
Kenneth Hammond, Center for Research on Judgment and Policy, University of Colorado, Boulder
I have worked on several projects during the past year: (a) a review of the literature on stress and J/DM (Hammond, K. R. (1990). The effects of stress on judgment and decision making: An overview and arguments for a new approach (Tech. Rep. No. 320). Boulder, CO: University of Colorado, Center for Research on Judgment and Policy); (b) the development of a new approach to this topic; and (c) preparation of an article on the application of J/DM theory to social policy (Hammond, K. R., Harvey, L. O., Jr., & Hastie, R. (in press). Making better use of scientific knowledge: Separating truth from justice. Psychological Science).
The review of the literature on stress and J/DM led me to conclude that there is no strong scientific knowledge concerning this topic. My current effort is to present the major theoretical approaches to J/DM and to raise questions as to how the concepts contained therein might be affected by various stressors. A second part of that effort is to develop further various aspects of Cognitive Continuum Theory and new offshoots from it and to theorize about the effects of stress upon the parameters of this theory. (Naturally, the latter part gets more attention than the others.)
My efforts with Harvey and Hastie represent a return to one of my persistent interests_the application of J/DM theory (in this case both Signal Detection Theory and the Lens Model)_to the development of social policy. (I think we made a significant step forward here.)
Cynthia Lusk and I published an article (Lusk, C. M., & Hammond, K. R. (1991). Judgment in a dynamic task: Microburst forecasting. Journal of Behavioral Decision Making, 4, 55-73) that demonstrated the successful use of a hierarchical Lens Model in highly representative task conditions (forecasting the microburst). We also published with a meteorologist (Rodney Potts) a closely related article on microburst forecasting (Lusk, C. M., Stewart, T. R., Hammond, K. R., & Potts, R. J. (1990). Judgment and decision making in dynamic tasks: The case of forecasting the microburst. Weather and Forecasting, 5, 627-639).
Suggestion: In your leisure hours, read Isaiah Berlin's "The Fox and the Hedgehog" in his Russian Thinkers.
Paul Werner, California School of Professional Psychology, Berkeley/Alameda
Paul Werner and his colleagues have applied a Brunswikian perspective to the study of clinical predictions of dangerousness. Lens Model analyses have investigated judgments made by professionals (psychiatrists, psychologists and social workers) and nonprofessionals (high school students and former psychiatric patients) about psychiatric patients' dangerousness. This approach has also been applied to studying judgments by correctional professionals (correctional psychologists and case managers) about prison inmates' dangerousness. Further cue utilization studies on judgments of dangerousness have compared decisions made by naturally occurring treatment teams on psychiatric inpatient wards with those of simulated teams of college undergraduates. In all of these studies, special attention has been directed to assessing interjudge reliability in forecasting dangerousness.
Alex Wearing, Department of Psychology, University of Melbourne
The foundations of the world are unlikely to be shaken by anything coming out of Melbourne. Old wine in old bottles, or even colored water in old bottles! Our broad focus continues to be dynamic decision making in complex environments, looking at nurses in hospitals (how they model their problem environment, and then manage/solve these problems), simulated firefighting (the affective and cognitive correlates of performance, expressed as a causal model), cognitive models of the economy, and police (managing their problem environment).
Roy Poses, Division of General Medicine, Medical College of Virginia
We are currently investigating how physicians make several critical judgments for patients with acute congestive heart failure, a common, serious medical condition. We are collecting judgments made by Emergency Department physicians when they first see these patients, and also collecting information about a wide variety of clinical cues from the physicians, patients, and patients' charts. We also are trying to determine the availability of relevant memories to the physicians and the severity of stress on the physicians at the time they make the judgments. Simultaneously, we seek to develop multivariable models of the relevant outcomes (ecology). We hope to use classic Lens Models to compare the physicians' judgments to these outcome models. We are particularly interested in the effects of stress on cue usage in this real and stressful setting.
D. Mark Chaput de Saintonge, Therapeutics Section, London Hospital Medical College
Collaborative research with a Swedish group continues on ways to change doctors' attitudes to cardiovascular risk factors. Paper patient validation studies are now complete, but we are still struggling with the problem of how to reduce the number of candidate cues (of which there are 30-40 in this problem area). Work has started on using JA to standardize scoring systems for Objective Structured Clinical Examinations in medicine.
C. R. B. Joyce, Basle, Switzerland
Dick Joyce is continuing work with the group under Kevin O'Malley and Ciaran O'Boyle at the Royal College of Surgeons in Ireland, on the application of JA to assessment by individual patients of their own Quality of Life (QoL). The first full paper on the SEIQoL (Schedule for Evaluation of Individual QoL) was due to appear in the August issue of "Psychological Medicine." Other examples of the versatility and utility of JA being developed at present include the expert assessment of experimental protocols for investigations in so-called "alternative" medicine, and of the "creativity" manifested by different kinds of students pursuing various projects.
Tape, T. G., Heckerling, P. S., Ornato, J. P., & Wigton, R. S. (1991). Use of clinical judgment analysis to explain regional variations in physicians' accuracies in diagnosing pneumonia. Medical Decision Making, 11, 189-197.
The authors sought to explain regional differences in physicians' accuracies in diagnosing pneumonia by prospectively studying emergency department patients at three sites and analyzing differences in physicians' diagnostic strategies and patient characteristics. They enrolled 1,119 Illinois patients, 150 Nebraska patients, and 142 Virginia patients presenting with fever or respiratory symptoms for whom physicians ordered a chest radiograph because of suspicion of pneumonia. Emergency department physicians recorded patients' clinical findings and estimated the probability that a chest radiograph would show pneumonia. A measure of accuracy, the correlation between physicians' probability estimates and actual outcomes, was 0.41 (95% CI 0.36_0.46) at Illinois, 0.66 (95% CI 0.54_0.75) at Nebraska and 0.55 (95% CI 0.42_0.65) at Virginia. Physicians' strategies at the three sites differed markedly in their weightings of asthma, signs of consolidation, cough, tachypnea, age, and gender. These differences in weighting parallel differences in the optimal clinical strategies derived from patient data at the three sites. Differences in diagnostic accuracy were best explained by differences in the difficulties of diagnosing pneumonia in the populations. Physicians at each site used clinical findings in a way that was close to optimal for their location. This type of analysis provides a new tool for understanding the sources of regional variations in clinical practice.
Marilyn Rothert, College of Nursing, Michigan State University
The MSU research team was funded May 1 for a three year grant by NIH, National Center for Nursing Research. Based on our previous study which clustered women's policies regarding Estrogen Replacement Therapy, this study is developing and testing an educational intervention to assist women to be informed, empowered consumers within the health care system. We are currently developing the instruments and content for the decision support intervention. One instrument we continue to wrestle with is a measure of satisfaction with a decision. We recently responded to an NIH RFP for a supplemental grant to expand the study to black women. If funded, this would allow us to begin to understand the perceptions and policies of black women related to menopause and hormone replacement therapy.
Len Adelman, Department of Systems Engineering, George Mason University
I have continued research on the two major projects discussed last year, both of which examine the effect of task characteristics on judgment and decision making. The first project (with Decision Sciences Consortium, Inc.) continues to examine the effect of information order on expert judgment. We performed an experiment with Patriot air defense officers utilizing the Patriot air defense simulators at Ft. Bliss, Texas. Although there were problems implementing the full experimental design with the simulator, the obtained results suggested that Patriot air defense officers were susceptible to an order effects bias under certain conditions and not others. In particular, when there was only one aircraft track on the Patriot display at a time, the same information resulted in different judgments depending on the sequence (i.e., order) in which the information was presented to the operators. Moreover, in certain instances, the order effect significantly affected the operators' engagement decision. These findings, which replicate those found with a paper-and-pencil instrument, are consistent with the predictions of the Hogarth-Einhorn belief updating model, which presumes the use of an anchoring and adjustment heuristic.
The second project (at GMU) continues to examine the effect of problem formulation on option generation. In particular, we performed an experiment comparing a technique for helping individuals break their causal focus with a traditional brainstorming technique. Participants generated the same number of options, and a broad range of options, with both techniques. However, the techniques significantly affected the kinds of options that participants emphasized during their option generation process. In addition, it affected the kinds of options they selected to solve the problem. The effects on option generation and selection were mediated, as predicted, by the techniques' effect on the type of hypotheses participants generated to explain the problem. The mediation effect was not, however, as strong as predicted. We are still analyzing the data to try to understand why.
Thomas Stewart, Rockefeller College, of Public Affairs & Policy, SUNY_Albany
Work continues on understanding the components of skill in forecasting and how skill is affected by increasing the amount of information. An experiment in progress will attempt to assess the reliability and validity of information acquistion and information processing in weather forecasting.
James Hogge, Department of Psychology, and Human Development, Peabody College of Vanderbilt University
Among my current SJT-related activities is the development of a computer program to generate cues with specified means, variances, and intercorrelations according to a procedure described by Mark E. Johnson in Multivariate Statistical Simulation (Wiley, 1987). The program will produce output compatible with POLICY PC and should be completed well before the November meeting of the Society. In separate studies focusing on student teachers and nursing students John Murrell (Homerton College, The University of Cambridge) and I have attempted to use social judgment analysis to explicate values underlying the assessment of professional competence.
In each study, assessors (experienced teachers, teacher educators, experienced nurses, or nurse educators) were first asked to distribute 100 points among several (six for teachers, seven for nurses) aspects of professional competence defined by an existing assessment instrument or handbook. The relative weights thus obtained were taken as representing "expressed values." Next, each assessor rated the overall competence of 50 hypothetical students for whom scores on the several aspects (cues) were available. Relative weights ("implemented values") applied to the cues were then computed from each assessor's regression equation.
Although data analyses are still in progress, individual differences with respect to both expressed and implemented values have been noted as well as discrepancies between the expressed and implemented values of some assessors. Further analyses will examine possible relationships between such discrepancies and various attributes of the assessors (e.g., professional status, experience).
Jeryl Mumpower, Rockefeller College of Public Affairs & Policy, SUNY_Allbany
I am currently involved in three research activities:
Negotiation
With Tom Darling, Rick Milter, and several other colleagues, I am involved in a series of empirical and computer simulation studies investigating negotiation from a Brunswikian point of view. We are interested in how negotiators come to terms with the constraints imposed and opportunities afforded by the negotiation problem structure. This structure itself results from the interaction of the negotiators' judgment policies. In one study that we just completed, we found that for two problems with identical formal characteristics (i.e., identical payoff tables, etc.), but different substantive characteristics (i.e., issue labels, etc.), negotiators were much more likely to agree to efficient settlements in one substantive version than in the other. We concluded that negotiation problems often provide cues to negotiators about what type of agreement is likely to be mutually acceptable.
Expert Judgment
Tom Stewart and I continue to grapple with the problem of expert judgment. I am trying to develop a taxonomy of expert disagreements that will be applicable in both basic and applied research settings.
Siting of Radioactive Facilities
Anna Vari and I are studying attempts in New York State and elsewhere to site a low-level radioactive waste treatment and disposal facility.
David Funder, Department of Psychology, University of California, Riverside
I am continuing my research on accuracy in personality judgment, in which judgments rendered by the self, by acquaintances, and by strangers are compared to each other and to measures of the actual behavior of the persons judged. Funder and Colvin (1991) report on the convergence between personality judgments and patterns of behavior across situations of the persons judged. Colvin and Funder (1991) compare the predictive validity of acquaintances' and strangers' judgments for predictions of personality and behavior. Copies of these articles are available from David Funder.
HTML formatting by Yong-Jin Cha, Center for Policy Research, University at Albany.
Return to Brunswik Society Home Page