Reference no: EM13120561 , Length: 776 words

" Applying Problem Solving
Resources: Ch. 10-12 of the text, personal experiences
# EBOOK COLLECTION: Kirby, G. R., & Goodpaster, J. R. (2007). Thinking: An interdisciplinary approach to critical thinking(4th ed.). Upper Saddle River, NJ: Pearson Prentice Hall.
Choose two different real-world problems you have encountered. The problems should be chosen with the intention of applying persuasive thinking as a solution to one and scientific thinking as a solution to the other.
· Write a 700- to 1,050-word paper in which you provide the following:

o A description of each of the problem situations

o An explanation of using persuasive thinking to solve the first problem

o An explanation of using scientific thinking to solve the second problem

· Summarize the solutions to each problem with a description of how the solution will solve the problem.

· Format your paper according to APA standards

CHAPTERCHAPTER 10 ISBN:CHAPTERCHAPTER 10 ISBN:SCIENTIFIC THINKING 

Science is a way of thinking much more than it is a body of knowledge. -CARL SAGAN, BROCA'S BRAIN 
Science. It is almost a second language as well as a method of inquiry. Rarely a day goes by when we don't hear about some new discovery in fields such as medicine, psychology, and physics. To think more critically about such discoveries, such as a cure for cancer, new treatments for depression, evidence for life on other planets, or an advertisement for a new "wonder drug," we need to know the language and methods of science. 
In this chapter we explore the nature of science, beginning with the basic steps of the scientific method. We identify some of the assumptions and requirements of this method and contrast it with other ways of knowing. We look at the empirical nature of science and its limitations, and we briefly consider the problem of proof. We also explore some research designs, their drawbacks, and the experimenter biases of the scientists themselves. Our goal is not to become scientists but to learn about the basis of research in order to become intelligent consumers of scientific information. 
THE SCIENTIFIC METHOD The worldwide technical and scientific literature was over 60 million pages a year by 1970 (Toffler, 1970). If scientific information comes close to doubling every twelve years as some suggest (Marien, 1998), it may now easily exceed 400 
220 
Thinking, Fourth Edition, by Gary R. Kirby and Jeffrey R. Goodpaster. Published by Pearson Prentice Hall. Copyright © 2007 by Pearson Education, Inc. 
The Scientific Method 221 
million pages. This explosion in knowledge began with an increased reliance on the scientific method as the tool for understanding our material and psychosocial universe. This method, which has so radically transformed our world, is a type of inductive thinking that moves through four major steps: 
1. Observation 2. Hypothesis formulation 3. Experimentation 4. Verification These same four steps were used by Galileo when he studied the effects of gravity on falling objects. Galileo observed bodies appearing to fall faster the longer they fell. He then formed a hypothesis that falling bodies increase their speed at a steady rate. He experimented by rolling balls down an inclined plane and measuring their speed at different points. He then attempted to verify his hypothesis by analyzing the experimental results, which showed that the balls increased their speed at a constant velocity of 32 feet/second every second, in agreement with his hypothesis. To further verify his results, Galileo and others ran his experiment again. 
Observation 
The scientific method begins with observation. Observation is the food for our wondering about the world. We might observe a phenomena that needs explaining, such as the rising sun or a comet tail. Or we might observe a possible relationship between two events that needs to be tested, such as our grandmother's vegetarianism and her longevity, or the bite of a skunk and the disease of rabies. Observations lead us to wonder about the causes and effects of what we observe, about its character and constitution, and how we might intervene to create desirable change. For example, as we observe that many human beings are stricken with cancer, we may begin wondering about the cause of cancer, the processes that maintain or strengthen cancer cells, and about ways to prevent it or remove it. This kind of wondering about cause-and-effect relationships can be called scientific thinking and takes us to the second step of the scientific method. 
Hypothesis 
A hypothesis is a tentative statement about the relationship between two variables, usually in the form of a prediction: "If A, then B." For example, if (1) we had observed that people dying of cancer are usually heavy cola drinkers, (2) we were aware that cancer rates were lower before cola was invented, and (3) there was considerable scientific debate about the safety of cola additives, then our thinking 
ISBN:
Thinking, Fourth Edition, by Gary R. Kirby and Jeffrey R. Goodpaster. Published by Pearson Prentice Hall. Copyright © 2007 by Pearson Education, Inc. 
ISBN:
222 CHAPTER 10 ¦ Scientific Thinking 
and observation might lead us to suspect that the cause of cancer is excessive cola drinking. We could express the hypothesis in an if-then statement, such as "If people drink large amounts of cola, then they are more likely to develop cancer." This if-then hypothesis could be simplified into a single statement: "The cause of cancer is drinking too much cola." No matter how the hypothesis is formulated, it must be tested for its truthfulness because the casual observations alone are not enough to support it. 
Experimentation 
Experimentation, the third step of the scientific method, tests the hypothesis through any of various research methods, including the formal experiment. There are many ways to conduct these studies, each with its own advantages and disadvantages, as we discuss later. For instance, in our cola example, we could feed large quantities of cola to chimpanzees and after a while compare their cancer rates with those of a group of chimpanzees that did not receive cola. Or we could find human beings with a history of excessive cola consumption and compare their cancer incidence with that of humans who avoid such consumption. Once the experiment or data collection is complete, we move on to the last step of the scientific method, verification. 
Verification 
Verification is the analysis of data to see if that data support or dispute the hypothesis. In our example, we would analyze the results of our experiment to see if the excessive cola drinkers did indeed have a higher incidence of cancer. If they did, then our hypothesis would be supported (but not proven). If there was no difference between the groups, then we would have to go back to our first step to look for new observations or begin thinking about other cause-and-effect relationships that might explain our observations. This last step of the scientific method can be fortified through replication, which means running the study again, or some variation of it, to ensure that the results are reliable. It is especially helpful if other researchers replicate the results. Verification can also be fortified through prediction, which is the ability to use our study's conclusions to reliably predict other outcomes. 
These are the basics of the scientific method, a model of inquiry that is sometimes supplemented with hunches, intuitions, good luck, and creative play: 
"To our knowledge, no one has ever been able to grow neurons from the brain, probably from any animal, much less a human," said Dr. Solomon Snyder. . . . "We didn't expect it to work. We can't tell you why it did work. . . . We did it by diddling around, by being at the right place at the right time." (Bor, 1990) 
Thinking, Fourth Edition, by Gary R. Kirby and Jeffrey R. Goodpaster. Published by Pearson Prentice Hall. Copyright © 2007 by Pearson Education, Inc. 
The Scientific Method 223 
THINK ABOUT IT: While creating technological marvels and pro

ducing vast amounts of worthwhile information, the products of the 
scientific method have also created an ecological nightmare, extended 
humankind's ability to kill a thousandfold, and raised ethical issues 
that seem to transcend our capacity to answer them. 
Science and Other Ways of Knowing 
The scientific method can be further understood by distinguishing it from other ways of knowing, such as philosophy and appeal to authority. Like science, philosophy has systems for investigating the world, and the questions philosophers address may be inspired by a set of observations. However, philosophy differs from science in its greater emphasis on reason for solving problems as opposed to observation. The two also differ in the objects of their investigations. The domain of science is the world of observation, also known as the empirical world. Philosophy, on the other hand, often makes its inquiry outside the empirical world, investigating values, meaning, the nature of God, and so on. 
The scientific method can also be distinguished from appeal to authority. Many people seek knowledge by appealing to an authority figure. This figure may be a well-respected doctor, teacher, or religious book. The scientific method, however, is at great odds with this way of knowing. When we appeal to authority, we believe something is true because an authoritative figure said so, and we do not require a set of systematic observations to support it. During the Middle Ages, for example, the Catholic Church taught that all the heavenly bodies revolved around the earth. Most people accepted this teaching because it came from the Church's interpretation of an authoritative source, the Bible, and casual observation supported it: the heavenly bodies did appear to go around the earth. But from the scientific point of view this observation led only to a hypothesis, which was not tested scientifically. There were, as we know now, other explanations that would just as well have supported the observation that planets and stars appear to go around the earth. These explanations were not tested scientifically because the hypothesis was assumed to be true since it came from an authoritative source. When the Church's teachings were eventually challenged by Copernicus and Galileo, they were deemed heretical, not because they were at odds with observation, but because they were at odds with authority! 
ISBN:
Thinking, Fourth Edition, by Gary R. Kirby and Jeffrey R. Goodpaster. Published by Pearson Prentice Hall. Copyright © 2007 by Pearson Education, Inc. 
COPERNICUS AND GALILEO ISBN:COPERNICUS AND GALILEO ISBN:224 CHAPTER 10 ¦ Scientific Thinking 
THINK ABOUT IT: Is science at the end of its useful life? Will it ever find another physical law? Will it ever discover the roots of consciousness? Will it ever find out what happened before the big bang? Will it ever answer the big questions? Or will it just give us technical trivia about esoteric matters that will have no real impact on our lives? 
In the sixteenth century, Copernicus argued that the earth moved around the sun. His idea was contrary to the teachings of the Catholic Church, which believed that the celestial bodies revolved around the earth. Needless to say, Copernicus's teachings inflamed many Christians, including Martin Luther who considered him a fool who wanted "to turn the whole of astronomy upside down" (Crowther, 1969, p. 48). In 1616, sixty years after Copernicus's death, the Catholic Church, fearing a great scandal and dissent if Copernicus's views were taken seriously, put his text outlining a heliocentric (sun-centered) theory of the solar system on the Index of Prohibited Books. 
Although Copernicus is credited with introducing the heliocentric solar system, it was Galileo who was prosecuted for supporting such a view. Relying on scientific observation of the sun, planets, and stars, instead of on religious doctrine, Galileo found strong empirical support for Copernicus's theory and was unafraid to go public with his views. Even though forbidden by the church, Galileo published a book in 1632 supporting Copernicus's ideas and was consequently forced to stand trial for heresy. Found guilty, he was ordered to recant his views and was sentenced to house arrest, which remained in effect until he died eight years later. Such was the price of science. Only in 1992 did the Pope finally recant and admit that Galileo was right. 
This story shows that appealing to authority is not always going to yield a valid picture of reality, and it shows the power of our worldviews to inhibit our consideration of opposing beliefs, no matter what the evidence. In this case, Christians had a worldview that placed earth and human beings at the center of the universe. This view prevented them from thinking objectively about alternate views, even when the scientific evidence was substantial. This story also shows us the necessity for courage in our critical thinking, courage to abandon beliefs that make us feel safe and secure, and to stand up for an unorthodox view that may make us vulnerable to criticism from others. Without such courage, thinking cannot take the creative leaps often necessary for a breakthrough in knowledge. 
Thinking, Fourth Edition, by Gary R. Kirby and Jeffrey R. Goodpaster. Published by Pearson Prentice Hall. Copyright © 2007 by Pearson Education, Inc. 
The Empirical Nature of Science 225 
THE EMPIRICAL NATURE OF SCIENCE 
The world of science is the empirical world, the world of observation. In order to apply the scientific method, scientists must be able to make observations and measurements. Therefore, all variables under study in science must be defined in observable, measurable terms. By giving operational definitions to variables in this way, we make it clear to others what those variables are and what observations or measurements will indicate their presence. Physicists must decide what physical traces from an atom collision will indicate or define certain atomic particles. Astronomers must define a black hole in a way that they can recognize it when it is present in their observations of deep space. And psychologists must define variables such as love, frustration, and stress in such a way that they can be observed and measured. 
An example of a nonoperational definition is Webster's definition of love as "strong affection" and "warm attachment" toward another. Although this definition conveys to others the meaning of the term love, it does not indicate to others what observations or measurements are necessary to indicate the presence of love. With only Webster's definition in mind, with no observational measures to indicate its presence, imagine trying to ascertain the percentage of passersby who are in love. But if we define love as walking hand in hand with someone for at least sixty seconds, then we are defining love operationally and we would be able to observe and count the number of people passing by who are in love. But have we, in this case, defined love accurately? 
Erroneous Operational Definitions 
When variables are defined operationally, they are sometimes defined incorrectly. When this happens, the conclusions of the research may be in error. In medical research, for example, an operational definition of low-fat eaters has been based on a person's response to a questionnaire designed to determine how much fat a respondent eats now-not in the past, not in the future. That information is then used in studies twenty years later to see if low-fat eaters had, for example, more or less cancer than the high-fat eaters over the twenty-year period. Since people's eating habits do change, one can certainly question whether twenty years of low-fat eating can be adequately defined by only one questionnaire twenty years ago. 
As another example, consider a 1991 survey by the National Centers for Disease Control ("Good News," 1991). According to the survey, 45 to 75 percent of Americans have "sedentary lifestyles," sedentary being defined as "fewer than three 20-minute sessions of exercise each week." One can imagine the reactions of millions of parents whose days are completely filled by employment, childcare, and housekeeping responsibilities that include miles of walking, hundreds of flights of stairs, and lifting babies and heavy bags of groceries, with no time left over for a 
ISBN:
Thinking, Fourth Edition, by Gary R. Kirby and Jeffrey R. Goodpaster. Published by Pearson Prentice Hall. Copyright © 2007 by Pearson Education, Inc. 
ISBN:
226 CHAPTER 10 ¦ Scientific Thinking 
regular exercise program. We could hardly call these people sedentary! We can see that, despite the good intentions of scientists, sometimes the concept under study is one thing, whereas the operational definition of that concept is another. 
THINK ABOUT IT: In the above operational definition of love, the 
meaning of love may have been lost by defining it as handholding for 
sixty seconds. When we count handholding, are we really counting 
love? Are we missing anyone? A better definition might be to define 
love as a "yes" response to the question "Are you in love?" Can you 
think of a better operational definition? 
Operational Debates 
Operational definitions that are acceptable to everyone are sometimes very difficult to achieve. Such difficulty often leads to debate. One area of debate in psychology, for instance, is whether or not the hypnotized state is an alternate state of consciousness. First, researchers have to define in nonoperational terms what they mean by an alternate state of consciousness, and then they have to define this state in observable, measurable terms. Those who believe that hypnosis does not lead to an alternate state typically define alternate state operationally as a pattern of brain waves different from those of the waking state. They then point out that such brain wave change does not occur during hypnosis, and therefore hypnosis is not an alternate state of consciousness. Supporters of the alternate state theory might respond by challenging this operational definition. They might argue that it is possible for a person to be experiencing an alternate state of consciousness even though their brain waves indicate nothing more than normal waking consciousness. Given that possibility, critics of the alternate state theory of hypnosis could be relying on an invalid operational definition! 
The Limits of Science 
Without an operational definition, the scientific method cannot be employed. Science cannot, for example, tell us whether or not a biblical heaven or hell exists. Such metaphysical concepts are generally not reducible to operational terms. They lie outside the realm of observation and are best left to the areas of religion and philosophy. 
Besides metaphysical questions, questions of values and ethics also lie outside the domain of science. Consider the issue of abortion. Is abortion right, or 
Thinking, Fourth Edition, by Gary R. Kirby and Jeffrey R. Goodpaster. Published by Pearson Prentice Hall. Copyright © 2007 by Pearson Education, Inc. 
The Empirical Nature of Science 227 
is it wrong? The answer cannot be found through observation. Scientists cannot find an answer to this question by looking through microscopes, observing biological changes in laboratory dishes, or observing how human beings respond to the abortion issue. The question of abortion is one of values, and although science can give us information that can be useful in answering such a question- for example, ascertaining when the heart starts beating in a fetus-it cannot by itself evaluate ethical statements. Value questions lie within the realm of religion and philosophy and outside the realm of science. 
Consider the value statement "It is wrong to kill human beings for any reason but self-defense." Can you imagine any scientific way to support or refute such a statement? Where would we look for the answer? Perhaps, you might say, in our emotions, for most human beings find killing emotionally repulsive. But how do you determine that human emotion should be the criterion for determining values? Is that a scientific fact or a philosophical statement? No kind of scientific observation could possibly tell us that human emotion is the criterion for determining values. Once again we are back to the realm of philosophy and have left the domain of science. 
In short, scientific investigation is a magnificent procedure for unlocking many secrets of our world, but it does have limitations and may never, as Schopenhauer put it, "reach a final goal or give an entirely satisfactory explanation" of our world (Schopenhauer 1859/1958a, p. 28). Carl Jung echoed this sentiment in an interview on his eightieth birthday: "True reality can only be approached and surmised spiritually" (Jung, 1957/1977). In the words of Wittgenstein, "We feel that even when all possible scientific questions have been answered, the problems of life remain completely untouched" (Wittgenstein, 1961, Tractatus Logico-Philosophicus, 6.52). 
THINKING ACTIVITY 10.1 Creating Operational Definitions If you were proposing to study the following variables, which ones could you operationally define? Which ones could not be so defined? The key to determining your success is to ask, "What could I observe that would indicate the presence of the variable?" and "Could my definition be defining another variable instead?" Try your definitions out on others to see if there is agreement that your definition indeed defines the term without losing its meaning. (continued)
ISBN:
Thinking, Fourth Edition, by Gary R. Kirby and Jeffrey R. Goodpaster. Published by Pearson Prentice Hall. Copyright © 2007 by Pearson Education, Inc. 
228 CHAPTER 10 ¦ Scientific Thinking 
THINKING ACTIVITY 10.1 (Continued) 1. Frustration 12. Gravity 2. Obesity 13. Telepathy 3. Aggression 14. Evolution 4. Soul 15. Pain 5. Scientist 16. God 6. Depression 17. Immoral behavior 7. Thumb sucking 18. Prejudice 8. Migraine 19. Meditation headache personality 9. Multiple 20. Hypnotized 10. Nothing subject 11. Black hole 21. Psychological stress 22. Altruistic behavior 23. Happiness 24. Life 25. Consciousness 26. Thinking 27. Death 28. Beginning of human life 29. Intelligence 30. Heaven 
THINKING ACTIVITY 10.2 (continued) The Domain of Science For which of the following questions would science be the appropriate method of investigation? Indicate your answer by putting "S" to the left of those questions. ________ 1. Do human beings have free will? ________ 2. How can we reduce pollution in the environment? ________ 3. Is there life on other planets? ________ 4. Considering the physical and psychological changes that occur, is a person the same person from birth through old age? If so, why? ________ 5. Does God exist? ________ 6. At what point in fetal development do brain waves occur? ________ 7. When does human life start? ________ 8. What is life? ________ 9. What principles should guide a person's behavior toward others? ________10. What is the origin of the human race? ________11. How can we increase our longevity? ________12. What was Shakespeare's purpose in writing Romeo and Juliet? 
ISBN: 
Thinking, Fourth Edition, by Gary R. Kirby and Jeffrey R. Goodpaster. Published by Pearson Prentice Hall. Copyright © 2007 by Pearson Education, Inc. 
Science and the Understanding of Human Nature 229 
THINKING ACTIVITY 10.2 (Continued) ________13. What is the human mind? ________14. Should human beings be punished for evil deeds? ________15. Are human beings basically good, or are they basically evil? ________16. What is beauty? ________17. Does stress cause most cases of depression? ________18. Does wearing a seat belt decrease the incidence of highway fatalities? ________19. Does drinking milk before bedtime aid sleeping? ________20. Is there life after death? ________21. What is intelligence? 
SCIENCE AND THE UNDERSTANDING 
OF HUMAN NATURE 
A scientific conception of human behavior dictates one practice, a philosophy of personal freedom another. 
-B. F. SKINNER, SCIENCE AND HUMAN BEHAVIOR 
Because of the remarkable success of the scientific method in understanding the material universe, psychologists and sociologists have applied scientific thinking to the understanding of the psychological and sociological dimensions of human beings. From a philosophical viewpoint, this scientific thinking rests upon a foundation of determinism, which leads to some interesting problems when applied to the study of human beings. Below we explore some of these problems through a discussion of determinism. 
Determinism as Foundation 
Scientists seek not only to discover phenomena but to discover the order underlying various phenomena-that is, the cause-and-effect relationships between things. The psychologists' lengthy surveys, the biologists' dish cultures, and the physicists' atomic accelerators are all designed to discover the components of nature and the laws that govern the actions of these components, whether those components are human beings, tsetse flies, or atomic matter. Most scientists assume that the world is orderly, predictable, and operating through complex mechanisms of cause and effect. In other words, they assume 
ISBN:
Thinking, Fourth Edition, by Gary R. Kirby and Jeffrey R. Goodpaster. Published by Pearson Prentice Hall. Copyright © 2007 by Pearson Education, Inc. 
ISBN:
230 CHAPTER 10 ¦ Scientific Thinking 
a deterministic universe. If the world were not determined, but completely chaotic, scientific investigation could not lead to the discovery of natural laws. 
There is considerable debate among philosophers and scientists about the extent of determinism, but most agree that for the macrocosmic physical universe determinism is a valid description of events. The debate centers on the role of determinism in the microcosmic world of particle physics and in the behavior of human beings. We will concern ourselves with the latter. 
Human Beings and Determinism 
Social scientists are concerned with the understanding and control of human behavior in order to promote optimal social and psychological functioning. The assumption behind this concern, in whole or in part, is determinism: Social scientists assume that genetic, psychological, and social forces in each person's history govern the character and behavior of each individual. Although not all psychologists adhere to a deterministic view of human nature, their dominant tendency to look for explanations of human behavior by examining past events seems to assume such a view, especially when the general goal is to discover laws or principles that may govern human behavior. Here is an example: 
STUDENT: Why did Mark become a psychopath? 
PROFESSOR: Well, our answer lies in the genetic, social, and psychological forces that shaped Mark through his early development. Interestingly, Mark's father was also a psychopath and might have passed on some "psychopathic genes" to him. Moreover, being a psychopath, Mark's father did not teach him a healthy value system and actually served as a negative role model for Mark. Mark's mother, of course, had to work ten hours a day, six days a week, because Mark's father was often unemployed and was not a reliable source of income. Consequently, Mark's mother was not around to help shape Mark's values either and never really formed a strong bond with him. She often neglected Mark and physically abused him when she was under stress, which was more often than not. Sadly, there was never any sign of affection expressed toward him at all. And that's why Mark became a psychopath! 
If Mark's behavior is shaped by his genetic constitution and his psychological and social environment, can he be held responsible for anything that he does? Hard determinists must answer "no," for they believe that every element of Mark's behavior, including the choices, judgments, and assessments underlying that behavior, are nothing but the result of a complicated chain of cause-and-effect relationships: 
Thinking, Fourth Edition, by Gary R. Kirby and Jeffrey R. Goodpaster. Published by Pearson Prentice Hall. Copyright © 2007 by Pearson Education, Inc. 
Science and the Understanding of Human Nature 231 
In the mind there is no absolute or free will; but the mind is determined to wish this or that by a cause, which has also been determined by another cause, and this last by another cause, and so on to infinity. (Spinoza, Ethics, Part II, Prop. XLVIII) 
Hard determinists argue that if all the variables about Mark were known, they would be able to predict his behavior with perfect precision. The reason they cannot ever predict perfectly what someone will do is not because people are free, but because they never know all of the variables bearing on that behavior. Thus, they talk about probabilities. They say that, given a certain kind of parenting style and a certain social environment, the chances are good that someone will become a psychopath. But rarely, if ever, can they be certain. 
Opposed to determinism are the indeterminists, who believe that even though much of our life is shaped by genetic and psychosocial forces, there is still an element of free will behind our behavior. We are free in the sense that we could have done otherwise, but we chose not to, and thus we are accountable for our actions. 
If the indeterminists are right about our freedom, can scientists ever understand and predict our behavior? Some philosophers argue that prediction and freedom are not incompatible. For example, you may know your friend well enough to know how he would choose to behave in a given situation. Thus, even though he is acting freely, you can predict his behavior. Then again, maybe you can't: 
I have observed instances of a person deliberately upsetting the predictions simply to reaffirm his unpredictability and therefore autonomy and self-governance. For instance, a ten-year-old girl, known for being always a good citizen, law-abiding and dutiful, unexpectedly disrupted classroom discipline by passing out French fried potatoes instead of notebooks simply because, as she later said, everyone just took her good behavior for granted. A young man who heard his fiancée say of him that he was so methodical that she always knew what to expect of him, deliberately did what was not expected of him. Somehow he felt her statement to be insulting. (Maslow, 1966, p. 42) 
Would any scientist have been able to predict that the ten-year-old girl would hand out french fries? The determinist, of course, would argue that an inability to predict the girl's behavior only reflects the complexity and enormity of the variables behind that behavior and in no way undermines determinism. Nonetheless, most of us would agree that a strong sense of freedom is evident in the example above. So strong is that sense of freedom in our lives that it may be more the burden of the determinists to show that we are not free than it is of the indeterminists to show that we are. 
The point is that social scientists generally act under the assumption of determinism when they look for the causes of human behaviors and thoughts, even though their subjects feel free and are held responsible for their acts by others. For example, when Mark the psychopath murders, we react with outrage, not against a psychosocial system that made Mark what he is, but against Mark 
ISBN:
Thinking, Fourth Edition, by Gary R. Kirby and Jeffrey R. Goodpaster. Published by Pearson Prentice Hall. Copyright © 2007 by Pearson Education, Inc. 
DETERMINISM AND PROBABILITY ISBN:DETERMINISM AND PROBABILITY ISBN:232 CHAPTER 10 ¦ Scientific Thinking 
himself as though he is responsible for what he did, as though he could have chosen otherwise. Thus we have a contradiction between the deterministic assumption of social scientists which absolves Mark from responsibility and the reaction of the world in general which presupposes Mark's responsibility and freedom. 
B. F. Skinner, a prominent determinist, explained the contradiction this way: All of this suggests that we are in transition. We have not wholly abandoned the traditional philosophy of human nature [that we are free]; at the same time we are far from adopting a scientific point of view that our behavior is determined without reservation. We have accepted the assumption of determinism in part; yet we allow our sympathies, our first allegiances, and our personal aspirations to rise to the defense of the traditional view [of human freedom]. (Skinner, 1953, p. 9) 
THINK ABOUT IT: In his statement below B. F. Skinner explains the contradiction between the deterministic assumption of social sci-ence and our general assumption of free will as due to an inability to fully embrace determinism because of our loyalty and attraction to the idea of free will. Do you agree? Is our "love" of free will and its impli-cations getting in the way of straight thinking about it? Or are there solid reasons for defending free will? Although we have not determined the extent of human free will, we can say with some confidence that the precision of social science may be limited by the extent of it: the greater our freedom, the less the rule of cause and effect applies and the more difficult human behavior is to predict and control. And if human freedom exists at all, then perhaps the goal of social scientists ought to be to encourage it. As Maslow (1966) wrote, "If humanistic science may be said to have any goals beyond sheer fascination with the human mystery and enjoyment of it, these would be to release the person from external controls and to make him less predictable to the observer" (p. 40). 
Although determinism underlies the scientific work of the physical and social scientists, many scientists work more with the concept of probability than with the concept of determinism. Probability is concerned with the likelihood of a particular event occurring in a particular situation. In quantum 
(continued) 
Thinking, Fourth Edition, by Gary R. Kirby and Jeffrey R. Goodpaster. Published by Pearson Prentice Hall. Copyright © 2007 by Pearson Education, Inc. 
Proving a Theory 233 
DETERMINISM AND PROBABILITY (Continued) mechanics, for example, physicists work with Werner Heisenberg's uncertainty principle. This principle states that, because atomic particles are so small, the methods we use to observe and measure those particles change them. Therefore, physicists can observe and measure the exact momentum of a particle, but they cannot simultaneously determine its exact velocity because that would have been changed by the act of observing momentum, and vice versa. If physicists want to know both momentum and velocity, they can only make statements about the probability that a particle will fall within a particular range of values; for a given probability, the smaller the range for one value, the larger the range for the other. 
Social scientists also deal with probability. Much of their research is done with groups of people as they compare the average value of one group with that of another. If, for example, they find that a group of people exposed to noisy working conditions have poorer marital relations than a similar group that is exposed to quiet working conditions, they might conclude that exposure to a noisy work environment leads to poor marital relations. However, not everyone in the noisy conditions is necessarily going to be affected that way. There are usually exceptions in every group. Thus, for a given individual who is exposed, we can talk about probable effects on marital relations, not certain effects. 
Do these probability theories undermine determinism? Not necessarily. Einstein, for example, thought that the uncertainty principle reflected the limitation of our ability to measure atomic particles, not an inherent indeterminism of atomic elements. Similarly, the work of social scientists with probability may only reflect their limitation in assessing all of the important characteristics of each individual; for although a group may have similar people, they do not have identical people. Those unmeasured differences foil our ability to predict behavior with certainty. 
PROVING A THEORY 
We cannot pretend to offer proofs. Proof is an idol before whom the pure mathematician tortures himself: In physics, we are generally content to sacrifice before the lesser shrine of plausibility. 
-A. EDDINGTON, "DEFENSE OF MYSTICISM" 
Imagine being a traveling nurse who cares for leukemia patients in a large city. After several months and hundreds of patient visits, you notice that many of your patients live very close to high-voltage power lines. You at once suspect that the magnetic fields from those power lines may be the cause of leukemia in your patients. At this 
ISBN:
Thinking, Fourth Edition, by Gary R. Kirby and Jeffrey R. Goodpaster. Published by Pearson Prentice Hall. Copyright © 2007 by Pearson Education, Inc. 
ISBN:
234 CHAPTER 10 ¦ Scientific Thinking 
point you have formed a hypothesis, a tentative statement of the relationship between events generally based on casual observation. Now you proceed scientifically. You look up the addresses of all your patients and find that 85 percent of them live within 1,000 feet of high-voltage wires or an electrical transformer, both of which generate a strong magnetic field. You then go to the city health department and collect addresses of all leukemia victims in the last ten years and find that most of them lived in urban areas, which is where high-voltage lines are more common. Furthermore, you find that as distance from high-voltage lines decreases, leukemia rates drop. You then write an article and state your theory that strong magnetic fields cause cancer in children at rates directly related to the strength of the magnetic field. 
Have you proven your case? No! There are other possible explanations for the results. For example, the fact that most of the cases come from urban areas may be nothing more than a reflection of the distribution of the population in the United States: Most cancers are in urban areas because that is where most people live. The higher rate of leukemia near power lines can also be explained in other ways: it could be that the majority of high-voltage lines run through the more industrial parts of the city, and the industrial pollutants could be responsible for the leukemia. Or if high-voltage power lines are more common in urban areas, then it could be the stress of urban life, and not power lines, that is responsible for the disease-and so on. 
THINK ABOUT IT: Can you think of other explanations for the 
leukemia results that are not connected to high-voltage lines? 
Suppose you continue your research, this time using laboratory rats, and you find that the rats exposed to high voltages develop leukemia. And suppose your research is so good that we just cannot think of any explanation for your results except the theory that you have formulated. Have you then proven your theory? You may have a strong case, for you would certainly have corroborating evidence, but you would not want to say that you have proven your theory. Scientists generally do not like to use the words proven and proof (despite how often you hear them in commercials), for even though no other explanation for one's results is available, there might still be one. As the Viennese philosopher Karl Popper (1965) put it, "The demand for scientific objectivity makes it inevitable that every scientific statement must remain tentative for ever" (p. 280). 
Ultimately, proof must remain subjective, for the amount of data necessary to convince one person of the validity of a theory may be insufficient to convince another. How do we determine, to everyone's satisfaction, how much and what kind of data are necessary to prove a particular theory? Certainly if someone 
Thinking, Fourth Edition, by Gary R. Kirby and Jeffrey R. Goodpaster. Published by Pearson Prentice Hall. Copyright © 2007 by Pearson Education, Inc. 
Controlled Experiments 235 
proposed a theory that challenged our worldview, we would want more data than if that person proposed a theory that did not usurp conventional beliefs. As we saw in Chapter 2, people are very resistant to changing their worldviews; a great deal of data would be required to do so. But how much? 
One scientist who challenged our worldview was Albert Einstein. His theory of the universe confronted basic beliefs about space and time. His idea, for example, that time is relative, that two individuals traveling at different speeds through the universe age at different rates, or that a person at the base of a mountain ages at a different rate from a person living at the top, seriously disturbs the commonsense view of time held by most people even today. Nonetheless, his theory has been supported by numerous experiments throughout the last several decades. Has his theory been proven? It certainly is convincing to many physicists and mathematicians; however, because proof is subjective, with each individual requiring more or less data or different kinds of data, some people probably won't accept relativity as proven until they actually travel through the universe at different speeds. 
In the rest of this chapter we explore some of the ways that science, particularly the social and medical sciences, can make progress toward "proof," and we illustrate in more detail why it is important for researchers and consumers to be cautious about jumping to hasty conclusions. 
THINK ABOUT IT: Relatively few people fully understand Einstein's theories of relativity. How can a layperson become convinced of the validity of his theories? CONTROLLED EXPERIMENTS A controlled experiment, also called experimental design or true experiment, is any research design that allows the experimenter to control the variables in an experiment so that the results of the experiment can better establish a cause-andeffect relationship. A controlled experiment requires at least one control group and one experimental group. The control group is the comparison group; the experimental group, also called the treatment group, is the group that receives the treatment. Theoretically, the two groups are identical except for the treatment. For example, if a chemist wants to find out if a new chemical added to ordinary detergent will improve its cleaning efficiency, one group of clothes will be washed with the new chemical (experimental group), while the other will not (control group). Both groups must have the same kinds and amount of clothing, with the same amount and type of stain, and must be washed in the same
ISBN:
Thinking, Fourth Edition, by Gary R. Kirby and Jeffrey R. Goodpaster. Published by Pearson Prentice Hall. Copyright © 2007 by Pearson Education, Inc. 
ISBN:
236 CHAPTER 10 ¦ Scientific Thinking 
amount of water, at the same temperature, in the same kind of machine, and so on. Except for the added chemical, the washing conditions must be identical. If this identity has been achieved, we have a well-controlled experiment, and if the experimental group comes out cleaner, the chemist shall have reason to be excited about the cleaning power of his chemical. 
To illustrate the importance of having control over all the variables, consider a fictitious study that is not well controlled: Farmer Smith wants to find out if a special additive that he has developed will increase the number of eggs his chickens will lay. To begin, he randomly separates his 500 chickens into two groups. He does this randomly because he has learned, correctly, that randomization is usually the best procedure for dividing groups when one has a large number of subjects. If he divides his chickens by size, or age, or activity level, he will have two groups that are not identical. His randomization reasonably ensures that his two groups of chickens have about the same number of active and passive chickens, average age, average size, and so on. In fact, his randomization reasonably assures him that his two groups are the same on variables he hasn't even thought about, such as diseases, appetites, genetics, and so forth. Any slight differences between his two groups should not be statistically significant. 
So far Farmer Smith is proceeding okay. He now puts one group of chickens in a hen shack near the farmhouse, and the other in an existing hen shack about 500 feet away. The chickens near the farmhouse become his experimental group and the chickens in the other shack become the control group. Farmer Smith then puts one cup of his secret ingredient in each five-gallon bucket that he uses to supply water for the chickens in the experimental group. For the chickens in the other hen shack he uses regular water. 
During the three months that Farmer Smith conducts his experiment, he keeps careful records of how many eggs each coop delivers each day. When his study is complete, he finds that the experimental group laid 25 percent more eggs on the average than the other group. Absolutely delighted, he finally shares his secret study with his wife and son at the dinner table one evening. Smug and proud as he finishes his story, he sits back waiting for the praise. His son looks puzzled and queries his father. "Dad, maybe the chickens in the distant hen shack didn't lay as many eggs because they were stressed out by that fox. He doesn't come up near the house much, but you know he tries to get at the chickens in the other shack near the road." "Or maybe it was the noise," said his wife, "That distant coop sets pretty close to the highway you know. All those trucks screaming by all day and night have to have some effect on those chickens, don't you think?" "Or the sun," said his son. "That one coop sits out there with no protection from the heat, while the one by the house is shaded most of the time. Maybe chickens that are cool and comfortable lay more eggs." "Just a thought," added his son. "Yea, just a thought," 
Thinking, Fourth Edition, by Gary R. Kirby and Jeffrey R. Goodpaster. Published by Pearson Prentice Hall. Copyright © 2007 by Pearson Education, Inc. 
Nonexperimental Designs 237 
said his wife. Farmer Smith, looking dejected, excuses himself as he slowly gets up from his chair to find a quiet room to plan his secret ingredient research-again. 
We now know that Farmer Smith didn't control enough variables to be confident that the differences between his groups were due to his secret ingredient and not the noise, fox, or heat conditions that also differed between his two groups. Much to the embarrassment of researchers, such as Farmer Smith, someone often develops a competing explanation for their results. The experiment must then be run again, this time controlling for the variables that slipped by last time. And so it goes. 
So next time you hear a commercial claim that says something like "Rinsing with our mouthwash before bedtime kills twice as many odor-causing germs as brushing alone," think! What other variables could explain the results? The researchers compared their product to brushing without rinsing, but did they compare their product to brushing and rinsing with water before bedtime? Perhaps it is rinsing with a fluid that's important in the removal of germs and not the expensive product's "special ingredients." 
QUASI-EXPERIMENTAL DESIGN 
In Farmer Smith's study, he correctly divided his subjects into two groups randomly. This is not always possible, however. For example, if we want to introduce a new teaching strategy for learning math, we might find that some schools are cooperative and others want nothing to do with our new technique. Thus, our experimental group becomes a sample of convenience instead of a randomly chosen group from the community. Whenever we select our two groups through nonrandom means we have a quasi-experimental design, as long as the researcher is still the one who makes the groups different by "treating" one group and not the other. Quasi-experimental designs, lacking randomness, are a little more vulnerable to the possibility that the two groups were not equal at the outset of the study. The class receiving the new teaching style, for example, might be different from the control class at the outset because it came from a different school environment, a different neighborhood, and perhaps a different socioeconomic class. These differences might explain any outcome differences. 
NONEXPERIMENTAL DESIGNS 
The experimental and quasi-experimental designs are not always appropriate or practical. Thankfully, there are many other research designs available. These other designs are often referred to as nonexperimental designs. These include, 
ISBN:
Thinking, Fourth Edition, by Gary R. Kirby and Jeffrey R. Goodpaster. Published by Pearson Prentice Hall. Copyright © 2007 by Pearson Education, Inc. 
ISBN:
238 CHAPTER 10 ¦ Scientific Thinking 
but are not limited to, the expost facto design, the correlational design, the survey method, and the case study. 
Ex Post Facto Design 
For reasons we will soon explain, researchers sometimes use an ex post facto design. In fact, it may very well be the most common design underlying the science news we receive from the popular media. In the ex post facto design researchers find the groups that have differences. In other words, the researcher walks onto the scene after the treatment conditions have been created, thus the name "ex post facto" or "after the fact." For example, using the ex post facto design in order to discover the effects of meditation, we would find meditators and nonmeditators. After finding the two groups, we would measure them on another variable such as emotional stability. If the meditation group is found to be more emotionally stable than the nonmeditation group, we might conclude that meditation increases emotional stability. The problem with this method, however, is that groups that are found generally have other differences between them besides the variable being studied, and those differences may be the real explanation for any outcome differences. In the example above, meditation may not be the only difference between the two groups. If it is not, how are we to know if it is meditation that causes emotional stability or if it is one of the other differences? 
What other differences might there be? Perhaps people who meditate are more educated than people who do not. Or maybe people who meditate also tend to be vegetarians. Or maybe people who choose to meditate have more leisure time and less stress. Or maybe they are spiritually inclined, which leads them to take up meditation. Who knows? The point is, any one of these other variables, called hidden variables, could explain the outcome differences between the two groups. In other words, education, vegetarianism, leisure time, or spiritual inclination could have been responsible for the increased stability of the meditation group and not the meditation itself. 
Whenever we find differences between groups instead of creating differences, we run into the problem of hidden variables. Scientists attempt to control for the variables that may obviously explain the differences by, for instance, making sure that the meditators and nonmeditators both eat meat, have the same level of education, and so forth. But what about intelligence, drug abuse history, or early family experiences? Research cannot always identify and control all the potentially important variables. Therefore, one or more of these uncontrolled variables might be the real reason for the outcome differences. 
Suppose you hear about an ex post facto study that compared vegetarians with nonvegetarians and found that vegetarians live longer. Most people uninformed about ex post facto designs would quickly conclude that adopting a 
Thinking, Fourth Edition, by Gary R. Kirby and Jeffrey R. Goodpaster. Published by Pearson Prentice Hall. Copyright © 2007 by Pearson Education, Inc. 
Nonexperimental Designs 239 
vegetarian diet will increase longevity. But the cause-and-effect relationship is not obvious in ex post facto designs. For example, it could be that vegetarians happen to be the kind of people who care about their health more than the typical nonvegetarian, and they not only avoid meat but also avoid alcohol and tobacco more than nonvegetarians, and they tend to see their doctor more frequently for regular check-ups. So is it vegetarianism that leads to longevity or the other good health habits that tend to accompany vegetarianism? 
Another problem with ex post facto designs is determining the direction of cause and effect. To illustrate, let's consider a study that tries to find out if people who are satisfied with their marriage have higher work motivation than people who are not. Obviously, we would have to find our subjects as it would be quite "challenging" to make one group satisfied with their marriage and another dissatisfied. Suppose we find that the group of subjects that was satisfied had higher work motivation than the other group. Does this mean that marital satisfaction somehow leads to more positive work attitudes, or is it that industrious people tend to put more energy into their marriages, consequently creating more marital satisfaction? In this case, aside from any hidden variable problems, the direction of cause and effect is not clear. But sometimes we can weed out a cause-and-effect direction. For example, an ex post facto design comparing youth and elderly on their differences in bone density, or atheists and Catholics on their longevity, will enable us to weed out one direction of cause and effect: Changing bone density will not make a person elderly, and living long will not make one Catholic! Nonetheless, hidden variables may still confound our interpretation and make it difficult to know precisely what cause-and-effect relationships exist to explain our results. 
So why use ex post facto designs instead of the more controlled experimental designs when the latter is better at determining cause and effect? Sometimes it is simply impractical to use the experimental design. Would you want to be assigned to an experimental group and asked to eat no meat for the next 25 years? Other times the experimental design does not represent the real-world situation adequately. Having children watch an hour of violent television in a laboratory might not represent the hours and hours of television viewing they experience in their real life. Lastly, the ethical problems of running controlled experiments are often grave. To study the effects of child abuse using the experimental approach would require the researcher to abuse the children in the experimental group. Finding abused kids and studying them is the only moral choice. 
Arguably, ethical problems are avoided by using animal subjects instead of humans. Granted, we cannot use animals to study child abuse, but a controlled experimental design looking for the possible cancer-causing effects of a particular drug could be conducted with animal subjects. However, there is the problem of generalization when using animal subjects instead of human ones. Generalization is the assumption that what is true of the sample is true of the 
ISBN:
Thinking, Fourth Edition, by Gary R. Kirby and Jeffrey R. Goodpaster. Published by Pearson Prentice Hall. Copyright © 2007 by Pearson Education, Inc. 
ISBN:
240 CHAPTER 10 ¦ Scientific Thinking 
larger population under study. To assume that a drug that causes cancer in laboratory mice also causes cancer in humans is to make a statement of generalization. Given the differences between human beings and animals, there is room for skepticism about such generalizations. 
THINK ABOUT IT: Disregarding the problems of generalization and 
ethics, can you think of any topics for social science research in which 
we could not use animals? 
Correlational Design 
Most of us have heard of the relationship between crime and unemployment, and between vegetable consumption and lowered risk of cancer. We've all been warned about the relationship between stress and health problems, and we might have heard that the more male children a mother has, the greater the chance is that the next child will be homosexual (Purcell, Blanchard, and Zucker, 2000). Much of this information comes from correlational research. 
The correlational research method is very similar to the ex post facto design. In both designs the investigator finds the variable under investigation; the researcher does not create it. However, unlike the ex post facto design, the correlational design looks for the degree of relationship between two or more variables, instead of examining differences between groups. The kinds of variables that can be studied are innumerable: from human stress and happiness, to solar magnetic fields and the earth's temperature. If a relationship exists between two variables, then as one variable moves the other will also. For example, we could collect data from a large group of people about how much meat they typically eat in one week. At the same time, we could administer a test to assess the level of iron in their blood. If people who ate more meat tended to have higher blood iron levels, then we have established a positive correlation. If we found that people who ate more meat had lower blood iron levels, then we have established a negative correlation. But the question we must always ask is, "How strong is the relationship between the two variables?" If a correlation exists, it can range from very weak to very strong. If there was only a very weak positive relationship between meat eating and iron levels, there would be little basis for altering one's eating habits. On the other hand, if the relationship was strong, it might be well-advised to eat more meat if one wanted to raise one's blood iron levels. 
If there is no correlation between two variables, we can be certain that there is no cause-and-effect relationship between them, for all cause-and-effect relationships are correlational ones. But if a correlation is found, there will 
Thinking, Fourth Edition, by Gary R. Kirby and Jeffrey R. Goodpaster. Published by Pearson Prentice Hall. Copyright © 2007 by Pearson Education, Inc. 
Nonexperimental Designs 241 
generally still be questions, as in the ex post facto design, about the direction of cause and effect, or even if a cause-and-effect relationship exists, because correlational designs are not meant to determine cause and effect! For example, if we find a strong correlation between aggressiveness and viewing violent television, does that mean that watching violent TV causes aggressive behavior? Or is it that aggressive personalities choose to watch more violent television? Or both? Or neither? We could have a good correlation with any of these possibilities. Perhaps more abusive parents, for example, let their children watch violent television more than less abusive parents, and it is the abuse that causes aggressiveness, not television. From this simple example, we can see how correlational studies can demonstrate a relationship between variables but can determine little about any cause-and-effect interactions. On the topic of violent television, the body of research over several decades, using a variety of research designs, indicates that there is a cause-and-effect relationship between viewing violent television and aggressive behavior, and the direction of cause and effect goes both ways. 
Sometimes common sense and advanced statistical methods can help to clarify the directional problem. In one of the examples above, we could probably rule out the possibility that blood iron levels cause meat eating. Could we then conclude that meat eating raises iron levels if we have found a positive correlation between meat eating and iron levels? Not necessarily, for as we have just seen in the ex post facto design, a third variable might explain the relationship. If for some reason meat consumption correlates with potato consumption, physical activity, smoking, or alcohol consumption, it could be one of those variables and not meat eating that is responsible for changes in iron levels. 
Given all these problems of correlational designs, why do we have these kinds of studies at all? One reason is that correlational designs yield statistics about the degree to which two variables are related-that is, the degree to which they correlate. The higher the degree of correlation, the more precisely we can predict one of the variables if we know the other-and we can do this without knowing anything about cause and effect. Life insurance companies, for instance, use correlations when they assess the gender and health habits of new subscribers to determine death potential and insurance risk. And college admissions committees use ACT and SAT scores because they correlate somewhat with academic success. Another reason for using correlational designs is the same as for using ex post facto: It avoids the ethical and practical problems of using experimental approaches. 
THINK ABOUT IT: Could a controlled experiment be used to deter-mine if meat eating raises blood iron levels?ISBN:
Thinking, Fourth Edition, by Gary R. Kirby and Jeffrey R. Goodpaster. Published by Pearson Prentice Hall. Copyright © 2007 by Pearson Education, Inc. 
242 CHAPTER 10 ¦ Scientific Thinking 
ISBN:
The Survey Method 
One of the most convenient and relatively inexpensive ways to gather data for research is through a survey. A survey is simply an instrument with questions designed to assess our attitudes and opinions about various issues. This instrument can be given to a subject through an oral or written medium. We have probably all experienced the oral method when we were approached in a shopping mall by someone who wanted to ask us a few questions about a new product, or when we answered a phone call from an independent research firm with questions about our leanings in an upcoming election. 
Without the survey method it would be difficult or impossible to acquire information about people's beliefs, attitudes, and opinions. We might be able to 
THINKING ACTIVITY 10.3 Determining the Research Design The following are titles of real news articles. Place an "E" before those articles in which you think the researcher used an experimental or quasi-experimental study and a "C" before those in which you think the researcher used the ex post facto or correlational design. In those cases in which you think either approach might have been used, you may indicate both "E" and "C." To help you with your thinking, ask yourself if an experimental approach to these topics, in which the researcher creates the treatment differences between groups, would be unethical or extremely impractical. ________ 1. Lead Exposure, Laziness Linked to Alzheimer's ________ 2. More Evidence That Smoking Moms Have Smoking Kids ________ 3. Vegetables Lower Prostate Cancer Risk ________ 4. Vitamin E May Help Ease Menstrual Cramps ________ 5. Acupuncture Helps Relieve Depression ________ 6. Hormones in Womb Linked to Sexual Orientation ________ 7. Women with Breast Implants Have Higher Suicide Risk ________ 8. Brain Pattern Differs in Boys with ADHD ________ 9. Loss of Parent Tied to Mental Illness ________10. Special Glue Assists Nerve Repair ________11. Moderate Drinking Tied to Arterial Disease ________12. Smoking During Pregnancy Linked to Child Psychiatric Disorders ________13. Fish Oil Found to Ease Manic Depression ________14. Overwork Only One Cause of Job Burnout ________15. Poor Parenting May Create Disruptive Children 
Thinking, Fourth Edition, by Gary R. Kirby and Jeffrey R. Goodpaster. Published by Pearson Prentice Hall. Copyright © 2007 by Pearson Education, Inc. 
Nonexperimental Designs 243 
infer some attitudes and beliefs by observing someone's behavior, but much of a person's subjective life cannot be reliably measured by this method. However, with the survey method a person's political affiliation could be operationally defined as a "yes" response to the question "Are you a Republican?" The survey technique transfers a political view, which we cannot see and measure, into an oral or written response, which we can observe and measure. Some information would otherwise be nearly impossible to accurately define operationally, such as people's daydreams, their conception of God, their attitude toward gun control, their sexual behavior, or their worst regrets. 
Surveys are a very popular method for accumulating data on large numbers of people, principally because of their relatively low costs, ease of administration, and ability to assess personal information and private experiences. However, surveys must meet four conditions for them to be efficient research tools: (1) They must be administered to people in a way that encourages honesty; (2) the questions must be clearly stated, and asked objectively-that is, without bias in one direction or another; (3) they must reach a representative sample of the population being studied; and (4) they must be returned in an unbiased manner. If any of these conditions is not met, the survey results become invalid. 
The best way to encourage honesty in survey responses is to assure the person that his or her answers will remain anonymous. This is especially important when dealing with sensitive topics like sexual behavior, childhood sexual abuse, and problems of addiction. A face-to-face interview by a stranger about sexual habits does not provide the anonymity required for honest answers. 
Anonymity alone, however, does not ensure valid results if the questions are asked in a biased manner-that is, by pressuring, intimidating, or otherwise indicating how they are to be answered. One "State of the Nation" survey violated the rules of anonymity and unbiased questioning when it instructed respondents to return the survey with their name on it and prefaced the survey with a four-page letter on how to vote. Part of the letter appears below: 
May God strengthen you as you continue to speak out against abortion, homosexuality, communism, pornography, anti-Christian TV programming and secularism in government. . . . There are more well-funded liberal activists than ever at work on Capitol Hill. They smell victory in the making because it appears on the surface that the Christian agenda has been defeated. . . . You and I must show them they are wrong. . . . Radical feminist groups, the American Civil Liberties Union, and People for the American Way would like nothing more than to see ministries like Coral Ridge silenced. 
A few years later the Democratic National Committee mailed a seven-page survey with sensitive political questions as part of its Democratic Party membership acceptance form. At the top of each page of the survey was printed the respondent's name! 
ISBN:
Thinking, Fourth Edition, by Gary R. Kirby and Jeffrey R. Goodpaster. Published by Pearson Prentice Hall. Copyright © 2007 by Pearson Education, Inc. 
ISBN:
244 CHAPTER 10 ¦ Scientific Thinking 
One would think that academic institutions would be one place where good surveys would be found, since this is where survey design is taught. But even in these institutions bad surveys are generated. It is not uncommon for college administrators to send surveys to faculty with questions about the respondent's age, sex, race, department, and highest degree. Obviously, in small departments such information is devastating to anonymity. 
Survey bias can also be found in academic arenas. One college survey on the topic of student retention asked respondents to rate the potential of various programs to improve retention from "low potential" to "high potential." Following each variable name, such as "advising" or "admissions selectivity," a paragraph explained the value of such a program. For example: 
Academic Advising. The importance of academic advising as a retention strategy is well documented in the literature. Advising provides the most significant mechanism by which students can clarify their educational/career goals and relate these goals to academic offerings. 
The above paragraph certainly steers respondents away from "low potential" and "moderate potential" ratings. Oftentimes administrators, managers, church leaders, and others have good intentions, but they are not well prepared in survey design. 
Even if a survey guarantees anonymity and objectivity, it is not necessarily going to generate useful data if no one returns it or if only a certain kind of person returns it. Probably the most difficult challenge in using the survey technique is selecting a representative sample and ensuring an unbiased return. An unbiased return occurs when everyone in the sample returns the survey or when the surveys are returned by a representative sample of people from the sample itself. But even an unbiased return is not going to yield valid results if the sample receiving the survey is not representative of the larger population being studied. For example, if we want to find out what men's attitudes are about a female candidate for the U.S. presidency, we need to solicit the views of men who are rich as well as poor, Protestants as well as Catholics, young as well as old, educated as well as uneducated, and so on. 
A popular survey among the American public in the 1980s was the Ann Landers survey that asked women if they would "be content to be held close and treated tenderly" and forget about the sex act (Landers, 1984, Nov. 4, and 1985, Jan. 14 and 15). This question generated the second-largest response in the history of her column and, to the amazement of many, men in particular, the results came back overwhelmingly in support of being held instead of having sex. But the manner in which the survey was conducted gives us little confidence in the validity of the results. One might argue that Ann Landers surveys did not reach a cross section, or representative sample, of American women. It might be 
Thinking, Fourth Edition, by Gary R. Kirby and Jeffrey R. Goodpaster. Published by Pearson Prentice Hall. Copyright © 2007 by Pearson Education, Inc. 
Nonexperimental Designs 245 
that women who read Ann Landers were disproportionately a certain type of woman compared with women in general in the United States. If this was true, then the results need to be qualified: Among women who read Ann Landers, a certain percentage of them would rather be held than have sex. 
But even this conclusion may not be justified. There might also have been a problem of return bias in the Ann Landers survey. Because there was no motivation to fill out the survey other than the desire to do so, we might wonder why some women desired to fill it out and send it in, whereas others did not. Clearly not every woman who read the survey sent it in. Was there something special about the women who did respond? 
Psychologists have shown that people who have strong negative feelings about an issue are more likely to express themselves on that issue than people who have strong positive feelings. With this knowledge in mind, we might suspect that women who were dissatisfied with their sex lives and were emotionally unfulfilled might have been more inclined to send in the survey. If this was true, then Ann Landers received a biased return; that is, she did not get responses from a representative sample of her readers, but instead she received a disproportionate number of survey returns from readers in emotionally unfulfilling relationships. All that we can conclude from her survey is that some women prefer to be held rather than have sex. We can conclude nothing about American women in general, nor can we even make general statements about Ann Landers readers. 
Unscientific surveys, such as the Ann Landers survey, magazine surveys, Internet surveys, and so forth, are abundant in American media. Even evening news programs are using them when they solicit their viewers' opinions by asking them to dial a telephone number to express their view on a certain issue or when they request a response to their Web site questionnaire. These unscientific surveys may elicit responses from only certain kinds of people, who may not be typical of the larger population. Internet questionnaires, for example, are only going to be available to those who have a computer with an Internet connection. And though most people have a telephone, telephones can also be used unscientifically. When a news program asks its viewers to dial a telephone number to record a vote on a political issue, it is likely that those who have the strongest feelings and greater wealth (because there is a charge for these calls) will be more inclined to respond. People on tight budgets with moderate feelings about the issue are less likely to be reached by such a method, yet they may constitute the majority of the public. One is reminded of the telephone survey conducted during the Dewey versus Truman presidential campaign. A phone survey was conducted to find out which candidate was likely to win the election. The phone survey showed such a lead for the Republican candidate Dewey that the Chicago Daily Tribune did not wait for election results and announced his win in the morning paper the day after the election. It turned out that Truman had won. 
ISBN:
Thinking, Fourth Edition, by Gary R. Kirby and Jeffrey R. Goodpaster. Published by Pearson Prentice Hall. Copyright © 2007 by Pearson Education, Inc. 
OPINION VERSUS FACT ISBN:OPINION VERSUS FACT ISBN:246 CHAPTER 10 ¦ Scientific Thinking 
The mistake? The phone interviews reached only the wealthy, for at that time only wealthier people could afford the luxury of a phone. And since wealthy people tend to vote Republican, the survey amounted to nothing more than asking Republicans who they were going to vote for! 
Surveys about people's opinions on a topic are sometimes misconstrued as fact about a topic. The only fact that an opinion survey can claim is the fact about people's opinions. Surveys that ask nonexperts for opinions on the cause of homosexuality, the theory of evolution, or whether life exists on other planets can be taken only as statements of people's opinions. They cannot be used as facts about homosexuality, evolution, or extraterrestrial life. If a majority of people believe that homosexuality is a matter of free choice, that widespread opinion cannot be used as evidence to support a theory of free choice. Similarly, the argument for evolution cannot be weakened by the general public's opinions about it. 
Evidence and strong argument lead to facts; inexpert opinions do not. Unfortunately, many people's opinions about matters of science and philosophy do not come from scientific evidence or philosophical study but from enculturation forces. Over many millennia those forces were responsible for teaching people to believe in witches, human sacrifice, a flat earth, a geocentric cosmological system, and the inferiority of virtually every minority and religious group on the planet. So much for people's opinions. 
The Case Study 
We have seen that in order to generalize from a sample of people to a larger population we need to have a representative sample, which requires an adequate and unbiased return. Without representativeness we cannot make meaningful statements about a larger population of people. This problem of generalization is especially acute in case-study methods of investigation. The case-study method involves studying one person thoroughly as opposed to studying a large sample of people. It was the principal research method used by such famous psychologists as Sigmund Freud and Carl Jung. But because only one person is being studied, statements of generalization cannot be made from a case study. Yet people make them all the time: 
PROFESSOR SMITH: According to a well-conducted study on the principle of reciprocity, we are more likely to succeed in attracting others by expressing a positive interest in those people whom we find attractive, as opposed to using the strategy of "playing hard to get." 
Thinking, Fourth Edition, by Gary R. Kirby and Jeffrey R. Goodpaster. Published by Pearson Prentice Hall. Copyright © 2007 by Pearson Education, Inc. 
The Role of Chance 247 
STUDENT: I don't think that's right at all. PROFESSOR: And why is that? STUDENT: Well, I got my husband by playing hard to get, so I think playing hard to get works just fine. I recommend it to everyone. It sure worked for me. 
In the above example, the student is using herself as a case study and then generalizing to others. Such reasoning is not valid. No one should question the student's honesty about her experience; what ought to be questioned, however, is the generalizability of the student's experience to everyone else's experience, particularly when the student's experience contradicts the results of a well-conducted study. 
Technically speaking, case studies can be conducted within the physical sciences as well. But in the physical sciences it is legitimate to discover certain principles about a single physical event or object and then assume that those principles apply to all other events or objects identical with the one studied. If one discovered, for example, that adding two atoms of hydrogen to one atom of oxygen produced water, we could assume that that would be the case for all hydrogen and oxygen atoms. Studying human beings, however, is a different matter entirely because no two human beings are identical in their histories or constitutions. Finding out that Joan developed a multiple-personality disorder because she was severely abused as a child does not allow us to assume that everyone severely abused as a child will develop a multiple-personality disorder, or that everyone with a multiple-personality disorder was abused as a child (though, in fact, most of them were). Maybe Joan's manner of developing a multiple personality was atypical, which is conceivable given each person's unique genetics, family history, peer relations, and interpretation of life events. 
If we cannot generalize from case studies in the social sciences, of what value are they? Case studies are valuable for clinical work with patients, and they can give us hints about what might be transpiring in similar cases. (When comparing human beings, there are never identical cases.) These hints can then be tested using a larger sample of people. 
THE ROLE OF CHANCE 
It is possible that the results we get from a study are due to chance and cannot be attributed to the variables being studied. This statement does not imply that the world is chaotic or that some miracle occurred to cause our results. Chance, as we use the term here, means that the results of our study were due to random influences. For example, in a controlled experiment the researcher takes all precautions to make sure that the control and experimental groups are equal before the experimental variable, such as a new drug, is introduced. If they are equal and the 
ISBN:
Thinking, Fourth Edition, by Gary R. Kirby and Jeffrey R. Goodpaster. Published by Pearson Prentice Hall. Copyright © 2007 by Pearson Education, Inc. 
ISBN:
248 CHAPTER 10 ¦ Scientific Thinking 
group that gets the treatment is affected differently than the control group, then we can attribute that difference to the treatment-maybe. There is always the chance that the two groups were not identical before the treatment was delivered. If so, that inequality, whatever it is, might be responsible for the outcome and not the treatment. In a drug study, for instance, it could be that when the researcher selects the experimental group, by chance that group, in spite of all precautions taken to assure equality with the control group, is a hardier bunch than the control group before the drug is even administered. In this case, a higher cure rate in the experimental group might have occurred had the researcher fed the treatment group peanut butter, or given them nothing at all! 
Other research designs can also be bedeviled by chance factors. In correlational research, for example, it is possible that the correlation we discover is simply bogus! We know there is no correlation between the number of nickels you have in your pocket and the number of trees in your backyard. Yet, if we randomly selected 200 people would it not be possible, albeit not likely, that those with more nickels happened to have more trees? Certainly! Such a correlation would be due to sampling error caused by chance. 
It may be evident by now that the larger our sample size, the less likely these errors are going to occur. But even with 500,000 people in each sample the possibility of sampling error still exists, however remote. That chance, called the significance level, is statistically calculated based on the size of the samples, the amount of variability within those samples, the size of the difference in outcome between the two groups, and the strength of a correlation. 
Obviously, when the odds of our results occurring by chance are one out of a million, we have great confidence that our results reflect a real difference between our groups. But what if the odds were one out of a thousand, or one out of fifty, or one out of ten? At what point do we lose confidence in our results? In the social sciences, for example, there are two acceptable standards: significance levels of .01 and .05. If research results are significant at the .01 level, the chance of the results occurring because of some sampling error is only one in one hundred, or 1 percent; thus we can be reasonably confident-but not certain-that our study is not that one time out of a hundred in which the results could have occurred by chance. A significance level of .05 is less stringent, indicating the results could occur by chance in five studies out of 100, but it is still acceptable to most scientists as indicating the likelihood of a real difference-that is, one not due to chance but to the variable being studied, such as a drug in a drug experiment, or the secret ingredient in a chicken egg-laying experiment. 
How are we ever to know if it is chance or one of the variables under study that causes our results? We can become more certain if we repeat the study and find the same result. This is the importance of replication in the last step of the scientific method. Unfortunately, many studies are not repeated because researchers rely on the confidence levels of .01 or .05 or fail to be interested 
Thinking, Fourth Edition, by Gary R. Kirby and Jeffrey R. Goodpaster. Published by Pearson Prentice Hall. Copyright © 2007 by Pearson Education, Inc. 
Experimenter Bias 249 
enough in a project to engage in a replication. But even results significant at the .01 level could still be due to chance. If someone else repeats the research design and gets the same results, however, then chance is probably not the explanation. Year after year we hear of possible cancer cures, only to be disappointed when others repeating the experimental design fail to get the same results. 
The importance of the role of chance in research, and the way it can confound results, will vary somewhat from one science to another. But most scientific disciplines must contend with it. In 1989, for example, the American public had a major disappointment when a supposedly successful cold fusion experiment failed replication tests. These replication failures suggest that something else other than, or in addition to, the variable being studied was responsible for the initial positive results in these experiments. 
In sum, when we hear about the results of a study, we must think about the role of chance and exercise appropriate caution in our interpretation. We should ask ourselves if the results are consistent with other findings; if not, it might be prudent to wait for replication studies. 
THINK ABOUT IT: How much confidence do you need to take ac-tion? What level of confidence do you need to run a red light, engage in unprotected sex, sky-dive, bungee-jump, or bet $1,000 on the lottery? SIZEABLE EFFECTS A sizeable effect is a large effect. Just because a study's results are significant at, say, .01, does not mean that there is much to be concerned about. It is one thing to say the results of a study are not likely the result of sampling error; it's another to say the study demonstrates a sizeable effect. If a well-conducted study finds that people who eat liver are less likely to get cancer, and the study has significance at .001, we can be quite confident that the results are not due to sampling error, especially if the results are replicated. But we can still ask, "How much does eating liver reduce the risk of cancer?" "Just a tad," the researcher might reply. In that case, we need not change our diet to include the insufferable liver. On the other hand, if eating liver cuts our risk in half, then it might be time to change our palate-but not necessarily! If the original risk is ever so small in the first place, half of "ever so small" might still be ever so small and no reason to change our ways. Unfortunately, the popular media often presents study results without mentioning the size of the effect or the original risk. 
ISBN:
Thinking, Fourth Edition, by Gary R. Kirby and Jeffrey R. Goodpaster. Published by Pearson Prentice Hall. Copyright © 2007 by Pearson Education, Inc. 
250 CHAPTER 10 ¦ Scientific Thinking 
EXPERIMENTER BIAS ISBN:THE GAMBLER'S FALLACY: DON'T BET ON IT! Lottery players and other casual gamblers sometimes fall prey to the gambler's fallacy, the belief that the frequency of a random event's occurrence in the past will affect the odds of that event occurring again. For example, it is the gambler's fallacy to believe that if a penny has shown heads ten times in a row, then the odds must be enormous against it coming up with heads again because the likelihood of eleven heads in a row are extremely remote. In fact, however, the odds of a penny coming up heads or tails is not dependent on what came before it. Each throw has a 50/50 chance of coming up heads; each throw is independent of past throws. The key to understanding this lies in the past throws. If somebody gets 10 heads in a row, that is remarkable, but the odds of making it 11 heads in a row at this point (after 10 heads have already been thrown) are 50/50. The odds are behind us for the ten consecutive throws; those odds have been beaten. The next throw is only 50/50, and one would be wise to bet accordingly. Roulette wheels, penny tosses, and other devices that are based on randomness are subject to this fallacy. Bets on nonrandom events, such as dog and horse races, are not. If a dog wins several races, it is probably because he is a strong racer, so it is not fallacious to assume the dog will win the next one. In short, if you discover that a certain bingo number has not been called in weeks, you should not assume that it will, therefore, probably be called tonight. 
No intellectual activity, science included, is ever free from the shaping force of one particular ideology or another. 
-W. BEVAN, CONTEMPORARY PSYCHOLOGY 
Sometimes scientific failures are not due to chance or poor research design but to the experimenters themselves. This kind of error is known as experimenter bias-the tendency on the part of researchers to make errors in perception or judgment because of their expectations or desire for a particular result. It is part of a general tendency among all of us to see what we would like to see or what we expect to see. Sigmund Freud, Carl Jung, William James, and others argued that objective, rational inquiry may be more a fiction than a reality, a mere rationalization dictated by unconscious motives, seething emotions, and cherished beliefs. Our liking or disliking of a person, event, or idea can alter our perceptions, even if the foundation of our liking is based on nothing more than hearsay or unhealthy personal needs. Such bias affects teachers grading student exams, jurors judging a defendant, and scientists conducting research. 
Thinking, Fourth Edition, by Gary R. Kirby and Jeffrey R. Goodpaster. Published by Pearson Prentice Hall. Copyright © 2007 by Pearson Education, Inc. 
Experimenter Bias 251 
Many psychological studies have demonstrated this effect. In one classic experiment (Rosenhan, 1973), normal graduate students lied to gain admission to a mental health hospital, were given a psychiatric diagnosis, generally "schizophrenia," and then behaved normally thereafter. Their normal behavior, however, was often seen by the hospital staff to be pathological. The label "schizophrenia" biased the staff's perception and judgment of normal behavior. 
In another early experiment (Rosenthal, 1966), two groups of graduate students were given mice to run in a maze. One group was told that their mice were "maze bright"-that is, bred particularly for adeptness at maze running. The other group was led to believe that their mice were "maze dull." In fact, however, the students were working with the same population of mice. But graduate students who were told that their mice were maze bright recorded significantly fewer maze-running errors than the other group and perceived their rats to be brighter, more pleasant, and more likable. Other studies support these findings. 
When interpretation of research variables is open to subjectivity, special care should be taken to guard against experimenter bias. Because such biased interpretation is not a conscious process, it is not enough to rely on a scientist's good judgment and care. What is needed are more objective means of measuring the variables, or special procedures in the research protocol that will eliminate the possibility of bias. 
A common procedure is to make sure that the researcher is unaware of some critical conditions of the experiment that would otherwise allow for experimenter bias. For example, if Dr. Z invented an antidepressant drug and runs an experiment to find out if the drug really can alleviate symptoms of depression, then Dr. Z, having a great deal of reputation and money at stake, might unknowingly bias his interpretation of the results. It would be more than just a little unwise to let Dr. Z assess the patients' recovery when at the same time he knows which patients received his drug and which did not. It would be better to keep him ignorant about who took the real drug or have someone else who is unaware of these facts do the assessment of the patients' recovery. 
The motivation of the researcher or research organization is obviously something that everyone should be wary of. There is a big difference in credibility between the research of Burt's Chemical Corporation on the carcinogenic properties of their own weed-control product and the research of an independent group that has nothing at stake in the outcome. It is certainly not impossible for good research to be conducted by organizations that have a vested interest in the research outcome, but the potential for experimenter bias effects and outright fraud is significant. "The rule, 'I sing the song of him whose bread I eat' has held good in all times" (Schopenhauer, 1859/1958b, p. xxviii). One survey of scientists found that 
ISBN:
Thinking, Fourth Edition, by Gary R. Kirby and Jeffrey R. Goodpaster. Published by Pearson Prentice Hall. Copyright © 2007 by Pearson Education, Inc. 
THE PLACEBO EFFECT ISBN:THE PLACEBO EFFECT ISBN:252 CHAPTER 10 ¦ Scientific Thinking 
15.5 percent said they had changed the way a study was conducted or its results because of pressure from those funding the study (Wadman, 2005). Therefore, the special interests of the researcher or research organization must be taken into consideration when assessing scientific results, and those results must be weighed against any independent sources doing research on the same subject. If the television industry cites studies financed by the television networks that show television violence to have no impact on the viewer, and if those studies conflict with well-conducted research from independent sources-well, you be the judge. A similar problem arises in the political arena in which, for example, a politician is expected to carefully weigh arguments for and against gun control while at the same time receiving generous amounts of campaign money from the National Rifle Association. As suggested in Chapter 2, the politician's ability to think objectively would very likely be impaired by motivational considerations. In science, one's ability to judge, perceive, and assess are also subject to these motivational factors. Unfortunately, safeguards against these factors are not always implemented because of practical reasons, a disregard for the bias tendency, or a deliberate attempt to defraud the public. In the mid-1990s the American public became painfully aware of the potential for scientific fraud and bias when the tobacco industries were found to have suppressed research that suggested nicotine was addictive. As it turned out, tobacco science was nothing more than "politicized science," as one politician correctly put it. 
In any research in which it could be a problem, we must control for a researcher's beliefs and expectations. But beliefs of subjects must also be controlled for, because such beliefs can often confuse study results. If subjects are given a drug that they believe will cure them, the belief itself may cause the cure and not the drug. This is called the placebo effect. To control for this, only the study's experimental group is given the real drug while the control group is given a placebo, a pill containing no medicine, but led to believe that it is the real drug. If belief is responsible for the cure, both groups will be cured. If it is really the drug and not belief, then only the experimental group will be cured. The extent to which belief can cure is suggested by the Sapirstein and Kirsch study (1996). They analyzed thirty-nine studies involving a total of 3,252 depressed people and found that one-half of the drug response was due to the placebo effect. Clearly this study underscores the need to control for it in research. 
(continued) 
Thinking, Fourth Edition, by Gary R. Kirby and Jeffrey R. Goodpaster. Published by Pearson Prentice Hall. Copyright © 2007 by Pearson Education, Inc. 
Experimenter Bias 253 
THE PLACEBO EFFECT (Continued) The use of placebos in research is not without problems. For one, placebos are not well regulated. Thus, there are different types with different ingredients, and these ingredients are not completely inert. Even a simple sugar pill can alter blood sugar. Any pharmacological effects of a placebo could confound some study results by increasing or decreasing the outcome difference between the placebo and the active pill. 
If a placebo has absolutely no detectable effects, we must be concerned about another problem. In a drug culture, like that of the United States, people are becoming quite sophisticated about potential side effects of medication. If people who participate in a drug study are told that they will be given a placebo or a real pill, they might easily figure out that they have the placebo because of an absence of side effects that often accompanies real medication. If subjects can catch on that they only have the placebo, placebo research won't be able to control adequately for belief. But if we give subjects placebos that induce side effects, then we have rather active placebos with more potential to confound results. Even more alarming is the thought that the conclusions of some earlier drug research might be wrong because of these kinds of problems. 
CASES OF FRAUD The following are some examples of scientific misconduct cases that occurred between 1989 and 2000, reported in the Chronicle of Higher Education. 
¦ [Oregon Regional Primate Center] An assistant scientist in the division of neurosciences used the same photographs of cells to represent different sets of data in published papers (Wheeler, 1991, p. A7). ¦ [Northwestern University] An associate professor of physiology fabricated data for two published abstracts and submitted a document with the forged signature of a graduate student to the investigating committee (Wheeler, 1991, p. A7). ¦ [Stanford University] Two professors of psychiatry misrepresented the status of research subjects in nine papers and plagiarized a book chapter (Wheeler, 1991, p. A7). ¦ [Medical College of Georgia] A nursing professor fabricated the existence of subjects and data in a mental health research study (Burd, 1995, p. A23). (continued)
ISBN:
Thinking, Fourth Edition, by Gary R. Kirby and Jeffrey R. Goodpaster. Published by Pearson Prentice Hall. Copyright © 2007 by Pearson Education, Inc. 
254 CHAPTER 10 ¦ Scientific Thinking 
PSEUDOSCIENCE ISBN:CASES OF FRAUD (Continued) ¦ [Johns Hopkins University School of Medicine] A research program coordinator in oncology fabricated data on patient interviews and falsified updates on patient status, giving the appearance of more favorable outcomes (Walker, 1997a, p. A33). ¦ [University of the Witwatersand (South Africa)] Hematology and oncology professor confessed that he falsified data in a breast cancer treatment study (Vergnani, 2000, p. A52). ¦ [University of Ulm, the Medical University of Lubeck, University of Freiburg, and others] In Germany's biggest case of scientific fraud in decades, two professors of hematology involved in gene therapy research stand accused by various university commissions of falsifying data in at least 47 scientific papers over a ten-year period. Hundreds of other papers written by them are currently being examined for fraud (Bollag, 1998, p. A57, A59-60). ¦ [University of Missouri at Columbia] Assistant professor in the department of veterinary biomedical sciences made up data on the weights of muscles and presented the data as if they were results of experiments that were, in fact, not conducted (Walker, 1997b, p. A29). In most of the above cases, the offenders resigned from their university positions and were barred from receiving federal grants for several years. Journals that had published the falsified data were notified. 
Cases of fraud occur at all levels, in all disciplines, and in all countries. According to one survey, 0.3 percent of scientists falsified data (Cook, 2005). As you can see, some of the most prestigious universities were associated with, and victimized by, the cases above. 
True scientific inquiry uses the steps of the scientific method in a careful, objective manner in an attempt to reach some truths about the world. At the same time it is open to the possibility of error in its conclusions and considers reasonable alternative explanations. True scientific inquiry looks at all the data and does not omit facts because they threaten a pet theory or belief or are difficult to explain; it carefully and objectively weighs all the evidence for and against various hypotheses and theories. True science develops hypotheses and theories that are testable and falsifiable. In other words, its inquiry is self-correcting: In principle there is the possibility of finding evidence or experimental results that would support or weaken a given hypothesis or belief such that the unsupported 
Thinking, Fourth Edition, by Gary R. Kirby and Jeffrey R. Goodpaster. Published by Pearson Prentice Hall. Copyright © 2007 by Pearson Education, Inc. 
Summary 255 
ideas are abandoned or modified. Any inquiry which pretends to be scientific but lacks these characteristics can be called a pseudoscience. In this sense of the term,"pseudoscience" is an activity, a flawed attempt at true scientific inquiry. 
One common characteristic of pseudoscience is the tendency to give a post hoc (after the fact) explanation for an unfulfilled expectation or prediction without planning to test the explanation, or to give it in a manner that is untestable. Such explanations appear only to save face and protect a desired belief. Those who claim or believe in a fortune-teller's psychic abilities, for example, could attribute a failure to perform psychic readings and feats to "bad timing," an "uncooperative spirit," or "negative energy." 
Some people use the term "pseudoscience" as a noun, labelling entire fields of inquiry as pseudosciences. However, even though some commonly labeled pseudosciences, such as astrology, are not built on a sound scientific foundation and fail miserably in their predictions, others cannot as easily be dismissed. Moreover, the tendency to label entire fields as pseudoscience can often be rooted in enculturation and personal barriers, such as religious bias or threats to a cherished world view. Darwin's preoccupation with evolution, for example, was undoubtedly seen by many in his era as pseudoscience because it offended cherished religious beliefs and the "common sense" views of his time: "I repudiate with abhorence these new-fangled theories!" (Disraeli [1870] cited in Seldes, 1985, p. 109). Because such personal barriers so often influence our judgment about pseudoscience areas, one writer defined pseudoscience as "scientific work undertaken by anyone of whom one disapproves" (Sutherland, 1989, p. 351). 
While there certainly are some theories and subjects that unquestionably have little if any scientific support and ought to be abandoned, there are others that are more arguable. In those areas reason dictates that we allow others the freedom to pursue their investigations and be critical only to the extent to which their inquiry fails to conform to the methods and spirit of true science. 
In sum, it may be best to view pseudoscience simply as the activity of bad science, ranging from a careless or misguided scientific approach to a serious psychological virus capable of affecting scientists in any discipline, shredding their objectivity and infecting them with an unreasonable passionate belief in hypotheses and theories in which good evidence strongly suggests otherwise. 
SUMMARY 
Because we live in an era in which science permeates our culture, it is important to understand its basic methodology, assumptions, and limitations in order to think more critically about the scientific world around us. The methodology of science consists of four basic steps: observation, hypothesis formation, experimentation, 
ISBN:
Thinking, Fourth Edition, by Gary R. Kirby and Jeffrey R. Goodpaster. Published by Pearson Prentice Hall. Copyright © 2007 by Pearson Education, Inc. 
ISBN:
256 CHAPTER 10 ¦ Scientific Thinking 
and verification. It differs from other forms of inquiry primarily in its emphasis on systematic observation. This is also its limitation, for science can study only the empirical world, the world of observation and measurement. Answering metaphysical questions and determining values, for example, are outside the reach of science. 
Although many scientists work with the concept of probability, science generally assumes a deterministic and orderly universe, including the universe of human behavior. Considerable debate occurs about the extent of this determinism when it is applied to human beings. Ironically, we tend to judge people as though they are free, but we study them as though they are not. 
The methods of researchers are many and include experimental and quasi-experimental designs, ex post facto studies, and correlational designs, surveys, and case studies. Because the ex post facto method must find the difference between groups instead of creating the difference, as is done in a controlled study, it has more problems with hidden variables as alternative explanations for the results, and cause-and-effect relationships cannot as easily be inferred. The correlational designs, which examine the degree of relationship between two or more variables, is also a weak approach for discovering cause-and-effect relationships. In spite of the limitations of ex post facto and correlational studies, they are well suited for situations in which more controlled studies would be impractical or unethical. Additionally, correlational methods are quite useful in making predictions when strong correlations exist between variables. Sometimes the use of animal subjects arguably avoids ethical problems and allows scientists to use more controlled studies, such as the experimental or quasi-experimental designs. However, the question about the validity of generalization often arises when using animals to learn about human beings. Generalization is a problem in all studies if the sample is not representative of the larger population. And it is invalid to generalize from a single case study. 
Even when there are no problems in the studies themselves, there is always the question of the results occurring by chance. Results are generally accepted if their significance level is .05 or better, meaning that the results could have occurred by some sampling error five times out of a hundred or less. Replication of research can help to strengthen confidence in study results. Such increased confidence, however, does not prove a theory, because everyone's standard for proof varies. 
Researchers are human beings with cherished beliefs, pet theories, and great hopes like everyone else. These biases can consciously or unconsciously influence their judgment of the research variables. Such influence is called experimenter bias. It is important for researchers to insulate their research from this bias as much as possible. One technique, used particularly in drug experiments, is to make the experimenter and subjects ignorant about crucial conditions in the experiment. 
Thinking, Fourth Edition, by Gary R. Kirby and Jeffrey R. Goodpaster. Published by Pearson Prentice Hall. Copyright © 2007 by Pearson Education, Inc. 
Summary 257 
The techniques of controlled, objective observation make science a valuable tool for unraveling the mysteries of the world. A failure to use science appropriately, and with the right scientific attitude, is considered pseudoscience. Sometimes pseudoscience is driven by a pet theory or other cherished belief which overrides our critical scientific judgment. 
As our awareness of the strengths and shortcomings of scientific procedures increases, we can make better judgments on the claims we see and hear, and we can apply the solid principles of science to our own thinking about the world, to our own attempts to find answers in an enigmatic universe. 
Scientific Thinking Challenges 
1. Is it possible to have a cause-and-effect order in the universe without determinism? Explain. 2. What kind of research method would you use to test the effects of depression on thinking? 3. Imagine you heard the following: "Doctors found a relationship between being underweight and having cancer." What hidden variable can you think of that might explain this relationship? 4. Were you ever in an argument in which you tried to prove a point and were unsuccessful? Why? 5. Short of actually discovering intelligent life elsewhere in the universe, what would it take to prove to you that such life probably exists? 6. Outline different ways in which you could determine if broccoli or some other food prevents cancer? What are the strengths and weaknesses of each method? 7. List the ten most important things that you would like to know. For how many of them is science the appropriate tool for finding an answer to your question? 8. When you hear the claim "Doctors recommend Goody's Pills," what questions should you ask? 9. How satisfied are you with significance levels of .05 and .01 for determining confidence in experimental results? Can you think of situations in which you would want stricter criteria? 10. Are you free enough to be held responsible for what you do? How much does your social and psychological environment determine your behavior? How much is determined by your genetics? 11. Is experimenter bias a factor in the classroom? 12. Given that the speed of light is constant, if you shine a flashlight ahead of you as you travel forward at half the speed of light, how fast would the light ISBN:
Thinking, Fourth Edition, by Gary R. Kirby and Jeffrey R. Goodpaster. Published by Pearson Prentice Hall. Copyright © 2007 by Pearson Education, Inc. 
ISBN:
258 CHAPTER 10 ¦ Scientific Thinking 
from your flashlight travel? If you find this intriguing, read a book for the layperson on Einstein's theory of relativity. 
13. What do you think about the definition of pseudoscience as "scientific work undertaken by anyone of whom one disapproves"? 14. Conduct your own survey about a topic of interest using the four criteria for good surveys. How did you do? Can you generalize from your sample to a larger population? 15. Ask people the same question but in different ways. Do you tend to get different responses depending on how the question is asked? 16. Surveys are often conducted in shopping malls. What is a drawback to this technique? 17. Have you ever made the mistake of using your personal experience and then generalizing to a larger population? 18. If you noticed poverty and crime in mainly inner-city areas, what would be some of your hypotheses? 19. If you conducted a survey on the Internet in such a fashion that you picked up a representative sample of Internet users, what do you think would be the makeup of your respondents? 20. You want to find out if praying is good for your health, so you compare the health of some monks in a nearby monastery who pray every day with the health of a local atheist group. You find that the monks are healthier. Besides praying, what other variables might explain the greater health of the monks? 21. Can the opinions of the general public about UFOs, astrology, or psychic phenomena be used to determine any facts about these subjects? Why or why not? 22. Think back to the last scientific study you heard or read. Was the size of the effect mentioned? What type of scientific design was probably used? 23. If you are taking medication, to what extent do you think your belief about the medication is contributing to its effect? Do you think some medications are more susceptible to the placebo effect than others? How can you know for sure? 24. The views of astrologers have not been supported by scientific studies. Why then do so many people continue to believe in astrology? Thinking, Fourth Edition, by Gary R. Kirby and Jeffrey R. Goodpaster.