SOC341-01/51 (Kuechler -- Fall 97)
Results of the final exam (separate scores for multiple choice and for essay part) as well as final course grades have been posted at the board next to my office (HW 1628) on Friday, December 19, 1997, at 1:30pm. Listing is by partial SON. Eventually, the course grades will also be posted at the board next to the front office - however, I have no control over when this will happen.
As a matter of standard policy, I do not return final exams. However, you are welcome to review your exam in my office. Since I will not be teaching next semester and thus will not have regular office hours, you need to contact me by either e-mail or phone to set up an appointment.
To protect the question pool, I do not make the solutions to the multiple choice questions generally available. Of course, your individual exams are corrected indicating the correct answers where necessary.
Below, however, I provide feedback on one essay assignment, the analysis of an experimental study. Very few students understood the design and, consequently, scores for this assignment were generally low - some exceptions notwithstanding. If you wonder about your score, consult this feedback first.
First, here is the assignment again to refresh your memory.
B. Experimental Research
Read the excerpts from the study by Lindsey and Zakahi below. Identify and describe the elements of the classical experimental design in this particular study, i.e. check whether randomization, split into experimental group and control group, pretest, treatment, and posttest were present in this study and describe what exact form they took. Also comment on the application of the treatment; were all conditions carefully controlled, any indications that contamination may have occurred? Offer a summary evaluation of the research design: was this design suitable to pursue the research question? If, yes, point out the strong points; if, no, point out how the design could have been improved.
[Make sure to critique this research in the context of experimental designs; do not dwell on issues that hold for almost any kind of experimental study, like the lack of random sampling.]
Excerpts from
Lindsey, A. Elizabeth and Walter R. Zakahi. 1996. " Women who tell and men who ask: perceptions of men and women departing from gender stereotypes during initial interaction." Sex Roles: A Journal of Research (34), p.767-87.
[Note that some parts of the research design have been left out intentionally -- to simplify things for the purpose of this exam.]
....... A commonplace occurrence in initial interaction, uncertainty reduction can be accomplished through the use of two strategics: the asking of questions and the use of self-disclosure, or talking about the self (Berger, 1979; Berger & Bradac, 1982; Douglas, 1990). A review of relevant literature suggests that question-asking is more likely the norm for women while talking about the self is a norm for men.
Question-asking is viewed as a gender stereotypical female behavior (e.g., Deaux, 1977; Harding, 1975; Lakoff, 1975; Spender, 1980; but see McKinney & Donaghy, 1993). Fishman's (1978) qualitative analysis of male and female dyads found that women in the dyads asked two and a half times more questions than males. ..... Fishman (1978) refers to the task of developing and maintaining a conversation as "shit-work," stating that women not only work harder to make a conversation happen than do men, but that they are expected to do so.
Men, apparently, are more interested in talking about themselves during initial interaction. While generally women engage in higher levels of disclosure than men (e.g., Dindia & Allen, 1992), research has shown consistently that men will self-disclose more to strangers and acquaintances than will women (Davis, 1978; Derlega, Winstead, Wong & Hunter, 1985; Lockheed & Hall, 1976 Stokes, Fuehre & Childs, 1981). ........
The purpose of this study is to determine when individuals deviating from gender stereotypes will be perceived less positively than individuals conforming to the stereotype.
METHODS
Participants and Composition of Dyads
Seventy-eight male and 78 female undergraduate students attending a Southwestern University participated in the study for course extra credit. ... The majority of participants were Caucasian. Approximately 30% of the participants were Hispanic. Most participants were middle-class and all participants were raised in the United States. Several participants were nontraditional students.
Participants were signed up for experimental sessions during class sessions, two classes at a time. Members of one class volunteered for time slots available, then, where possible, members of a second class were signed up so that they were paired with a different-gender member of the first class. Male and female members of the first class were designated as the target in each dyad and were randomly assigned to either the "ask" or "tell" conditions prior to their session. Participants were unaware of their status as target or perceiver, and were told only that the experimenters were investigating initial interaction between "opposite-sex members."
Measures
Ratings of Positive Affect. The ten positive items from Watson et al.'s (1988) Positive and Negative Affect Schedule (PANAS) were used to obtain an assessment of participants' interest and pleasurable engagement during the interaction. Each item on the scale consisted of a single emotionally descriptive term such as "active," "interested," or "alert." Participants indicated the extent to which they experienced feelings during the interaction as described by each term on a scale of 1 to 5. Coefficient alpha for the scale was .83. Final scores for positive affect were created by averaging the scores across the ten items for each participant.
Social Attraction. The social attraction items from a revised version of McCroskey & McCain's (1974) Interpersonal Attraction Scale (McCroskey & Richmond, 1979) were used to assess perceptions of partners' social attractiveness. Participants rated their partner on twelve 7-point Likert items, indicating the extent to which the partner was perceived as friendly and the extent to which the partner had potential for future friendship and further interaction. The alpha reliability for the scale in this study was .87. Scores were averaged across items for each participant to create the final index of social attractiveness.
Perceived Manipulativeness. Because targets would be instructed to either ask questions or talk about themselves regardless of the partner's behavior during the interaction, we included a measure of perceived manipulativeness in the study as a covariate in the main analyses. The measure consisted of eight 7-point items, included in the Appendix. Alpha reliability for the items was .75. The index of perceived manipulativeness was calculated by averaging the scores across items for each participant.
Procedure
At the time they signed up for the experiment, participants and targets were asked to report to separate rooms for their experimental session. Shortly after arriving for their session, perceivers were seated in a room and given the following information: We will be examining what occurs between men and women during an initial interaction. In a moment, we are going to pair you with a member of the opposite sex and ask you to engage in conversation with each other for approximately 5 minutes.
This information deliberately emphasized "men," "women," and "member of the opposite sex" in order to activate perceivers' gender-schemata. Targets, reporting to a second room, were given the same information but also received additional instructions. Targets assigned to the "ask" condition were told: During this interaction, it is very important that you ask your partner as much about him or herself as you comfortably can. YOU MUST NOT REVEAL TO YOUR PARTNER THAT YOU HAVE RECEIVED THESE INSTRUCTIONS. If you have any questions about these instructions, please ask now. Targets in the tell condition were given similar instructions, asking them to "tell your partner as much about yourself as you comfortably can." After receiving these instructions, targets were reminded to follow instructions to the best of their ability, regardless of the behavior of their partner.
Targets were taken to the same room as their perceiver partner. Partners were seated diagonally across the corner of a table from each other at a distance of approximately 20 inches to 24 inches. After both participants were seated, the experimenter repeated instructions to conduct a 5 minute conversation, emphasizing again the opposite-sex nature of the interaction. The experimenter then left the room, returning after a five minute period. Targets and perceivers were moved to another room and seated at separate tables in order to complete a packet of measures. Total time of participation in the experimental session was approximately 25 minutes. At the end of each session, participants were thanked, debriefed and asked not to reveal the procedures of the study to anyone else.
The best way to start such an analysis is always to identify the underlying hypothesis. Do not confuse the general discussion of the larger research question and previous finding with the "hypotheses", i.e., with what is to be tested in this particular study. Here the hypothesis was very clearly stated:
The purpose of this study is to determine when individuals deviating from gender stereotypes will be perceived less positively than individuals conforming to the stereotype.
Independent variable: Deviation from gender stereotype (yes/no) in initial contact
Dependent variable: Positiveness of perception by other
Note that "deviation from gender stereotype" means "asking questions" for male subjects and "talking about herself" for female subjects. And, the other way round, "no deviation from gender stereotype" mean "talking about himself" for male subjects and "asking questions" for female subjects.
Units: The units in this study are 'dyads' or pairs of one man and one woman. In each pair, one party gets specific instructions to "treat" the other party - by either displaying gender-stereotypical or non-gender-stereotypical behavior. This party is called the "target" (of the experimenter) in each pair, the other party is called the perceiver. The perceivers are the ones that get one or the other treatment, and measuring the impressions of the perceivers constitutes the posttest. So, on the one hand, we have dyads as units; but in terms of the underlying hypothesis only the "perceivers" count. The "targets" are just "blind" helpers of the experimenter.
Treatment(s): The "perceivers" in those pairs where the "target" gets the instruction to act different from the gender stereotype of his/her sex form the experimental group; the perceivers in the other pairs from the control group. - One could also say that there are two different experimental groups based on the fact that on the surface all the pairs are engaged in supposedly the same activity ("engage in opposite sex conversation").
Randomization: The determination which "target" subject gets what instruction was determined randomly, so, yes, there was randomization. The formation of pairs to begin with had nothing to do with the determination of experimental and control group; therefore this process is largely irrelevant for the design of the core experiment.
Pretest: At least, as described in the (simplified) excerpt, there was no pretest. Given the nature of the variable (which is defined as a reaction to a specific encounter), it is not possible to do a pretest in the strict sense. One possibility would be to test general perceptions towards members of the opposite sex, but this would very likely introduce a "testing" effect and thus a threat to the internal validity of the experiment.
Posttest: Measuring the affect towards the partner using two established instruments (scales), PANAS and IAS. A third measure was used "as a covariate" - partially compensating for the lack of a pretest. All subjects were given the test package; however, for the analysis only the scores of the "perceivers" were important. The "targets" were treated the same to maintain the "cover" and not to raise any suspicions about the true intent of the experiment.
So, the design was a randomized comparative posttest design - as in homework #7. But, it was a bit harder to identify the pieces.
1. Given the hypothesis, to assess perceptions at first contact, it was crucial that the two partners in each dyad did not know if each other beforehand. The researcher tried to ensure this by taking one partner from one class and then let someone from a second class sign up in available slots. So, the partners in each pair came from different classes. However, the excerpts does not mention any additional effort to check that the two partners had never met before. This is a potential problem.
2. It is not described how compliance was checked, i.e., how did the researcher make sure that the "targets" were following the instructions? Having the experimenter leaving the room during the actual conversation is advantageous, but tape-recording it would have provided an opportunity to check on compliance and may have also helped to discover any previous acquaintance.
3. By making people behave in ways that may not usually do, they may project a "fake" image that in itself may lead to a negative perception. If we assume that the large majority of subjects ("targets") do normally behave stereotypical, more subjects in the experimental group than in the control group were forced to act "fake", and thus a more negative perception in the experimental group could also be caused by fake, non-habitual behavior not just by non-stereotypical behavior. To strengthen the design one would need to recruit a sufficient number of subjects that display naturally non-stereotypical behavior ("assertive, extrovert women ready to talk plenty about themselves", "laid-back, introvert men more likely to ask and listen") and use these as the "experimenter's agents" in the pairs in the experimental group. This could be done as part of a general screening test administered several weeks before the actual experiment.