PSY 715.06: Research Design and Methods
Professor Keith A. Markus
Time: Thursdays 4:15-6:15 PM
Presents the nature of the research process and guidelines for formulating researchable questions and testable hypotheses. Reviews the methods of operationalizing variables and indicators, and collecting data, including designing experiments and carrying out surveys. Explains data analysis strategies leading to a written report. (Graduate Bulletin, pg. 68).
Frankfort-Nachmias, C. & Nachmias, D. (2000). Research methods in the social sciences. New York: Worth.
de Vaus, D. (2001). Research design in
social research. London: Sage.
The course also requires assigned articles. The Lloyd Seally Library makes all of these available through its online full-text collection. To locate an article, find it in PsychInfo and click the full text link.
Come to class prepared to ask questions and discuss the reading. For my part, I will come to class prepared to answer the questions that you post regarding that week's reading. Form a habit of bringing to class copies of the assigned reading for that week. This will allow us to focus on specific passages from the text during class discussion. If you have questions about the assignments, you can also ask them in class or post them to Blackboard. Questions about the assignments do not count toward your grade.
Questions About Readings
Over the course of the semester, you are each responsible for contributing at least four questions regarding the reading before the class corresponding to that reading. Before class, I will review the questions and address them in class. I encourage you to submit as many questions as you have. The four required questions reflect a base minimum. Make your questions specific and concise, and always include the page number and source for the passage that you ask about. If you submit a vague question, I may ask you to clarify it before class begins. Overly vague questions will not count toward the required four.
|I looked up the word somnambulist in the dictionary, but I still do not understand how the author uses it on page 31. The dictionary definition does not seem to fit the rest of the paragraph.||I do not understand the discussion from pages 223 to 245 of the book.|
|On page 106, the author seems to say that the reliability of the dependent variable does not matter, but on page 119 he seems to say that it does matter. What am I missing?||Can you go over Chapter Three?|
|The second paragraph on page 94 seems to imply that we should not consider experiments to provide objective support for theories even if different experimenters get the same result. Do I understand that correctly? I thought that was what the word objective meant.||Me too, I have the same question.|
Use the designated discussion board in
(Bb6) to post your questions. Put each question in a separate
Post questions by 4 PM on the day the reading is due in order to
All assignments come due at the beginning of class for the date noted. Concentrate on producing clear and concise writing. You should expect to spend a larger portion of the time thinking and a lesser portion of the time writing. Give yourself time to revise what you write after your first draft.
Grading: Each assignment comprises a series of numbered components. I will grade each of these on the following four-point scale.
0 = Completely missing.
6 = Not missing, but clearly wrong (demonstrates poor understanding).
8 = Partially correct but unclear, incomplete, or otherwise flawed (demonstrates partial understanding).
10 = Completely correct (demonstrates full understanding).
I will then average the component scores and
by 10 to assign a percent grade. (This should never come up, but
should I receive a late assignment it will receive a grade no higher
Assignment 1. Sign onto
course page and check your email address. If it is not
use www.cuny.edu to correct it. Note that you cannot do this
directly on Bb6, you have to log into the CUNY portal and change it
there (as of now).
Assignment 2. Choose a continuous variable that varies over New York City residents of high-school age. Think through your choice to make sure that it meets the criteria for a continuous quantitative variable. First, make sure that people in the above population will indeed vary in their values on the variable. For example, high-school aged residents will not show much variation in age or education level.
Second, make sure that your variable represents a single dimension of variation, so that you could imagine lining people up in order from low to high without getting into a quandary over people who seem high in one way but low in another. For example, parents’ socio-economic status (SES) would make a poor choice because some people earn higher incomes despite lower levels of education, whereas others (like college professors) have high levels of education and lower incomes. So, one can create different SES variables depending upon how much weight one gives to education relative to income, and these different versions of SES will not necessarily sort people into the same order.
Third, the variable should have many possible values that count something (even if it counts something relatively abstract). For example, gender or biological sex would make poor choices because biological sex and gender have only a few values that do not come in any particular order (nominal dichotomous or polytomous depending upon one's theory of gender). Even something like birth order presses the limit of what counts as a continuous variable. Although it does count something (siblings born before the person in question), just a handful of values will account for the vast majority of people in most human populations. The exact range of values and unit of measure will depend upon how you choose to measure the variable. Although you can abstract over the specific measure at this point, it does not hurt to think ahead.
Fourth, make sure somebody else has not already taken your variable. Skim through the variables already posted to make sure that yours adds something new to the list. (The odds of two people posting the same variable at exactly the same time virtually preclude this problem from occurring.) You can choose a variable closely related to one already on the list, but not the same variable. (So give yourself time if you have to choose again. Do not wait until the last minute.)
Post a description of your variable to Bb6 using the
1. Your name.
2. Name of your variable (put this in your subject line too).
3. A brief description of your variable, clarifying any details not rendered clear from the name.
4. A brief explanation of why your variable meets the first three criteria described above.
Assignment 3. One can generally measure the same abstract variable in a variety of different ways. The choice of measurement method will determine the exact range of values, variability, and unit of measure in a given population. For example, one can measure body weight either by weighing people or by self report. Research suggests that self-report will generally correlate very highly with weight measured using a scale, but will introduce a slight downward bias. In other words, the two variables have roughly the same variance and roughly the same shape distribution (a normal distribution), but scale weight has a slightly higher mean. Two measures of depression might correlate very highly despite different means, variances, and ranges of scores (one might range from 1 to 10 and another from 25 to 950). In contrast, different measures of IQ (whatever that means) frequently correlate only moderately despite identical scales and distribution shapes.
For the variable you chose in Assignment 2, post a
of a method for measuring that variable using the following format.
1. Your name.
2. Variable name (put this in subject line too).
3. A survey item that you could use to measure the variable. Type the item as it would read on the survey.
4. The response format for the survey item. Type response options and/or instructions as they would appear on the survey.
5. Coding. Briefly describe how you would code the responses. Your answer should allow somebody reading it to translate from responses to numbers. Include at least one code to indicate missing values and explain its (their) use.
6. Unit of measure:
(6a) If your measure uses a non-arbitrary unit of measure (e.g., dollars, months, arrests, incidents-per-year, square-feet-per-person-in-household, etc.), then list that unit of measure.
(6b) If your measure uses an arbitrary unit of measure, then indicate that.
7. Approximate anticipated mean. List the value you would expect for the average NYC high-school-aged resident.
8. Approximate anticipated standard deviation (SD). Imagine the middle two-thirds of the distribution of NYC high schools students on your variable. The middle two-thirds of cases lie between one SD below the mean and one SD above the mean (for a normally distributed variable). List the value of the SD that would contain the middle two-thirds according to your expectations.
Assignment 4. Imagine a
member of the population described above and take the survey consisting
of each student's question from Assignment 3 item number 3 as that
hypothetical person (not as yourself). Post your responses to Bb6
using the digital drop box. Format your answers as follows.
1. Save your answers in rich text format (rtf) or
text format (txt).
2. Use your full name as the name of the file (e.g., Sigmund Freud.rtf, B F Skinner.txt).
3. List your answers to the questions in order, separated by commas, with no intervening text. For example, if the survey contains 30 questions, your file might look like this.
Do not put anything else inside the file. Try
not to skip any questions but if you must, simply put a space between
the commas. Never include a comma as part of an answer. I
will use your hypothetical responses to compute a correlation matrix
for the next assignment.
Post your hypothesis to Bb6 using the following
1. Your name.
2. Type of hypothesis (chain or disjunctive fork).
3. Variables separated by arrows to indicate causal structure. Use a hyphen and a greater-than or less-than sign to form text arrows. (Place this in the subject line too.)
4. Rationale for first causal connection (e.g., for A->B, explain why A causes B and also why B does not cause A).
5. Rationale for second causal connection (as with 4).
6. Rational for absence of omitted causal connection (between X and Z in above examples).
Assignment 6. Find the correlations for your three variables on Bb6. Enter the correlations into the F-test spreadsheet on Bb5. Follow the instructions in the spreadsheet to test the statistical implications of your hypothesis. Report the results in a post to Bb5 using the following format.
1. Your name.
2. Your hypothesis (repeat #3 from Assignment 6, place this in subject line too).
3-5. Correlations between variables (three pairs).
6. Results of F test including value of F, degrees of freedom, sample size, and p value.
7. Statistical conclusion regarding F test (using alpha = .05, partial correlation statistically significantly different from zero or not statistically significantly different from zero).
8. Substantive conclusion: Taken as a whole, do the results support your hypothesis?
--If either pair of variables hypothesized to affect one another have correlations very close to zero, and you expected more than a marginal effect, you should most likely take this as evidence against your causal hypothesis.
--If the F test indicated a statistically significant difference from zero, this provides evidence against your hypothesized causal structure.
--If the F test indicated no statistically significant difference from zero, this provides partial support in favor of your hypothesis assuming that you had reasonably large correlations (not near-zero) between both pairs of variables hypothesized to have direct causal effects between them.
Assignment 7. Find a research report from a peer-reviewed journal that relates (tangentially if necessary) to your above hypothesis. Post a review of the article to Bb6 using the following format. Write each section in your own words without using quotations from the research report.
1. Your name.
2. Complete reference for study. Use APA format for reference citations in text and the reference list. Do not guess. Use the APA manual or the Library handout to verify the correct format. If you encounter a case you find unclear, ask me before the paper comes due. (In lieu of italics, place an underscore before and after italicized text: _like this_.)
3. A hypothesis tested in the study. (Many studies test more than one, you need only discuss one of them).*
4. Relationship to your hypothesis.
5. Type of study (experiment, quasi-experiment, passive observation, etc.).
6. Brief description of sample.*
7. Brief description of data collection including measures or manipulations used.*
8. Brief description of results relevant to hypothesis given in #3 above.*
9. State whether the findings (a) support your hypothesis, (b) conflict with your hypothesis, or (c) neither. Briefly explain why.
10. Describe a limitation of the internal validity of the study.
11. Describe a limitation of the construct validity of any variable in the study.
12. Describe a limitation of the external validity of the study.
13. For any one of the above limitations (9-11), describe a modification of the procedure used in the study that would improve the corresponding aspect of the validity of the study (internal, construct, or external). Indicate how the modification would improve the study.
*Note that the most common source of
faced by students competing this kind of task comes from incomplete
of the research report. Your ability to select important points
the description given in the report and provide a concise summary rests
on your ability to understand the report and apply your knowledge of
material to that understanding. Absent this understanding, you
find yourself inclined to write too much and repeat too much verbatim
the report. If you recognize these symptoms in yourself when
on the assignment, stop writing and review the report and the relevant
material from the textbook until it makes sense. If you find
that stumps you (quite possible given that published research will
rely on methods that go beyond an introductory methods course), ask
me about it. Get started on assignments as early as possible
in order to leave time for dealing with these sorts of difficulties.
The Examinations will contain only multiple choice items. The majority of the items will test material from the text book. The remaining items will test reading comprehension with passages taken from published research comparable to the reading assignments from the Prentice Hall database. I will not test you directly on you understanding of the assigned articles. Instead, I will test you on your ability to read and comprehend comparable material included in the examination. As such, you do not need to memorize the studies we discuss in class but you do need to use them to build your reading comprehension skills. Each examination covers the chapters indicated on the course schedule.
Your final grade comprises your examination grades (40%), your questions about the reading and Assignment One (20%), and the remaining six assignments (40%). Each of the two examinations is worth 20% of your total grade. Each of the four required questions about the reading counts as 4%. Assignment 1 also counts 4%. The remaining six assignments (i.e., Assignments 2-7) count as 5% (Assignment 2), 6% (3 & 4), 7% (5) and 8% (6 & 7). I will return all grades in percent form, so to compute your final grade you multiply by the above percents and add them up: Final Grade = .20(E1) + .20(E2) + .16(Q) + .04(A1) + .05(A2) + .06(A3) + .06(A4) + .07(A5) + .08(A6) + .08(A7). (E means exam, Q means questions, and A means assignment.) I will assign letter grades as indicated below. I will round x.5 and above up and anything below x.5 down.
|Letter Grade||Percent Grade|
Office Hours: Thursdays 1:00 PM - 2:00 PM.
|9/2||Frankfort-Nachmias & Nachmias (2000) Chapter 1 (FN
|9/9||FN 2 & de Vaus (2002) Chapter 1 (dV 1).
||Theories, Models & Other Basics
does not meet.
|9/23||Assignment 1 Due.
FN3 & dV 2.
|Elements of Research Designs|
|9/30||Assignment 2 Due.
dV3 & FN4.
|Causal Inference & Research Ethics|
|Psychological Measurement & Survey
|10/14||Assignment 3 Due.
FN10 & FN11
|Survey Research & Survey Construction
|10/21||Midterm Examination (FN1-5, 7 & dV 1-3).|
|10/28||Assignment 4 Due.
FN5 & dV5 (Optional: dV4)
Mazzoni, G. A. L., Loftus, E. F., Kirsch, I. (2001). Changing Beliefs About Implausible Autobiographical Events: A Little Plausibility Goes a Long Way. Journal of Experimental Psychology: Applied, 7, 51-59.
|11/4|| FN6 & dV11
Smit, F., Cuijpers, P., Lemmers, L., Jonkers, R. & de Weerdt, I. (2003). Same Prevention, Different Effects? Effect modification in an alcohol misuse prevention project among high-school juniors. Drugs: education, prevention and policy, 10, 185–193.
|Quasiexperiments & Cross-sectional Designs|
|11/10 (Wednesday)||Last day to resign without academic penalty.|
|11/11||Assignment 5 due.
dV7 & dV8
d'Silva, K., Ferriter, M. (2003). Substance use by the mentally disordered committing serious offences – a high-security hospital study. Journal of Forensic Psychiatry & Psychology, 14, 178-193.
Sainsbury, L., Krishnan, G,.Evans, C. (2004). Motivating factors for male forensic patients with personality disorder. Criminal Behaviour & Mental Health, 14, 29-38.
|11/18||Assignment 6 Due.
Ho, T. (1999). Examination of Racial Disparity in Competency to Stand Trial Between White and African American Retarded Defendants. Journal of Black Studies, 29, 771-789.
|Secondary Data Analysis & Archival
|11/25||Class does not meet.|
|12/2||Assignment 7 Due.
dV13 & dV14
Renn, P. (2002). The link between childhood trauma and later violent offending: The application of attachment theory in a probation setting. Attachment & Human Development, 4, 294-317.
|Case Studies & Case Histories
|12/9|| FN9 & FN14
||Summary of Observational Data & Preparing
Data for Analysis
|12/16||Final Examination 4:15-6:15 PM (FN9, 12-14 & dV5, 7-8, 11-14).|
|12/29||Final grades due in Registrar's Office.|