Ben Collier, PhD

Carnegie Mellon University in Qatar · 2012 to 2016 · Undergraduate

70-323 Empirical Research Methods

70-323

Ask a question about people, design a survey, collect the data, and test it with regression.

About 23 students

This course taught students to evaluate and run original research on human behavior. Each student picked a question about life in Qatar, wrote a proposal, completed human subjects training, went through IRB review, wrote and piloted a survey, and collected their own data. We covered measurement, scales, sampling, survey design, and experiments along the way, then spent the last weeks on analysis so that every final paper tested its main hypothesis with regression. The semester ended with three days of student research presentations.

Week by week

Spring 2013 · Mondays and Wednesdays. Scroll the schedule and the slides follow along: five from each session, picked from the decks I taught from.

  1. Part I: Designing research
  2. Week 1
    Introduction to research designCovered January 14 and 16
    Errors in inquiry: inaccurate observation, overgeneralization, selective observationConcepts become variables, and variables have attributesWhat a relationship between education and prejudice looks like, and what no relationship looks likeThe elements of theory constructionA worked example: the technology acceptance model
  3. Week 2
    Research ethics
    Groups of three: would you approve this Facebook study?No harm to participants, and informed consentAnonymity versus confidentialityDeception, and debriefing afterwardInstitutional review boards
  4. Week 2
    Research design
    A good hypothesis says which way the relationship goesNecessary and sufficient conditionsCorrelation, spurious relationships, and real causesUnit of analysis: do more tickets in poorer Doha neighborhoods mean poorer drivers speed?Cross-sectional and longitudinal designs
  5. Part II: Measurement
  6. Week 3
    Measurement: conceptualization and operationalization
    How do we measure things we cannot see?Conceptualization: nominal and operational definitionsOperationalization: range of variation and precisionLevels of measurementGroup exercise: conceptualize and operationalize 'love'
  7. Week 3
    Indexes and scales
    Absorptive capacity, a construct you cannot observe directlyReliable but not valid, valid but not reliable, bothThe kinds of validityIndex versus scaleHow a scale's items build on each other
  8. Week 4
    Scale construction
    Social distance, one question at a timeThurstone scales: items weighted by judgesLikert scalesGuttman scales: agreeing with a strong item implies agreeing with weaker onesCard sorting to group items
  9. Part III: Sampling and surveys
  10. Week 5
    SamplingExam 1 was on February 11
    Convenience sampling, and when it might be acceptableTarget population, sampling frame, and who you missThe sampling distribution of ten casesBigger samples give tighter sampling distributionsThe central limit theorem
  11. Week 6
    Sampling, part 2
  12. Week 6
    Survey designIRB forms due
    Simple random sampling from a numbered populationWhat makes a good survey questionA confusing double negative: 22% doubted the Holocaust in 1992, 1% with clearer wording in 1994Double-barreled questionsLeading questions
  13. Week 7
    Survey workshop: peer review of survey drafts In-class exercise
  14. Week 8
    Spring break No class
  15. Part IV: Experiments
  16. Week 9
    Experimental design
    The classical experimentExperimental and control groups, measured before and afterMatching subjects across groupsPre-experimental designs and what they leave outGroup exercise: design an experiment on whether review sessions raise exam grades
  17. Week 9
    Experiments, part 2: validity and settings
    Rigor, relevance, and generalizability: you can't maximize all threeWhere each research method sits between rigor and relevanceThreats to internal validityA pretest-posttest design with a control groupA framing experiment run on the class with clickers
  18. Week 10
    Evaluation research
  19. Part V: Analysis and presentation
  20. Week 12
    Analysis: from survey items to a modelExam 2 was on April 1
    Less than 15% of English Wikipedia editors are women. Why so few?Turning a hypothesis into a survey item: free timeDiscomfort with criticizing or editing others' workConfidence in one's own knowledgeA mediation model of gender and editing
  21. Week 13
    Analysis: distributions and regression
    The normal distribution68, 95, and 99.7 percent within one, two, and three standard deviationsPredicting promotion from production contributionsPredicting promotion from organization buildingY = a + bX, term by term
  22. Week 13
    Multiple regression and presenting research
    Multiple regression: income on education, parents' income, and experienceWhat 'controlling for' meansDoes playing the piano raise your income?Reading the stars on a regression tableMagnitude versus significance: QR 8 a year is not much
  23. Week 14
    In-class analysis in Excel In-class exercise
  24. Week 14
    Student research presentations Presentations
Errors in inquiry: inaccurate observation, overgeneralization, selective observation
Mon, Jan 14 Introduction to research design Errors in inquiry: inaccurate observation, overgeneralization, selective observation

Major assignments

  • Final research report15%

    A 2,000 to 2,500 word paper with introduction, literature review of at least three research articles, hypotheses, methods, results, and the survey as an appendix. Every paper had to test its main hypothesis with regression.

  • Final presentation15%

    A short talk on the study and its results, over three days at the end of the semester.

  • Pilot studies and drafts10%

    CITI human subjects training, two proposal drafts, IRB forms, and a survey draft that classmates took and critiqued in class before the final version went out.

  • Two midterm exams40%

    20% each, on research design and measurement, then sampling, surveys, and experiments.

  • Online quizzes10%

    Eight quizzes on the Babbie readings, lowest score dropped.

  • Participation10%

    Attendance by clicker, and in-class design exercises.

Student work

Every project, by term.

Spring 2013: student survey studies 22 projects
  • Student satisfaction in Education City versus Qatar University. Compared satisfaction at the private branch campuses and the national public university. Survey
  • School environment and the desire to study. Whether the school environment relates to students' motivation to keep studying. Survey
  • Employment expectations of CMU Qatar students. Whether nationality, family, and gender shape what students expect after graduation. Survey
  • First-year friendships. Whether first-year students befriend people who are academically similar or similar in other ways. Survey
  • Pedestrian safety in Qatar. Whether pedestrians feel safe on Doha's roads, and what urban planners could change. Survey
  • Mass media and body image. Effects of mass media on body image, eating habits, and exercise. Survey
  • Attitudes toward weight-loss surgery. Whether people accept surgical methods of losing weight over diet and exercise. Survey
  • Creativity. What relates to self-reported creativity among students. Survey
  • Online piracy by students in Qatar. Whether money and the lack of paid alternatives predict how much students pirate. Survey
  • Extracurricular activities and grades. Whether taking part in extracurricular activities affects academic performance. Survey
  • Eating habits on and off campus. Whether students living off campus eat more healthily than those in the dorms. Survey
  • Social capital and academic achievement. Whether relationships with friends and professors relate to grades. Survey
  • Music and aggression. Whether taste in music relates to aggressive attitudes. Survey
  • Buying a car in Qatar. Consumer views of the car market and dealers. Survey
  • Parental control and childhood obesity. Whether parents' control over eating and activity relates to obesity, given high overweight rates in Qatar. Survey
  • Motivation and academic performance. How student motivation relates to grades. Survey
  • Sports in university life. What keeps students from taking part in campus sports. Survey
  • Mobile phone use and academic productivity. Whether heavier phone use goes with lower academic performance. Survey
  • Social media in Qatar. How people in Qatar use social media. Survey
  • Internet use and happiness. Whether how often people use the internet relates to depression and happiness. Survey
  • Why people lie. Whether people with different characteristics tell different kinds of lies. Survey
  • Gender and computer literacy in Qatar. Whether men and women in Qatar differ in computer literacy. Survey

Offerings

  • Spring 2013

All courses