Empirical research: answering questions with confidence
Observational Research
- Descriptive Research (observational research)
- Exploratory studies
- Case studies
- illustrate, facilitate recall, entertain
- but can any conclusions be generalized to other cases?
- are the reported results accurate?
- Do aggressive children grow up to be aggressive adults?
- How do we identify aggression?
- Does aggression in a child look like aggression in an adult?
- form (topology) vs. function
- B.F. Skinner and behavior
- lever pressing in a rat: a functional definition of behavior
- but in real life function can be difficult to assess
- Lovaas’ video and “aggression” among some teenage boys
- B.F. Skinner and behavior
- reliability of observational data
- reliability of observational data
- Do aggressive children grow up to be aggressive adults?
- are the results representative?
- sampling
- population studies vs. clinic studies
- Do we know what the relevant variables are ? (age, gender, intelligence, hair color [subject state variables]; hunger, degree of arousal, fatigue, color of shirt [subject state variables]; SES of family, urban/rural, nationality [environment trait-like variables]; parent presence, substitute teacher, winter, full moon [environmental state-like variables])
- How often do we observe, where, how?
- How situationally specific is the behavior we are interest in?
- sampling
- are the reported results accurate?
Epidemiological studies
San Diego Cohort, 1981 (Kraus et al., 1990)
- Dead at scene/DOA 5%
- Severe brain injury 6%
- Moderate brain injury 8%
- Mild brain injury 82%
- <1 year of age 4%
- 1-4 year of age 15%
- 5-9 year of age 19%
- 10-14 year of age 20%
- 15-19 year of age 43%
- male: 293.0 per 100,000
- female 137.9 per 100,000
- incidence of brain injury 2.1 times higher for males
- External causes of brain injuries:
- Motor vehicle 37%
- Falls 24%
- Assault 10%
- Sports/recreation 21%
- other 8%
- Estimate of national disabilities:
- assumptions:
- 1. overall incidence rate of hospitalized brain injured person in U.S. of 210 per 100,000 persons aged 0 to 19 years
- 2. U.S. population of 74,255,000 persons aged 0 to 19 years in 1986
- 3. severity proportions of 86% mild, 8% moderate, and 6% severe
- 4. in-hospital case-fatality rates of 0%, 2%, and 42% per 100 patients, respectively, by severity
- 5. disability rates (any neurologic limitation or disability on hospital discharge) of 10%, 90%, and 100% for mild, moderate, and severe brain injury
- Then:
- About 29,000 persons aged 0 to 19 years have a resultant disability each year from brain injury.
- assumptions:
Correlational Research
- identifying patterns and relationships
- Are variables associated?
- correlation coefficients
- measuring the strength of an association
correlations, negative and positive
-1 -.5 0 +.5 +1
- “Third body problems” and the search for causes
- William C. McGrew (1972) An Ecological Study of Children’s Behavior
- studied “agonistic behavior” in preschool children in England
- very “molecular” elements of behavior
- threat, verbal: “I’m gonna hit you”
- threat, physical: dominant arm raised
- wrist cock
- hit
- conditional probabilities
- neither verbal threats nor physical threats strongly predicted actual blows; however, if the wrist rotated (“wrist cock”), there was a high probability of a strike
9-12-13
Experimental Research (casual explanations, cause and effect)
- Experimental research attempts to create conditions in which we can conclude that one variable causes another.
- What do we mean when we say one thing “causes” another?
- Variables: What are we interested in?
- Dependent variable: the qualities we are interested in understanding (effect)
- Independent variable: the qualities we manipulate to see what effect occurs with the dependent variable (cause)
- How can we know if the Independent Variable affects the Dependent Variable?
- Creating conditions in which no other explanation seems viable: No other explanation makes any sense.
- Are our results reliable?
- Reliability is always about consistency: if we look again, do we find the same thing?
- Are our results valid?
- Validity is about what this means (some would say “truth”): do these results mean what we think they mean?
- Threats to validity:
- Internal validity: do our results demonstrate what we believe (wish) these results to demonstrate?
- External validity (generalizability): do our results apply to anyone beyond the group we worked with?
- Issues in internal validity:
- alternative explanations for our results
- some problems of children tend to remit of time: “spontaneous recovery”
- within a group of children with a problem, some wil get better
- Control Groups
- placebo effects and psychotherapy/drug research
- classical medication study:
- patients with a problem
- are randomly assigned to one of two groups:
- Group A (medication)
- Group B (placebo)
- treatment prceeds until a comparison is made: Does Group A do better than Group B? [We actually expect that both groups will improve somewhat and both groups will have some undesired “side effects”]
- for “medication” you can substitute any “active treatment” versus “control”[comparison group]
- Group A [new medication]
- Group B [currently accepted medication]
- or
- Group A (new psychological treatment)
- Group B (UCT: “usual and customary treatment” or “usual and customary care)
- or
- Group A (new psychological treatment)
- Group B (UCT)
- Group C (waiting list control group)
- random assignment (why selection studies are not experimental)
- classical medication study:
- placebo effects and psychotherapy/drug research
- detection of differences
- real differences might be small (and likely to be missed)
- real differences (valid) may also be trivial (relatively unimportant in the real world)
- significance testing
- measurement error
- quality of our measure
- sample size
- significance testing
- alternative explanations for our results
Issues in external validity
- sampling: Is the group we studied representative of the population we wish to draw conclusions about?
- How are populations described?
- Do we know what the important variables are?
- Lovaas’ autism treatment research
- groups randomly assigned and then observed to be similar on:
- symptoms of autism
- diagnosis of “autism” by independent psychologist or psychiatrist
- age of set [prognostic variable in literature]
- tested intelligence [prognostic variable in literature]
- porportion of males/females [sex difference in prevalence widely reported in literature; possible differential prognosis of males vs. females]
- functional language [prognostic variable in literature]
- the outcome saw clear superiority of positive changes in more of the “intensive treatment” than “usual and customary treatment” groups
- Lovaas attributed this to the new treatment program
- some critics questioned whether the children had “really been autistic”: that is, they did not question the good outcome but rather these results had implications for treating children who were “really autistic”
- groups randomly assigned and then observed to be similar on:
Ethics in research on child psychopathology
- sources of ethical guidance in research
- laws
- government regulations governing grant funding
- ethical standards of professional associations
- institutional policies
- conscience and personal morality
- What makes research ethical?
- Informed consent of participants
- voluntary
- freedom from coercion
- negative coercion (threat, bullying, intimidation)
- positive coercion (excessive positive rewards)
- freedom to withdraw
- freedom from coercion
- voluntary
- informed
- risk
- clear communication of risks
- procedures for identifying and dealing with untoward consequences
- purpose
- freedom to withdraw and consequences
- risk
- Who can give consent?
- competence (often assumed due to status)
- status (adult)
- vulnerable populations (e.g., children, prisoners, pregnant women)
- From children we may seek “assent” [they agree to participate–this has no legal standing], from parents/guardians we must seek “consent”
- Quality of research
- unbiased, objective, clearly conceptualized
- makes a contribution to our shared knowledge and understanding
- potential benefits outweigh potential risks
- results communicated accurately and responsibly
- honesty in reporting results
- data/methods made available to others for replication
- unbiased, objective, clearly conceptualized
- Integrity of researcher
- public acknowledgement of potential conflicts of interest
- disclosure of financial entanglements
- source of funding
- accurate acknowledgment of contributions of others
- credit for contributions to the research: authorship
- credit for contributions to conceptualization: references
- plagiarism
- public acknowledgement of potential conflicts of interest
- Informed consent of participants