Please enable JavaScript.
Coggle requires JavaScript to display documents.
The Replication Crisis in Social Psychology (Y2) - Coggle Diagram
The Replication Crisis in Social Psychology (Y2)
Criticisms of social psychology
Giner-Sorolla, 2019 -
Limited populations, artificial measures, not effective in communicating research findings to the public and lack of pre-registration of research plans to registry’s such as the the Open Science Framework
Incentive structure - published work is important in getting a job, tenure, awarding of grants and being viewed favourably
As a result, people try to publish as much as they can
Balancing the desire to stay truthful to psychological science with necessity to publish
https://www-tandfonline-com.surrey.idm.oclc.org/doi/pdf/10.1080/10463283.2018.1542902?needAccess=true
Replicability - Nosek et al, 2022 - testing the reliability of prior findings with different data; same study but different participants, do you get the same data?
The credibility of scientific findings depends in part on the replicability of the supporting evidence - the more positive evidence, the stronger the conclusions
Replication seems stragithforward - but it is not easy to determine what is the same study rather than the same evidence
Direct replication - everything is kept the same; study population, procedure and measures (Shrout and Rogers, 2018)
Systematic replication - Shrout and Rogers, 2018 - it is a direct replication in which some secondary features have been changed such as order of stimuli presentation
Conceptual replication - intentionally different from a direct replication - designed to examine the validity and generalisability of the original findings, similar but the same as the original study - alternatre measures or sample from a different population
Such as a different age group or measure
Reproducibility - Nosek et al, 2022 - testing reliability of findings by using the same data and analysis strategy - re analyse to see if findings are the same
Original analysis sometimes cannot be repeated because data is not available or software is not
Robustness - Nosek et al, 2022 - testing reliability using the same data but using a different analysis
Fragile findings are not wrong, but they risk generalizability and replicability
The replication crisis
Giner-Sorrolla, 2018 - 1960-70; social psychology started to express doubts about the validity of their research
Common practice of declaring an effect based on a single piece of statistically significant data
This criterion, as it encouraged publication bias in terms of favourable results, was seen as risking unsupported conclusions
Single published experimental results often failed to replicate, leading to equivocal literature
Findings that have not been replicated - Strack et al, 1988 - seminal study on facial feedback hypothesis
the idea that facial expressions do not only reflect feelings but cause them
Found that those forced to smile at cartoons found them funnier than those forced to pout
Wagenmakers et al, 2016 - found these results to not be replicable
https://web-p-ebscohost-com.surrey.idm.oclc.org/ehost/pdfviewer/pdfviewer?vid=0&sid=47de985b-cc6d-416e-9283-93f8618ce7c1%40redis
Large sample sizes are needed - this reaffirms conclusions
Sometimes difficult to establish if the replication or original study is wrong - multiple replications reinforce the probable likely answer to a posed question
Issues in replication - social norms, ability to get similar samples, ability to repeat studies due to ethics, ability to see original data, accessibility to analysis software
Although 80% power level is most practical, it still leaves a very large room for error as 20% of studies might not be trustworthy
Also does not take into account original sample variability with different size
Sample size also relies on presumed effect size, and if this is unknown this can cause doubt
Original studies are frequently underpowered - theories of using confidence intervals (Yuan and Maxwell, 2005) and taking into account effect size uncertainty and censoring (Taylor and Muller, 1996) are impractically applied
Interpreting null replications with apparently appropriate power is complicated - no deficient study can be trusted in comparison
Making any theory fact off of one original study and one replication is unwise - need to create a bank of probability with many replications to create consistency
Boundary conditions and moderators need to be established effectively to improve generalisability
Solutions to the replication crisis
Publishing null and significant findings
Rigorous and transparent methodology - increase disclosure in hypotheses methods and results presentation
Pre-register hypotheses and studies - data collection rules and analytic strategies
Share data
Integrity - doing research ethically and responsibly
Solution - meta analysis all replications (Alogna et al, 2014) to get a confidence interval, and if there is no zero value an effect exists, but this involves using 0 as a confidence interval in all replications; meta analysis showed a significant effect even if no replications individually did so
Hedges (1987) - multiple studies of the phenomenon and using a meta analysis is a good solution
Physical sciences also do not always fully replicate a theory, and so use of intervals produced by a body of work gives way to an effect
Need appropriately powered studies and to address practices of inflated effect size estimates
Replication is important - but single replication is not enough
Statistical replication is a key area of focus, solved by meta analysis, but methodological replication is also important (Brandt et al, 2014)
Statistical power - use of statistically incorrect sample size may lead to inadequate results
Determination of the effective sample size is crucial to enable efficient study with high significance - can be determined through power analysis
The statistical power of the study (Sensitivity) is how likely the study is to dsitinguish an actual effect from chance
Generally set to 80%
80% of finding that the effect exists