Please enable JavaScript.
Coggle requires JavaScript to display documents.
ADRESSING THE THEORY CRISIS IN PSYCHOLOGY - Coggle Diagram
ADRESSING THE THEORY CRISIS IN PSYCHOLOGY
INTRODUCTION
DISCOVERY-ORIENTED RESEARCH
Theory doesn't strong imply the hypothesis (evidences chosen from a broad class)
A failure doesn't disconfirm the theory
At the same time a confirmation of the theory is not relevant
The theory doesn't predict very well the phenomenon
Lower base probability enanches false positive
High probability to committ Type I error
Importance in reducing it!
THEORY-TESTING RESEARCH
Theory is strictly related to the hypothesis (evidences chosen from a broad class)
Ideally, theory predict the hypothesis 100% (P(X|T) = 1
In the reality, the theory prediction depends on strenght of logical link
Increasing P(X|T)
Lower level of false positive
An acceptable false positive here, is intollerable in discovery-oriented research
Researchers do discovery-oriented research based on theory-testing criterion!
Decreasing P(X|-T)
The conclusive way to go over the theory crisis in psychology
Formulating theoretical ideas more precisely, also incorporating the uncertainty
Example of Bayesian modeling framework
Focus on exploit evidence in data (methods to demonstrate the hypothesis is false)
Aim to discover the reasons for "replication crisis", derived from a weak link between theories and empirical tests
Scientific reasoning based on two levels
Empirical
From datas to hypothesis
Much discussed for the crisis
Inductive Inference (real effect)
Deductive Inference (generalizable effect)
Theoretical
From theory to hypothesis
PROPOSED REMEDIES FOR THE REPLICATION CRISIS
STATISTICAL STANDARDS AND DIRECT REPLICATION
Better for discovery-oriented, Not Better for theory-testing
For theory-testing, better testing a second hypothesis
Especially if there is a strong link between theory and hypothesis
EXPLORATORY AND CONFIRMATORY RESEARCH
Importance of temporal order
Confirmatory researches
Hypothesis and data analysis plan BEFORE observing the data
Good practice
Exploratory researches
Hypothesis and data analysis plan AFTER observing the data
Bad practice
HARKing (post-hoc hypothesis)
Paradox of predictivism
1) Better a confirmation from a prediction non considered by the researcher
2) We have not to consider the history of researcher's state of mind
How to solve it?
Doesn't consider value of temporal order per se (philosophers)
Preferring prior prediction to post hoc explanations
Difference in degree of indipendent gustification
But prior prediction must have strong link with theory
No need auxiliary assumptions
Cause they can fail, the probability of confirming hypothesis reduces
PREREGISTRATION AND RESEARCHER DEGREES OF FREEDOM
Problem of multiple comparisons
Increasing possibility to committ Type I error
Same hypothesis, multiple analysis paths
Risks of p-hacking
Reducible by pre-registration
Per se, not increase prior, but help to think
DF in data analysis, compromise assesment of the evidence
Same Analysis path, multiple hypothesis
Increase to committ Type I Error among all hypothesis tested
Joint Null Hypothesis
Problem in fishing expeditions
DF in hypothesis selection, implies low priors of the hypothesis
From data to empirical generalizations: Degrees of freedom in analysis plans
In choosing different analysis paths alternatives, we are biased by possible outcomes
How to solve it?
Multiverse Analysis
Run all posssibilities (or a sample) and consider the degree of convergence
Considered the exploratory way
Preregistration
Chosen one of them
Considered the protective way
But not influence on thinking of researcher, only arbitrary choosing of one of the possibility
Cannot replace thinking
Not always the solution
From theories to hypothesis: Degrees of freedom in hypothesis selection
If the link with theory is vague, I can support my hypothesis with auxiliary assumptions
Auxiliary Assumptions chosen post-hoc to support my hypothesis
Especially when discovery-oriented research isn't supported by theory (HARKing bad reputation)
More difficult a future predictive success for this hypothesis
Preregistration reduces researcher's degrees of freedom
Discovery-oriented
Increased multitude of data transformations
Theory-testing
Increased multitude of hypothesis to motivate
TOWARD STRONGER THEORIES: FORMALIZATION AND COMPUTATIONAL MODELING
Free parameters and arbitrary assumptions
Considering flexibility in the models
Constrain values of free parameters
Placing bounds on parameter values
Reducing flexibility, increasing power function
Considering some parameters like universal free parameters
They let us determine prior distributions
Decisions fo build a formal model
Free decisions but not all equally justified
Defining the justified means reducing degrees of freedom
Does formal modelling exaggerate the replication problem?
Fiedler considers formal models a problem for replication crisis
Problem of specificity
Specific assumptions about psychological mechanisms have low prior probability
Conseguently, prior probability of hypothesis is low
Confirmation of hypothesis is not diagnostic
Other models can entail same hypothesis
This assumption neutralizes the first
If hypothesis is implied in more models, increases its prior probability
Multiple models can do the same prediction
Probability generally high in the theory-testing researches
We have also to consider equally prior that hypothesis is false and prior hypothesis is true
In theory-testing the two probabilities are not equal
He reccomends to derive hypothesis from necessary statistical truths (but it's not allow to gain knowledge)
Deriving Theories from mathematical analysis or formal logic, also using simulations
Help us discover if an hypothesis follows the theory (simulation)