Please enable JavaScript.
Coggle requires JavaScript to display documents.
Critiquing qualitative research - Coggle Diagram
Critiquing qualitative research
Critical thinking
A questioning approach to psychological studies, actively evaluating if claims made are valid
Truth seeking, trying to be objective and honest about what is really the case
Confidence to challenge ideas that lack evidence or reason
Open minded, considering different point of view on other merits
Inquisitive and curious, a desire to fully understand and be knowledgeable about a problem, a phenomenon or theory
Organised and systematic with evidence and reason
Creative, original ideas about the topic
In psychology -
Psychology offers answers to important questions in health, society, the environment as well as fundamental scientific questions
We need to look at the answers / data we are given to these questions, examine the conflicting evidence and figure out what we believe is true
For QRM:
Epistemology and reflexivity
Credibility, transferability, dependability, confirmability, sensitivity to context, commitment and rigour, transparency, coherence, reflexivity, impact and importance
Differences between quantitative and qualitative research:
Quantitative - aims to quantify, generalise, large samples, structured and standardised data collection, numerical and statistical analysis and outcome of a specific relationship
Qualitative - aims to narrate an experience, to specify, small samples, semi-structured and flexible data collection, non-numerical analysis and outcome of an experience being described in someone’s own words
-> Should draw you in, immerse yourself in the experience, narratives and perspectives
-> Should not be on the outside looking in, you should be on the inside
-> Permeability - the capacity of understanding to be changed by encounters with observations (Stiles, 1991)
How does this affect evaluation of research?
We are expecting different types of research to achieve different things -> therefore we must use different metrics
Terms used in quantitative evaluation -
-> Validity
-> Reliability
-> Generalisability
-> Reproducibility
However, for qualitative, evaluation must be done in the appropriate framework, and the process / procedure must be sufficiently exposed for others to evaluate
Yardley’s (2000) four broad principles for assessing the quality of qualitative research
Sensitivity to context
Being aware of the context
Sensitivity to culture, prior literature, quotes provided
In the interview, appreciation of the interactional nature of data collection - good IPA interviews require dedication, awareness and skill
Sense-making of the participant’s experiences (analysis) requires immersive and disciplined attention to detail
Careful writing, only making claims appropriate to sample interviewed
Commitment and rigour
Carrying out detailed analysis:
Attentiveness to the participant during interviews and ensuring the participant is comfortable
Active listening
Care for each case in analysis
Personal commitment and investment of time
Rigour - quality of being thorough and careful:
Thoroughness of the study, systematically conducted
Rigorous research must be transparent and explicit
For IPA, we would expect each theme to be supported with quotes from several participants
several participants
Should be -
-> Focused - specific
-> Clear in paradigm - critical realism etc
-> Have an informed research design - question meets method, paradigm and outcomes
-> Be clear and transparent
-> Be reflexive - awareness of role in methods
Methods to achieve rigour
Triangulation:
Explore the topic in various ways
Similar results from each perspective present a more valid picture
Collecting data from more than one source using different methods
Using various approaches to confirm results
Using different methods to confirm outcomes
Can triangulate through approach and by method
Respondent validation / participant feedback:
Ask the participant for their confirmation of accuracy
Could include - returning interview transcripts to respondents to verify the accuracy of what was said, returning analysed data to respondents via synthesised theme, discussing the interview transcript with the respondent, additional interview based on notes
Member checking - peer review with your team:
Explore credibility by having members of the team check each other’s work
Everything coded multiple times - check coding agreement
Disconfirming cases:
Once themes are identified, disconfirming cases can be searched for
Choosing cases based on whether they confirm or disconfirm patterns in the data
Unexpected findings can strengthen the analysis
Can help you to understand the limitations of your research findings
Paper trail:
Record of how research was conducted
Provides evidence linking raw data to results
Includes coding, coding manuals, evidence of how themes were developed, how the data was interpreted, memos
Demonstrates trustworthiness and transparency, allows replication
Could keep a lab book of notes
Reflexivity - being aware of your own biases:
Reflect on research process, thinking about how your background, views and knowledge or experience could have shaped the interview schedule, interpretation, emotional response and group analytic process
What did you do to try and minimise this?
Think about impact and importance of research
Cannot use valid measurement tools like we can in quantitative, so we establish rigour in other ways
Think about benefits of this approach over quantitative e.g. deeper understanding
Transparency & coherence
How well are the stages of the research study described
Detailed description of - recruitment, interview schedule construction, how interviews were conducted and steps of analysis (Smith et al, 2009)
Being able to see how everything works and showing people how everything works
-> If something went less well, acknowledge it
-> Exclusion, coding, knowing the interviewee, starting position, evidence
Coherence -
-> Is there a coherent analytic story, is the argument coherent, to the themes fit together logically, has the researcher dealt with ambiguities and contradictions clearly?
Impact & importance
Does your research tell the reader something interesting, important and useful?
This is a good test of the validity of qualitative research
What makes it stand out
Other relevant terms for qualitative research:
Credibility - extent to which results are trustworthy
Confirmability - extent to which findings represent situation being researched over the beliefs and biases of the researcher
Dependability - extent to which study procedures are consistent and repeatable - coherence between introduction, methods and findings
Transferability -
-> Relevance of concepts and findings to other settings
-> Transferability is established by providing readers with evidence that the study’s findings could be applicable to other contexts, situations, times and populations
-> Degree to which results can be generalised or transferred to other contexts or settings (qual) v extent to which statistic applies to whole population (quant)
Trustworthiness - do people outside believe work / interpretation
-> Strategies to achieve this - transparency, rigour, reflexivity