Please enable JavaScript.
Coggle requires JavaScript to display documents.
Week 5 - Coggle Diagram
Week 5
Demonstrating Rigor & Quality
Criteria for Evaluation
EPISTEMOLOGICAL CHOICES INFLUENCING CRITERIA
RELIABILITY, VALIDITY AND REALISM
more in quantitative paradigm
phenomena under study exist independently of the researcher and the research endeavour
explore very specific questions
as self-explanatory, unproblematic and perhaps emerging from a review of the literature or prior research.
Criteria
Reliability
the degree to which findings can be deemed accurate and repeatable
asking another researcher independently to code a portion of the data and comparing coding frame
Validity
the extent to which claims for the findings truly reflect the nature of the phenomena under study
reference to an ‘expert’ group
Generalisability
the degree to which findings have explanatory power within other contexts and situations
enhanced by describing in detail the context and participants.
TRUSTWORTHINESS AND SUBTLE REALISM
‘naturalist’ (Lincoln and Guba, 1985) or ‘critical realist’ (Cook and Campbell,1979).
underlying single reality that is to be explored,
various parties or players will have different views and explanations about what is happening.
criteria proposed by Lincoln and Guba (1985)
Trustworthyness
3 stratergies
Triangulation
different methods are used to elicit information about the same phenomenon
thick description
extensive details about both the context and the participants are included
audit trail
Researcher demonstrates how her work and thinking progressed throughout the project with the use of verifiable documents, such as a research diary, personal memos, dated computer files etc.
REFLEXIVITY, PRAGMATISM AND RELATIVISM
deny the existence of any external reality, claiming that nothing exists independently of the language and perspectives that bring phenomena into being (Edwards et al., 1995).
not to privilege their own interpretation over competing explanations,
instead presenting it as one of a number of possible representations.
reflexively ‘write themselves into’ their accounts.
Not like established criteria.
alternatives
reflexivity
conscious and systematic efforts to view the subject matter from different angles’;Alvesson, 2002
pragmatism
the extent to which knowledge generated is practically useful and helpful
Different paths
Ontology
way they view the nature of the social world
Epistemology
the way knowledge is constructed
influences which criteria
Position - Key questions
How are reality and truth represented
What - claim to be accessing when generating data
How stable and universal do they claim their interpretation
What roles - when interpreting data?
How - authors - competing explanations about the phenomena under investigation
How - own views, agendas and experiences feature in descriptions of their work
World view
view the data as ‘out there’, waiting to be garnered or elicited?
Interpretations as constructions, perhaps fashioned by yourself alone or jointly with research participants
participants’ beliefs and attitudes? or accounts and representations?
interpretation is the correct way of understanding the data // alternative explanations originating from the same data?
‘CONSIDERATIONS’ FOR EVALUATION
coherence
matching
research aim
, view of research endeavour
researchers’ accounting for their role
epistemological approach adopted
systematic research conduct
depending on the methodology adopted
convincing interpretation
account of researcher role.
EVALUATION IN PRACTICE
Eight “Big-Tent” Criteria for
Excellent Qualitative Research
Why Criteria and Why This Model?
Unhelpful
Bochner (2000) argues that traditional empiricist criteria are unhelpful and even “silly
universal criteria are problematic, if not fruitless (Guba & Lincoln, 2005)
Schwandt (1996) argues - virtual cult
Criteria formed
(e.g., Bochner, 2000; Schwandt, 1996)
(Ellingson, 2008; Golafshani, 2003). - more flexible and contextually situated
Lather (1993) - Poststructuralist -
Reasons
Useful
encourage dialogue with members of the scientific, experimental, and quantitative communities
Rules and guidelines help us learn, practice, and perfect
s helpful pedagogical launching pads across a variety of interpretive arts.
frame our work, if desired, as systematic and structured
different areas
—narrative, phenomenological, grounded theory, ethnographic research, and case study research
Eight Criteria of Quality in Qualitative Research
Rich Rigor
Requisite variety : a concept borrowed from cybernetics, refers to the need for a tool or instrument to be at least as complex, flexible, and multifaceted as the phenomena being studied
face validity—which is concerned with whether a study appears, on its face, to be reasonable and appropriate (Golafshani, 2003)
evidence their due diligence, exercising appropriate time, effort, care, and thoroughness.
Means to achieve rigor
Enough Data
time spent
Appropriate context & samples
Practices
care and practice of data collection and analysis procedures.
transparency regarding the process of sorting, choosing, and organizing the data.
Worthy Topic
Sincerity
end goal can be achieved through self-reflexivity, vulnerability, honesty, transparency, and data auditing.
Self reflexivity
frank about their strengths and shortcomings
examine their impact on the scene and note others’ reactions to them.
self-reflexive commentary - subjective feeling and sense making
use of the first person voice
“how much self-reflexivity
show rather than tell self-reflexivity by weaving one’s reactions or reflexive considerations of self-as-instrument
Transparency
honesty about the research process.
Auditing
“a methodologically self-critical account of how the research was done”
“clear documentation of all research decisions and activities”
credit is given where due in terms of author order and acknowledgements to participants, funding sources, research assistants, and supportive colleagues.
authenticity and genuineness
Credibility
trustworthiness, verisimilitude, and plausibility of the research findings
achieved through practices including thick description, triangulation or crystallization, and multivocality and partiality.
Thick description
in-depth illustration that explicates culturally situated meanings (Geertz, 1973) and abundant concrete detail (Bochner, 2000)
show rather than tell.
Crystallization and triangulation.
Triangulation
if two or more sources of data, theoretical frameworks, types of data collected, or researchers converge on the same conclusion, then the conclusion is more credible
aimed to rid research of subjective bias. - Realism
Crystalisation
relates to the practice of using multiple data sources, researchers, and lenses—but is motivated by poststructural and performative assumptions—is crystallization
open up a more complex, in-depth, but still thoroughly partial, understanding of the issue.
Multivocality
Closely aligned with the notion of crystallization and showing rather than telling, is multivocality.
the verstehen practice of analyzing social action from the participants’ point of view.
the verstehen practice of analyzing social action from the participants’ point of view.
Member reflections
the verstehen practice of analyzing social action from the participants’ point of view.
allow for sharing and dialoguing with participants about the study’s findings, and providing opportunities for questions, critique, feedback, affirmation, and even collaboration
test of research findings as they are an opportunity for collaboration and reflexive elaboration.
resonance
refer research’s ability to meaningfully reverberate and affect an audience.
empathic validity
The potential of research to transform the emotional dispositions of people and promote greater mutual regard has been termed “empathic validity” by Dadds (2008)
2 practices
Aesthetic merit
text is presented in a beautiful, evocative, and artistic way
“interactive introspection”—a method in which researchers consciously self-examine how their felt emotions affect their private and social experiences.
Transferability and naturalistic generalizations.
potential to be valuable across a variety of contexts or situation
Transferability : readers feel as though the story of the research overlaps with their own situation and they intuitively transfer the research to their own action.
Naturalistic generalization
readers make choices based on their own intuitive understanding of the scene, rather than feeling as though the research report is instructing them what to do
Significant Contribution
Theoretically significant research
“intellectually implicative for the scholarly community” (Tracy, 1995, p. 210), extending, building, and critiquing disciplinary knowledge
examining how existing theory or concepts make sense in a new and different context.
Heuristic significance
Providing readers with suggestions for future research
Practically significant research
is the knowledge useful
Catalytic validity
political consciousness to cultural members to act
Tactical authenticity
Train participants to political action
Phronetic research
enable practical wisdom and space for transformation
Following, Playing, and Improvising