Please enable JavaScript.

Coggle requires JavaScript to display documents.

Mixed Methods Literature Review #, Qualitative Rigor, a - Coggle Diagram

- - - - Quantitative method and a deductive approach
        
        Include experimental and quasi-experimental designs, econometric models, survey methods, quantitative case studies
        
        are typically aligned with positivist/post-positivist presuppositions
      - Qualitative methods and inductive/abductive approach
        
        normally aligned with interpretivist presuppositions and tend to favor methods that focus on language and representation
        
        such as narrative inquiry, qualitative case studies, ethnographic, phenomenological, hermeneutical or historical analysis, and participatory research
    - - Rationales for selecting a MM design (part of what and why)
      - Illustrates the options available when sequencing methods (how)
      - Draws attention to the importance of connecting method, hence results (how)
    - - Most frequent reason is triangulation
      - MM are chosen over single method design with the aim to
        
        Corroborate or complement findings, thus increasing the validity of results
        
        Gain multilevel understanding of the phenomenon under study.
    - - the logical and chronological combination of methods in a design
      - Leads to employ a parsimonious classification of MM: Parallel or sequential of core mixed methods designs
      - Two main types
        
        Parallel design
        
        Parallel/simultaneous/concurrent
        
        Two parts of the research are to be conducted separately from one another.
        
        Their findings are analyzed independently and compared to gain a better understanding of the problem, corroboration, and validation purposes.
        
        to triangulate results that serve the same research purpose, although they are obtained through different methods.
        
        Sequential design
        
        Includes two or more consecutive phases, quanti-qualitative (explanatory) design or quali-quantitative (explanatory) design.
        
        In the quanti-qualitative - the 2nd phase is aimed at explaining and refining the results in the 1st phase
        
        because it may be puzzling because they differ from expectations.
        
        may feel the data are not revealing enough and employ qualitative methods to understand the causal mechanism assumed to underlie correlational or causal quantitative findings
        
        The qualitative phase may contribute to a refinement of the findings, by shedding light on the inner views of the actors
        
        In the quali-quantitative - the 2nd phase is used to augment the findings of the 1st phase.
        
        The fist phase generates the main innovative contribution of the study
        
        The second is meant to test the findings, thus generalizing or refining them
        
        A hybrid design type could be included - such as qualitative-quantitative-qualitative method sequence
    - - Every MM design needs integration through connecting points between the research components
      - In a parallel design, integration requires assembly and comparison of the results obtained from the two different methods
      - In quanti-qualitative (explanatory) design, the key stage for integration should be the connection between the results of the first and the data collection of the second phase.
- - - - 71 pass screened method
        
        103 identified through key words
        
        32 excluded
      - 37 excluded
    - - 117 pass screened method
        
        2147 menually screened
        
        2030 excluded
      - 47 excluded
  - - - Interviews (78 out of 104 studies, 75%)
    - - Survey data (45 out of 104 studies, 43%
      - Archival data (34 out of 104 studies, 33%)
    - - Qualitative and quantitative analysis of interviews
      - Qualitative and quantitative analysis of archival data
      - Quantitative analysis of archival data mixed with case studies
      - Quantitative analysis of survey data mixed with case studies
      - Qualitative and quantitative analysis of data from the same survey
  - - - Interview combined with archival data (7 out of 32 studies, 22%)
      - Interview combined with survey data (12 out of 32 studies, 38%)
      - Interview combined with interviews (2 out of 32 studies, 6%)
      - Archival data as both qual and quant analyses (4 studies (13%)
      - Survey with both closed and open ended questions (4 studies, 13%)
    - - A quanti-qualitative (explanatory) design (39 out of 72 studies, 54%)
        
        Example
        
        Empirical critique of performance systems and New Public Management (Soss, Fording, and Schram, 2011, 204)
        
        Use administrative data
        
        Establishing a pattern linking performance pressures on providers to sanctions imposed on clients
        
        Field research on case manager discretion to explain relationship and clarify underlying mechanisms
        
        Study on representative bureaucracy investigating the effects of teachers' representation on teens pregnancy rate
        
        Combined linear regression with interview
        
        To substantiate the hypotheses and to broaden and deepen the understanding of the possible causal mechanisms that underpin them
        
        Usually driven by the desire to delve deeper in the findings
      - A quali-quantitative (explanatory) design (29 out of 72 studies, 54%)
        
        Employed the qualitative phase to generate either hypotheses or questions that are then tested in a survey
        
        Often did so when facing theoritical alternatives
        
        Example
        
        A study on the effects of reputational threats on the enforcement pace of a public agency
        
        Started with a preliminary qualitative analysis to identify the specific aspect of the agency's reputation that was challenged
        
        This enabled researchers to identify an appropriate testable hypothesis for the quantitative analysis selecting from two theoretical alternatives
      - Explanatory and exploratory sequential (hybrid) design (4 out of 72, 6%)
        
        A typical case was a study starting with the qualitative analysis of interview
        
        that informed the survey questions
        
        Followed by another distinct qualitative focus to understand the survey results in greater depth
    - - All the 32 studies had empirical context as a connecting device
      - There did not seem to be a specific type of context for a MM design
      - A more compelling connecting point that should characterize all parallel MM studies is the combination by comparison or integration of the finding obtained through separate research processes
    - - Found studies where the qualitative interviews of the first phase informed the survey questions rarely showed how the survey questions were connected to the qualitative phase.
      - Example An article on collaborative policy making outcomes the authors merely stated that they constructed the online survey with the support of interviews
      - Albeit they provided no evidence yielded by the interviews
- - - - Generalizability
        
        In deductive sample, observations that not conform to the model disregard as error or outliers, may be removed entirely.
        
        The aim of deductive inquiry is to test the presence of an a priori relationship in the population based on a limited, representative sample
        
        Inductive research attempt to do something quite different.
        
        The sample is never purely random nor convinient
        
        Instead, each case or participant should be purposively selected because they represent a theoretically interesting exemplar of a phenomenon of interest.
    - - Replication
        
        Inductive methods are seeking to develop a deep understanding of causal factors, processes, nature, meaning, and/or significance of a particular phenomenon
        
        Each tradition (phenomenology, ethnography, etc) has its own systematic process of data collection and analysis.
        
        Regardless of the tradition, the analyst must draw inference and interpretation from the vast array of qualitative information in front of them.
        
        Just like doctors working independently on the same data sources may also come to different interpretations. One is not necessarily right and the other wrong.
        
        In inductive analysis, the appropriate criteria of rigor is based on the evidence provided, is a given interpretation credible?
  - - - Research design
        
        Sampling
        
        The number of sampling
        
        Qualitative saturation - to demonstrate that the dataset is robust in terms of capturing the important variability that exists around the phenomenon of interest.
        
        Data collection protocols and procedures
        
        The investigator should be able to clearly describe what data was considered relevant for a given research question and how this data was obtained.
        
        Data collection protocol design should be consistent with the specific methodological tradition embraced by the study
        
        Data analysis and interpretation - no magic behind hundreds of pages of transcript transformed into findings, there should be provided a clear and concise description of the analysis process and its relationship to the reported findings.
    - - Whether the interpretation is credible in light of the data
      - Whether it was the result of a robust and systematic analytical process designed to move beyond superficial findings and minimize and/or account for investigator bias
      - Whether it is reported with sufficient attention to context so as to facilitate the potential relevance of insights to similar contexts.
- - - - Qualitative methods are useful in the theory development process
        
        Seminal work on the role of routines as sources of change and innovation in organizations was based on organizational ethnography (Feldman 2000, Feldman and Pentland 2003)
        
        Organizational culture and sense making case studies (Schein 2003, Weick 1993)
        
        Case study of a CEO in crisis (Toepler's 2005)
        
        The phenomena of iron triangles (Freeman 1965)
        
        The social construction of target populations (Schneider and Ingram 1993)
      - Qualitative methods have a strong tradition of clarifying and expanding upon existing theory
        
        Phenomenological approach to present the views of public workers who must navigate their administrative and political responsibilities every day (Maynard-Moody and Kelly (1993)
        
        Agency and stewardship theories (Schillemans 2013)
        
        Theories of goal-directed networks and managerial tensions around unity and diversity (Saz-Carranza and Ospina 2010)
  - - - Lipsky (1971) examination and discussion of the street-level bureaucrat.
      - Public Service Motivation (PSM) was born from Perry and Wise's (1990) discussion citing both cases and quotes from public servants.
- - - - have dual burden of adhering to all the requirements of rigorous design associated with both inductive and deductive models. - This generally makes such studies relatively large and ambitious.
      - Should ideally reflect some degree of complementarity between the approaches, maximizing the different advantages in inductive vs deductive designs.
- - - - The precept that finding must have relevance, is common to both qualitative and quantitative scholarship in public management.
      - Focus is to inform practice as well as advance theory
    - - Using the right tool for the job is a value common to both inductive and deductive traditions
      - Different research questions need appropriate designs. Inductive approaches are designed to pose and address fundamentally different kinds of questions that necessitate different types of data and criteria of rigor.
    - - Both inductive and deductive need a community of scholars to build a body of knowledge.
      - We seek to investigate something overlooked, under-appreciated, or potentially misunderstood in our current understanding of a phenomenon.
      - Those contributions are a shared element of quality for both approaches.