Criteria for Assessing a Classic Grounded Theory Study: A Brief Methodological Review with Minimum Reporting Recommendations

Kara L. Vander Linden, Saybrook University, USA

Patrick A. Palmieri, South American Center for Qualitative Research, Perú

Abstract

Introduction: Reporting criteria for research studies are essential to assess the methods and to evaluate the usefulness of the findings. The purpose of this review was to identify the essential criteria to report a classic grounded theory (classic GT) study.

Method: A methodological review of the reporting criteria for a classic GT study.

Results: Grounded theory studies generally report theoretical sampling, theoretical saturation, memoing, and constant comparative analysis. In addition, classic GT studies reported unstructured interviews, a grand tour question, substantive and theoretical coding, and hypothetical probability statements. However, they did not report comprehensive literature reviews. An early focus on useability of the resulting theory was expanded to include criteria for fit, understandability, relevance, grab, general, work, control, and modifiability.

Conclusion: Essential criteria were identified for reporting grounded theory research with differentiations for classic GT. The classic GT criteria should be included as a reporting extension to complement the existing reporting guidelines.

Key Words: Classic grounded theory (Classic GT), theoretical sampling, theoretical coding, theoretical sensitivity, theoretical saturation, constant comparative analysis, grand tour question, reporting guidelines, peer-review, research evaluation, COREQ, SRQR, EQUATOR

Introduction

Recommended reporting criteria are essential for researchers developing a manuscript to guide the disclosure of their research methods and findings (Moher et al., 2008). This paper is the first in a series about the methodological processes and procedures that should be disclosed in a manuscript reporting a classic grounded theory (classic GT) study. Reporting criteria are also important for reviewers tasked with evaluating the methodological quality of a manuscript reporting a qualitative research study. In this regard, the reviewer is responsible for carefully assessing the manuscript to identify study limitations requiring revision, or fatal flaws requiring rejection. All qualitative research methods have established procedures to demonstrate rigor and techniques to establish trustworthiness. The same is true for classic grounded theory.

The purpose of this methodological review is to identify the essential reporting criteria for qualitative research manuscripts reporting a classic GT study. First, an overview of the methods and results sections will be described in terms of the research protocol and the work to implement the study. Next, the minimum reporting criteria for a classic GT study are identified and discussed. Then, the importance of reporting criteria for a classic GT study are explained within the context of the two primary guidelines for reporting qualitative research. Finally, minimum reporting criteria for a classic GT study are recommended as an extension to strengthen existing guidelines for reporting qualitative research.

Research Designs and Protocol Development

The protocol for a research study is a plan for how a study will be conducted (Salkind, 2010). The protocol should be based on the methodological norms for the selected study design with a clear explanation about how the procedures will be operationalized. Researchers should cite the appropriate methodological literature to support the procedures. The protocol should guide the implementation of the research study. Although implementation should follow the protocol as closely as possible, minor variations are common occurrences, especially within the context of qualitative research. Any variation from the original research protocol, should be clearly identified and briefly justified. The protocol implementation should result in a manuscript that clearly and concisely reports the study findings in alignment with the research methods.

Criteria for Reporting the Research Methodology for a Classic Grounded Theory

When reviewing a manuscript reporting a grounded theory study, the first criterion for assessment is the demonstrated familiarity with the multiple approaches to grounded theory. The three most common approaches, or versions, for grounded theory studies are classic (Glaser & Strauss, 1967), Straussian (Strauss & Corbin, 1990), and constructivist (Charmaz, 2006). Although the approaches appear similar in the case of inexperienced researchers, there are unique characteristics for the research protocol of a classic GT study essential for reporting in the manuscript. For this reason, the reviewers need to be able to clearly identify the approach, understand the justification, and assess the implementation.

Avoiding a preliminary literature review is a basic tenet of classic GT (Glaser, 1978). Thus, reviewers should examine whether a preliminary literature was conducted. There are times when a researcher must conduct a preliminary literature review, for example, to meet the requirements of a university or ethics review board. In these cases, the reviewer should look for a statement about how the researcher limited the influence of the literature during the study implementation.

A classic GT study should not begin with a predetermined research problem, whether it is based on literature or personal or professional experience (Glaser & Holton, 2004). Instead, within a classic GT study, the problem should emerge from the data analysis as the problem or issue that the people within the area of study are trying to resolve. Reviewers should be able to determine if the problem statement was developed in advance by the researcher or if it was discovered from an analysis of the data.

A general research question for a classic GT is articulated in general terms related to the topic area of the study. For example, a research question in a study on infertility may be framed as “what is the main concern (issues, problem) for people who are living with infertility, and how do they resolve this concern (issues, problem)?” This allows for the problem and theory to emerge from the data. Similarly, a purpose statement for the same study may be something like “The purpose of this study is to develop a theory about the main concern (issues, problem) for people who are living with infertility and the patterns of behaviors used to resolve this concern (issues, problem).” The purpose statement should indicate generating theory as a primary part of the purpose. Regardless of the exact wording used in the research question and purpose statement, the reviewer should consider if the question and purpose statement are sufficiently broad to allow both the problem and a theory about the patterns of behaviors used to address the problem to emerge from the data.

The next area for a review to consider is the sample and method of sampling. Within a classic GT study, the sample is comprised of people who have firsthand knowledge and experience within the topic area of study. However, later participants are selected based on theoretical sampling. Theoretical sampling involves analyzing each piece of data as it is collected and prior to the collection of the next piece of data so that the data analysis can guide what data to collect next (Glaser & Strauss, 1967). Within all the approaches to grounded theory, theoretical sampling is the specific sampling technique to support theory development. The technique guides the researchers to determine what data is needed to facilitate reaching theoretical saturation, the point where no new variations are among the significant concepts are found in the data (Glaser & Strauss, 1967). Due to the nature of theoretical sampling, the size of the sample cannot be predetermined. Some theories may have a small sample of 10-20 while other theories may have much larger samples. The sample size depends on the quality of the data and the scope of the theory. The reviewer should identify whether the initial sample seems appropriate to the topic area (i.e., did they have firsthand experience?), whether data were analyzed one piece at a time, and whether theoretical sampling was used to determine what data were needed next.

The purpose of grounded theory is to generate a theory, rather than suggest or verify a theory. Thus, the types of data and how they are collected and analyzed should align with this purpose. While classic GT may use any type of data (Glaser, 1998), interviewing is the most common form of data collection. When interviewing participants, researchers use unstructured, in-depth interviews with a broadly worded question about the topic area called the grand tour question (Simmons, 2010). Building on the previous examples, a grand tour question might ask “Can you tell me about your experience with infertility?” The question invites participants to discuss what is significant to them within the topic area. Probing questions are then used to focus on only the topics brought up by the participants. While engaging in theoretical sampling, the grand tour question usually changes and become less “grand” as the researcher works to theoretically saturate various aspects of the emerging theory. Reviewers should initially determine if unstructured, in-depth interviews along with probing questions were used for the research. Reviewers should question the fidelity to the classic GT if the researchers reported semi-structured or structured interviews.

Data analysis procedures in classic GT involves substantive coding, which includes open and selective coding, and theoretical coding later in the process (Glaser, 1978). Constant comparative analysis is central to data analysis in any grounded theory study, and memos are used to capture the theoretical ideas generated from constant comparative analysis (Glaser, 1965). Two aspects of data analysis often missed when reporting the protocol and implementation are sorting and creating a theoretical outline. These are tied to theoretical coding, which helps the researcher identify the theoretical structure for the resulting theory (Glaser, 1978). The reviewer should carefully determine whether all components of the data analysis are included, especially constant comparative analysis, as critical reporting elements.

After sorting and creating a theoretical outline, the final step is to “write up” the theory. At this point in the process, relevant examples are included in the theory and external evidence from the literature is referenced (Holton & Walsh, 2017). The structure and presentation of the theory is the result or research product. This is the next area for reviewers to evaluate for a classic GT study.

Criteria for Reporting Research Findings from a Classic Grounded Theory Study

For grounded theory, first and foremost, the results should be a theory (Glaser & Strauss, 1967). The theory that results should identify the theoretical concepts and relationships which are often expressed as hypothetical probability statements (Yalof, 2014). Hypothetical probability statements (Glaser, 1978) explain how one concept interacts with and affects another concept. Thus, the theory should be written conceptually (i.e., about the concepts) and theoretically (i.e., about the relationships). The product/result should not describe the data or participants as is common in many qualitative approaches. Thus, the theory should not reference the participants unless using them as an example of a theoretical concept or relationship. It should also not produce themes, as thematic analysis does.

Constant comparative analysis (Glaser, 1965) leads to the discovery of a core category (Glaser, 2002), which is the central pattern of behavior that addressed the problem that emerged from the data. In some literature, the core category is used interchangeably with core variable (Glaser, 1965, 1978; Glaser & Strauss, 1967) and core concept (Glaser, 2014). The core category is the central idea of the theory that explains how people within the area of study resolve what is problematic to them. All the other concepts in the theory relate to this core category and help explain it. The reviewer should first determine if the theory clearly identifies a core category and the centrality to the theory. Then the reviewer should determine if the other concepts relate to the centrality of theory and help explain the core category.

Glaser and Strauss (1967) recognized grounded theory required criteria to define the unique research design and, more importantly, to evaluate the resulting theory. When Glaser and Strauss (1967) first articulated grounded theory as a method, they presented criteria focused on establishing the useability of the theory. With time, Glaser (1978, 1998) expanded the criteria to include fit, understandability, relevance, grab, general, work, control, modifiability. These criteria are met by a theory grounded in the data.

The theory must closely fit the data and the substitute area. Because it fits the data and substantive areas, the theory should be understandable and relevant to both professionals and laypersons concerned with the area (Glaser, 1978; Glaser & Strauss, 1967). When a theory is understandable and relevant is often has grab, meaning it is interesting and is memorable (Glaser, 1978) to those within the substantive area. In addition, the theory must also be general enough to be “applicable to a multitude of diverse daily situations within the substantive area” (Glaser & Strauss, 1967, p. 237), allowing it to work by “explain[ing] what happened, predict[ing] what will happen and interpret[ing] what is happening” (Glaser, 1978, p. 4) in the substantive area and allowing the user a degree of control within the situation. Finally, the theory should also be modifiable as new data is introduced and as the substantive area changes over time.

When reviewing the “write up” of a theory, the reviewer should make sure the theory goes beyond merely describing concepts by explaining the relationships among the concepts. As mentioned earlier, these are often expressed as hypothetical probability statements that clarify how one concept influences other concepts within the context of the theory. Furthermore, Glaser (1998) discussed the need for a classic GT to be parsimonious. Thus, the theory should only contain the concepts that represent the applicable patterns of behavior and that are necessary to explain, predict and interpret what is occurring with the substantive area.

Beyond the criteria presented by Glaser and Strauss (Glaser, 1978, 1998; Glaser & Strauss, 1967), reviewers should also examine several elements related to how the theory is written. Simmons (personal communication May 25, 2004) explained that when “writing up” the theory, theoretical concepts are presented and discussed prior to providing examples from the data or the related relevant literature. Examples should be limited in use and primarily used to illustrate concepts or relationships that maybe clarified using an example. The theory should be written in the present tense with only examples from the data expressed in past tense.

Grounded Theory Research and Existing Reporting Guidelines

Appropriate criteria to assess the quality of a manuscript reporting a research study depends on many factors including the research approach, study design, methods, findings, and limitations (Guyatt et al., 2008).  Although there has been robust development of standard reporting criteria for most quantitative study designs (Simera et al., 2010), there has been minimal advancement for qualitative research. With inadequate reporting of qualitative study designs, reviewers cannot adequately assess the trustworthiness and rigor of a research manuscript and the readers are unable to interpret the validity of the data and the strength of the findings. Arguably, the lack of a reporting checklist and limited reviewer knowledge about existing classic GT reporting criteria has contributed to the remodeling of the method (Glaser & Holton, 2004).

Enhancing the Quality and Transparency of Health Research

The EQUATOR (Enhancing the QUAlity and Transparency Of health Research) Network established in 2006 is an international initiative that seeks “to improve the reliability and value of published health research literature by promoting transparent and accurate reporting and wider use of robust reporting guidelines” (Equator Network, 2021, p. 5) They have published evidence-based reporting guidelines (Altman et al., 2008) organized within eleven main study areas specific to study designs, methodological strategies, procedures and techniques, sections of manuscripts, and protocol development. For qualitative research, there are two main reporting guidelines, including the Consolidated Criteria for Reporting Qualitative Studies (Tong et al., 2007), or COREQ, and the Standards for Reporting Qualitative Research (O’Brien et al., 2014), or SRQR. Although the guidelines provide checklists for reporting the minimum criteria for a qualitative research study, neither provides appropriate criteria for reporting a properly constructed and rigorously conducted grounded theory study (Leyva-Moral et al., 2021).

Consolidated Criteria for Reporting Qualitative Studies

Consolidated Criteria for Reporting Qualitative Studies (COREQ) checklist was developed to support the adequate reporting of methodological criteria for interviews and focus group study designs. The 32-items in the checklist are recommendations derived from the literature for reporting studies using interviews and focus groups for data collection. The items are organized into three domains including research team and reflexivity, study design, and analysis and findings. The subdomains with individual criteria for quality reporting are specifically focused on the study design attributes associated with the trustworthiness and rigor of the interviews. As such, the COREQ is too narrowly focused to be useful for evaluating the special features defining a classic GT study.

Standards for Reporting Qualitative Research

Developed from an assessment of the qualitative research literature, the Standards for Reporting Qualitative Research (SRQR) checklist improves the transparency of reporting ‘all aspects’ of qualitative research. The 32-item checklist is more comprehensive in scope than the COREQ with the criteria for reporting organized into six areas aligned with the standard manuscript sections for reporting a qualitative study. The areas include title and abstract, introduction, methods, results and findings, discussion, and others (disclosures). Whereas the COREQ is heavily concentrated on reporting key criteria for conducting interviews, the SRQR is focused on the research design including reporting the data collection processes and analysis. In the context of classic GT, the SRQR is too broad to cover the narrow but essential reporting criteria identified in this article as conceptualized by the original, unmodified, works of Glaser.

Conclusion

This article identified the essential methodological criteria for reporting a classic GT study. The evaluation of a classic GT study requires a reviewer to determine how the study was constructed (protocol) and how the study was conducted (implementation) based upon essential reporting criteria. Despite the existing reporting criteria for classic GT studies, the current qualitative reporting guidelines are too general to identify a manuscript with an inadequate study design or attempts to remodel the method based upon researcher preferences. As such, the classic GT criteria discussed in this article should be incorporated into existing guidelines to preserve the integrity and rigor of grounded theory as a distinct research method. The criteria identified in this review should be included as a classic GT study design extension in the EQUATOR framework to support the existing qualitative reporting guidelines.

Disclosures

Declaration of Conflicting Interests: The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding: The authors received no financial support for the research, authorship, and/or publication of this article.

References

Altman, D. G., Simera, I., Hoey, J., Moher, D., & Schulz, K. (2008). EQUATOR: Reporting guidelines for health research. The Lancet, 371(9619), 1149-1150. https://doi.org/10.1016/S0140-6736(08)60505-X

Charmaz, K. (2006). Constructing grounded theory: A practical guide through qualitative analysis. SAGE Publishing. https://us.sagepub.com/en-us/nam/constructing-grounded-theory/book235960

Equator Network. (2021). EQUATOR Network: What we do and how we are organised. UK EQUATOR Centre. Retrieved November 1 from https://www.equator-network.org/about-us/equator-network-what-we-do-and-how-we-are-organised/

Glaser, B. G. (1965). The constant comparative method of qualitative analysis. Social Problems, 12(4), 436-445. https://doi.org/10.2307/798843

Glaser, B. G. (1978). Theoretical sensitivity: Advances in the methodology of grounded theory. Sociology Press.

Glaser, B. G. (1998). Doing grounded theory: Issues and discussions. Sociology Press.

Glaser, B. G. (2002). Conceptualization: On theory and theorizing using grounded theory. International Journal of Qualitative Methods, 1(2), 23-38. https://doi.org/10.1177/160940690200100203

Glaser, B. G. (2014). Applying grounded theory. Grounded Theory Review, 13(1), 46-50. http://groundedtheoryreview.com/2014/06/22/applying-grounded-theory/

Glaser, B. G., & Holton, J. (2004). Remodeling grounded theory. Qualitative Social Research Forum, 5(2). https://doi.org/10.17169/fqs-5.2.607

Glaser, B. G., & Strauss, A. (1967). The discovery of grounded theory. Aldine Publishing Company.

Guyatt, G. H., Oxman, A. D., Kunz, R., Vist, G. E., Falck-Ytter, Y., & Schünemann, H. J. (2008). What is “quality of evidence” and why is it important to clinicians? BMJ, 336(7651), 995-998. https://doi.org/10.1136/bmj.39490.551019.BE

Holton, J. A., & Walsh, I. (2017). Classic grounded theory: Applications with qualitative and quantitative data. SAGE Publishing.

Leyva-Moral, J. M., Palmieri, P. A., Loayza-Enriquez, B. K., Vander Linden, K. L., Elias-Bravo, U. E., Guevara-Vasquez, G. M., Davila-Olano, L. Y., & Aguayo-Gonzalez, M. P. (2021). ‘Staying alive’ with antiretroviral therapy: A grounded theory study of people living with HIV in Peru. BMJ Global Health, 6(10). https://doi.org/10.1136/bmjgh-2021-006772

Moher, D., Simera, I., Schulz, K. F., Hoey, J., & Altman, D. G. (2008). Helping editors, peer reviewers and authors improve the clarity, completeness and transparency of reporting health research. BMC Medicine, 6, 13. https://doi.org/10.1186/1741-7015-6-13

O’Brien, B. C., Harris, I. B., Beckman, T. J., Reed, D. A., & Cook, D. A. (2014). Standards for reporting qualitative research: a synthesis of recommendations. Academic Medicine, 89(9), 1245-1251. https://doi.org/10.1097/acm.0000000000000388

Simera, I., Moher, D., Hirst, A., Hoey, J., Schulz, K. F., & Altman, D. G. (2010). Transparent and accurate reporting increases reliability, utility, and impact of your research: reporting guidelines and the EQUATOR Network. BMC Medicine, 8, 24. https://doi.org/10.1186/1741-7015-8-24

Simmons, O. E. (2010). Is that a real theory or did you just make it up? Teaching classic grounded theory. Grounded Theory Review, 9(2), 15-38. http://groundedtheoryreview.com/2010/06/25/is-that-a-real-theory-or-did-you-just-make-it-up-teaching-classic-grounded-theory/

Strauss, A., & Corbin, J. M. (1990). Basics of qualitative research: Grounded theory procedures and techniques. Sage Publications.

Tong, A., Sainsbury, P., & Craig, J. (2007). Consolidated criteria for reporting qualitative research (COREQ): A 32-item checklist for interviews and focus groups. International Journal for Quality in Health Care, 19(6), 349-357. https://doi.org/10.1093/intqhc/mzm042

Yalof, B. (2014). Marshaling resources: A classic grounded theory study of online learners. Grounded Theory Review, 13(1), 3. http://groundedtheoryreview.com/2014/06/22/marshaling-resources-a-classic-grounded-theory-study-of-online-learners/