Article

Empirical Indicators:
Conceptual and Theoretical Origins

Indicadores empíricos:
orígenes teóricos y conceptuales

Indicadores empíricos:
origens teóricas e conceituais




10.5294/aqui.2021.21.4.4


Jacqueline Fawcett1

1 0000-0002-1091-8873 Department of Nursing, University of Massachusetts, United States of America. jacqueline.fawcett@umb.edu


Received: 08/05/2021
Sent to peers: 29/09/2021
Approved by peers: 21/10/2021
Accepted: 21/10/2021


Theme: Epistemology.

Contribution to the discipline: This paper advances nursology knowledge by clarifying the meaning and importance of empirical indicators as measures of human beings' health-related experiences.

Para citar este artículo / To reference this article / Para citar este artigo:Fawcett J. Empirical Indicators: Conceptual and Theoretical Origins. Aquichan. 2021;21(4):e2144. DOI: https://doi.org/10.5294/aqui.2021.21.4.4

Abstract

This paper discusses the connections between nursing conceptual model concepts, middle-range theory, and situation-specific theory concepts, as well as between the theory concepts and how they are measured, that is, empirical indicators. Three types of empirical indicators are described—instruments, assessment tools, and interventions—and an example of each type is given. The paper's central thesis is that a conceptual model concept is —or should be— the starting point for selecting or constructing an empirical indicator.

Keywords (Source MeSH): Models; theoretical; nursing theories; data collection; surveys and questionnaires.


Resumen

En el presente artículo, se analizan las conexiones entre los conceptos del modelo conceptual, la teoría de rango medio y los conceptos de la teoría de situaciones específicas de enfermería, así como entre los conceptos teóricos y cómo se miden estos, es decir, los indicadores empíricos. Se describen tres tipos de indicadores empíricos —instrumentos, herramientas de evaluación e intervenciones— y se da un ejemplo de cada tipo. La tesis central del artículo es que un concepto de modelo conceptual es —o debería ser— el punto de partida para seleccionar o construir un indicador empírico.

Palabras clave (Fuente MeSH): Modelos; teórico; teorías de enfermería; recopilación de datos; encuestas y cuestionarios.


Resumo

Neste artigo, são analisadas as conexões entre os conceitos do modelo conceitual, a teoria de médio alcance e os conceitos da teoria de situações específicas de enfermagem, bem como entre os conceitos teóricos e como estes são medidos, isto é, os indicadores empíricos. São descritos três tipos de indicadores empíricos (instrumentos, ferramentas de avaliação e intervenções) e é dado um exemplo de cada tipo. Argumenta-se que um conceito de modelo conceitual é — ou deveria ser — o ponto de partida para selecionar ou construir um indicador empírico.

Palavras-Chave (Fonte Decs): Modelos; teórico; teorias de enfermagem; coleta de dados; pesquisas e questionários.


All scholarly work and practice is based on a conceptual-theoretical-empirical (CTE) structure. The C component is the abstract and general conceptual model that guides the selection of phenomena for the scholarly work or practice, such as the Roy Adaptation Model (1). The T component is the relatively specific and concrete theory that is the outcome of the scholarly work or the why, when, where, and how of what is done in practice. Both conceptual models and theories are made up of concepts, which express ideas about things, and statements, which are definitions of and associations between these concepts. The E component encompasses the design of scholarly work or the practice delivery model; the people who are involved in the scholarly work or practice; how the theoretical concepts are measured, that is, the empirical indicators; the procedures used to administer the empirical indicators; and how the information obtained from the empirical indicators is analyzed (2). The purpose of this paper is to present a discussion of empirical indicators and their connection with the concepts of conceptual models and theories.


Empirical Indicators

An empirical indicator is a very concrete and specific real-world proxy for a concept of a middle-range theory or a situation-specific theory (2). The function of empirical indicators is to provide how middle-range theories and situation-specific theories are generated or tested by measuring the theory concepts (3).


Types of Empirical Indicators

Empirical indicators include instruments, assessment tools, and interventions. An instrument, such as an interview guide or a questionnaire, is typically used for scholarly work. An assessment tool is used in practice; the format can be an interview guide, a questionnaire, or a combination of the two. An an intervention is a procedure used to observe or measure a theory concept about what is being done in scholarly work or in practice, such as experimental treatment or a practice protocol.

Interview guides typically include one or more open-ended items and may include prompts or probes to encourage interviewees to provide additional details about their responses to the open-ended questions. Questionnaires include one or more fixed-choice items and various types of rating scales, such as yes/no or a numeric rating scale that usually has descriptors for at least some of the numbers (e.g., 0 = No pain to 10 = Worst pain imaginable). Rating scales with an even number of rating points are less prone to response set bias than rating scales with an odd number of rating points. Response set bias refers to the tendency to select the mid-point or neutral rating point when there is an odd number of rating points. Assessment tools can include either or both open-ended items and fixed choice items.

The information obtained from empirical indicators is typically called data. Data obtained from empirical indicators that are instruments or assessment tools are qualitative words transformed into themes or categories or quantitative numbers transformed into scores. Thus, responses to an interview guide made up of open-ended items can be analyzed to yield themes or categories, and responses to questionnaires made up of fixed-choice items can be subjected to mathematical calculations that yield a number or score. Examples are given in Table 1.

Table 1. Examples of Empirical Indicators that are Instruments or Assessment Tools

Types of Empirical Indicator

Examples of Items and Scoring

Interview Guide with Open-Ended Items

Item: Please tell me how you are feeling physically today.
Scoring: Themes or categories are extracted from responses to the item, such as feeling fine, feeling just ok, feeling tired, feeling sick.

Questionnaire with Fixed-Choice Items

Item: On a scale of 0 to 10, where 0 means no pain and 10 means worst pain imaginable, how much pain are you feeling right now? Scoring. Each response is scored as anywhere from 0 to 10.

Source: Own elaboration


Empirical indicators that are interventions tell the person using them exactly what to do and when, where, and how to implement the intervention. Thus, they are protocols or scripts that direct actions in a precise manner. Examples are given in Table 2.

Table 2. Examples of Empirical Indicators that are Interventions

Types of Empirical Indicator

Examples

An experimental treatment

A Roy Adaptation Model (1) guided experimental intervention—cognitive stimulation therapy (CST)—was developed by Lok and colleagues (4) for people with a medical diagnosis of Alzheimer's Disease. This empirical indicator was linked to the Roy Adaptation Model concept of cognator coping process. The outcomes of the intervention were linked to the Roy Adaptation Model adaptive modes. Lok et al. (4, pp 585-6) explained, "CST consisted of 14 sessions with different themes... The sessions lasted 45 minutes. The introduction, activity, and final sessions lasted 10, 25, and 10 minutes, respectively. The therapy was conducted for 7 weeks (two sessions per week)."
The themes for the 14 sessions were physical games, sounds, childhood, food, current affairs faces/scenes, word associations, being creative, categorizing objects, orientation, using money, number games, word games, and team quiz, respectively. Lok et al. (4, p585) provided a description of each theme. For example, physical games are described as "The nurse encourages the patients to introduce themselves to the group, mention their favorite team, meal, color, and so forth. Supporting the self-perception and playing basketball are also included."

A practice protocol

Seah and Tham (5) described a Roy Adaptation Model (1) practice protocol used for care of a young woman (fictional name of Julie) with the medical diagnosis of bulimia nervosa. The practice protocol included three parts, each for management of stimuli for a different nursing diagnosis. For example, one part of the practice protocol is the intervention that was management of stimuli for the nursing diagnosis of imbalanced nutrition. The intervention was targeted to restoring a balance of electrolytes and nutrition. Seah and Tham (5, p139) explained,
As Julie was placed under the hospital's eating disorder (ED) protocol, the nurse weighed her daily. This ensured that her daily weight was charted for review by the multi-disciplinary team (MDT)... Julie had to eat 3 meals daily with 2 snacks that included a protein drink, as a form of nutrition rehabilitation... This was crucial as the main goal was to achieve optimal nutrition and a BMI of at least 18.5 during the course of treatment. She also participated in a lunch group support session, which was facilitated by a nurse who ate with her, while monitoring her eating behavior and duration. This allowed the nurse to note any abnormal eating patterns and feedback to the MDT. When a nurse eats with the client, the client will also learn proper eating habits from the nurse.

Source: Own elaboration


Practical Considerations

The length, reading level, and ease of completion should be considered when selecting an empirical indicator. In addition, if the empirical indicator is to be used with people of diverse cultures, translation of the empirical indicator must be considered.


Length of Empirical Indicators

Although no definitive rules are available for the length of empirical indicators, a "rule of thumb" is the shorter, the better, in that fewer items are more likely to be administered or answered by people than more items. Noteworthy is that one-item empirical indicators (called single-item indicators) may yield as much information as many-item empirical indicators (6).


Reading Level

The reading level of an empirical indicator should be consistent with the literacy level of the people who are administering and responding to the empirical indicator. Some methods for determining the reading level of empirical indicators are Flesch's (7), Fry's (8), and the Simple Measure of Gobbledygook (9). Reading level calculators also can be found on the internet; for example, see http://www.readabilityformulas.com/free-readability-formula-tests.php


Ease of Completion

An empirical indicator should be easy to complete and analyze the data, which means that the instrument and assessment tool items (open-ended items or fixed-choice items) should be written clearly and concisely, and the response options should be easy to understand. Ease of completion can be checked using cognitive interviews prior to administering the instrument or assessment tool to the people who will be asked to respond to it. A cognitive interview is conducted by the person planning to use the empirical indicator with a few (typically 5-10) people who have the same characteristics as the people who will be asked to respond to the empirical indicator. The interviewer may use "probing questions [to] help to identify whether changes in format and presentation might affect the way [people] respond to the [items]" (10, p41).


Cultural Equivalence of Empirical Indicators

Fawcett and Garity (2) explained that back-translation is a frequently used procedure to translate empirical indicators from one language to another. This procedure involves the translation of the empirical indicator from the language in which it was originally written to another language and then translation from the other language back to the original language. Although back-translation may seem to be an adequate method to ensure cultural relevance of an empirical indicator, determining the cultural relevance or equivalence of an empirical indicator requires a great deal of effort to maintain the original meaning of the concept being measured and the words used for each item of the empirical indicator.

Drawing from Flaherty et al. (11), Fawcett and Garity (2) identified five different types of cross-cultural equivalence for empirical indicators.

1.   Content equivalence refers to the relevance of each item to the culture of interest.

2.   Semantic equivalence refers to the extent to which the connotative meaning of each item is the same in the original culture and the culture for which the instrument is being translated.

3.   Technical equivalence refers to the extent to which the way the data were collected—such as an interview guide or a questionnaire—is similar in each culture.

4.   Criterion equivalence refers to the extent to which the interpretation of the data is similar across cultures.

5.   Conceptual equivalence refers to the extent to which the same theory concept is measured in each culture.


Explicit and Implicit Conceptual-Theoretical-Empirical Connections

Ideally, each empirical indicator is explicitly connected to a theory concept derived from a conceptual model concept. For example, as shown in Figure 1, several inventories of functional status have been designed to measure the theory concept of functional status. The theory concept of functional status, which is defined as the performance of usual and special activities, was directly derived from the Roy Adaptation Model (1, 12) concept of role function adaptive mode, which is defined as the performance of usual and special role activities of living based on the developmental stage of life (1, 12). An example of these inventories is the Inventory of Functional Status After Childbirth (13), which measures the functional status of women who have experienced childbirth.

Figure 1. Linkage of an empirical indicator with the theory concept derived from a conceptual model concept


However, not all empirical indicators are connected to a theory concept, and not all theory concepts are connected to a conceptual model concept. Thus, the user of the empirical indicator must guess what the empirical indicator is measuring, that is, the definition of the theory concept that is supposed to be measured and the connection between the theory concept and a conceptual model concept that initially guided identification of the theory concept.


Measurement Validity

The connection of an empirical indicator with a theory concept and the connection of the theory concept with a conceptual model concept are important if measurement validity is to be assured. Measurement validity refers to the appropriateness of an empirical indicator as a measure of the theory concept, as that concept is defined (2, 14). The question to be asked is:

•     Does the empirical indicator measure the theory concept as that concept is defined?

Fawcett (14) pointed out that a frequently neglected consideration in determining measurement validity is identifying whether the definition of the theory concept measured by the empirical indicator is congruent with the definition of the conceptual model concept that guides the scholarly work or practice. For example, if a theory concept is defined as people's actual performance of role activities, and if the theory concept has been directly derived from a conceptual model concept about the performance of role activities, then an empirical indicator that measures people's ability to perform role activities is not congruent with either the conceptual model concept or the theory concept.

Various methodological techniques can estimate measurement validity. Frequently used methods are content validity and construct validity (2).

In the real world of scholarly work and practice, there is a tendency to find an empirical indicator to measure a theory concept of interest and then determine whether the empirical indicator "fits" with a concept of the conceptual model that guides the scholarly work or practice. Although creating new empirical indicators is time-con­suming and challenging, measurement validity is more likely to be supported if the starting point is a conceptual model (14). The steps of creating a new empirical indicator rather than searching for an empirical indicator that might be an acceptable fit with the theory concept and the conceptual model concept are listed in Table 3.

Table 3. Phases of Constructing an Empirical Indicator

Phases

Description

Phase 1

Identify a conceptual model to guide the scholarly work or practice (2, 15)

Phase 2

Derive a theory concept from a concept of that conceptual model (2, 15)

Phase 3

Generate items from the literature that are appropriate measures of the theory concept (15)

Phase 4

Determine measurement validity (13)

Phase 5

Calculate a content validity index based on ratings by a panel of 5 to 10 experts (15)

Phase 6

Conduct cognitive interviews with 5 to 10 people (10, 15)

Phase 7

Administer the empirical indicator to a large number of people (at least 10—30 persons for each item) (2)

Phase 8

Use the data obtained from open-ended interview items to determine estimates of trustworthiness of the empirical indicator, including dependability (e.g., inquiry audit) and credibility (e.g., member checks) (2)
Use the data obtained from fixed choice questionnaire items to calculate the psychometric properties of the empirical indicator, including internal consistency reliability and construct validity (2)

Source: Own elaboration


Conclusion

In conclusion, a conceptual model concept should be the start of a search for an existing empirical indicator or the development of a new empirical indicator. A theory concept should be directly derived from the conceptual model concept, such that there is a clear and logical connection between the conceptual model and theory concepts. The empirical indicator then should be a precise, logical, and valid measure of the theory concept.

Conflict of interests: None declared.


References

1. Roy C. The Roy adaptation model. 3rd ed., Upper Saddle River (NJ): Pearson; 2009.

2. Fawcett J, Garity J. Evaluating research for evidence-based practice. Philadelphia (PA): F. A. Davis; 2009.

3. Fawcett J, DeSanto-Madeya S. Contemporary nursing knowledge. Analysis and evaluation of nursing models and theories. 3rd ed. Philadelphia (PA): F. A. Davis; 2013.

4. Lok N, Buldukoglu K, Barcin E. Effects of the cognitive stimulation therapy based on Roy's adaptation model on Alzheimer's patients' cognitive functions, coping-adaptation skills, and quality of life: A randomized controlled trial. Perspect Psychiatr Care. 2020(3);56:581-592. DOI: https://doi.org/10.1111/ppc.12472

5. Seah XY, Tham XC. Management of bulimia nervosa: A case study with the Roy Adaptation Model. Nurs Sci Quart. 2015;28(2):136-141. DOI: https://doi.org/10.1177/0894318415571599

6. Youngblut J, Casper G. Single-item indicators in nursing research. Res Nurs Health. 1993;16(6)459-465. DOI: https://doi.org/10.1002/nur.4770160610

7. Flesch R. A new readability yardstick. J Appl Psych. 1948;32(3):221-233. DOI: https://doi.org/10.1037/h0057532

8. Fry EB. A readability formula that saves time. J Reading. 1968;11(7):513-516, 578.

9. McLaughlin GH. SMOG grading-A new readability formula. J Reading. 1969;12(8):639-646. Available from: https://www.jstor.org/stable/40011226

10. Muehlhausen W, Byrom B, Skerritt B, McCarth, M, McDowell B, Sohn J. Standards for instrument migration when implementing paper patient-reported outcome instruments electronically: Recommendations from a qualitative synthesis of cognitive interview and usability studies. Value Health. 2018;21(1):41-48. DOI: https://doi.org/10.1016/j .jval .2017.07.002

11. Flaherty JA, Gaviria FM, Pathak D, Mitchell T, Wintrob R, Richman JA, Birz S. Developing instruments for cross-cultural psychiatric research. J Nerv Ment Dis. 1988;176(5):257-263. DOI: https://doi.org/10.1097/00005053-198805000-00001

12. Roy C. Introduction to nursing: An adaptation model. 2nd ed.. Englewood Cliffs (NJ): Prentice-Hall; 1984.

13. Fawcett J, Tulman L, Myers ST. Development of the Inventory of Functional Status After Childbirth. J Nurse Midwifery. 1988;33(6):252-260. DOI: https://doi.org/10.1016/0091-2182(88)90080-8

14. Fawcett J. Thoughts about conceptual models and measurement validity. Nurs Sci Q. 2013;26(2):189-191. DOI: https://doi.org/10.1177/0894318413477143

15. Waddell A. Measuring nurses' health policy participation: WSPPIR instrument development and psychometric evaluation. J Adv Nurs. 2021;7700:461-472. DOI: https://doi.org/10.1111/jan.14582


Home