Kuwaiti Journal of Health Care Management 2013;1(1):5-7
Research Article

Measuring Quality of Care Using a Vignette-Based Survey in Turkish Hospitals

Ayşegül Yıldırım Kaptanoğlu*, a and Ilknur Aktaşb

[±] Authors Affiliations

Abstract

Purpose: Clinical vignettes are cost-effective tool for clinical managers. They offer opportunities to discover nuances in the clinical decision-making process. This clinical vignette-based study demonstrates actual variation in practice and advocates standardization in health care delivery. Methods: Two public hospitals were randomly selected from general public hospitals located in Istanbul. The vignettes used in this study include particular open-ended questions to elucidate the perspectives of surgeons and internal medicine physicians. Results: Analysis showed that vignette-based surveys produce effective measures of quality of care. Statistically significant differences were found between care provided by internists and surgeons. Findings were consistent across all hypothetical vignette scenarios and were independent of case complexity. Conclusions: Use of vignettes provides cost-effective decision-making opportunities for clinical managers. Furthermore, clinical vignettes are simple and economical tools in measuring physicians' productivity and quality.

Keywords: health care quality, clinical vignettes, Turkish hospitals

Introduction

Clinical vignettes were developed for measuring and evaluating the quality of clinical care. Clinical vignettes are responsive to variation in clinical care quality (Peabody et al., 2004a). They are usually followed by open-ended questions that aim to elicit interactive responses that evaluate the physician's knowledge. Vignettes have been validated against the gold standard and have proven that they reflect actual clinical practice and not just physician's knowledge. They can be administered to a clinician via paper, computer, or the Internet (Vanderbush et al., 2005).

Vignettes serve several other advantages. They are useful for pre- and post- evaluations of policy interventions designed to improve quality of clinical care much like health care quality indicators. Clinical vignettes are inexpensive to administer and easy to score. These advantages make them useful, particularly in developing countries. Clinical vignettes are one way for assessing quality of care among difference providers and between different organizations and they have been widely used by health service managers (Peabody et al., 2006).

One basic method to measure improvement in quality of hospital-based care is through assessments of improvement clinical practice quality. Clinical vignettes are accurate, valid, feasible and an inexpensive tool to measure quality of health care (Pham et al., 2009). They, therefore, have been widely used to compare quality of clinical care across countries, health care systems, specialties, or individual clinicians.

Methods

Using clinical vignette-based surveys, physicians were given open-ended questions that have been validated as a measure of the quality of clinical practice. Vignettes were administered in two randomly selected public hospitals: one secondary care hospital (Hospital A) and one tertiary care hospital (Hospital B). From each hospital, 10% of the internists and surgeons were randomly selected and invited to participate in the study.

Consequently, 24 physicians agreed to participate and took the vignettes for three medical conditions: irritable bowel syndrome, coronary artery disease and low back pain. All vignettes selected relatively common medical conditions frequently seen by both internists and surgeons. In order to complete each vignette, the physicians were asked to take a case history, perform physical examination, request necessary diagnostic tests and devise a treatment plan.

A total of 72 clinical vignettes were prepared by the authors and were administered on paper to the 24 participating physicians. Because there was one case scenario for each vignette, physicians were not told about the disease involved in the vignettes ahead of time.

Box: Example of a clinical vignette used in the study
A 23-year-old sexually active woman presented to the office complaining of abdominal pain, nausea, alternating diarrhea and constipation, bloating, and constant tiredness, none of which were helped by antispasmodics.

Each vignette was randomly determined for each physician to avoid repeating vignettes in the study. Participants' responses to the vignette-based survey were coded. Based on the responses, an average score was calculated for each physician and each of the two hospitals.

Each scoring category was assigned to one of the five domains of care as follows:

  1. History taking.
  2. Physical examination.
  3. Radiological and/or laboratory tests ordered.
  4. Diagnosis.
  5. Treatment plan.

Vignette scores were calibrated (0-70) to allow the same number of points and to contribute equally to the total score.

Statistical analysis

A chi-squared (Χ2) test was used to compare the items selected by physician. A p-value of < 0.05 was considered statistically significant. SPSS® 13.0 was used for statistical analysis.

Results

Twenty-four physicians participated in the study, 12 from each hospital. The mean age of physicians in Hospital A was 49 ± 5.6 and in Hospital B was 43 ± 4.6. The socio-demographic data of the sample group are shown in Table 1. Furthermore, analysis has demonstrated that there are statistically significant differences in vignette scores between physicians of different age, gender, work experience, and specialty (Table 2).

Analysis has shown that 74% of internal medicine physicians' diagnoses were correct (95% CI: 64.5%-81.0%), compared to 72% of surgeons' diagnoses (95% CI: 68.1%-73.2%). According to average scores of physicians within each of the two specialties, differences in score are statistically significant (Χ2 = 19.74; p < 0.05).

The vignette scores for the whole sample ranged from 40% to 70%. The average quality of care for the 72 cases completed in the study was 61.0%.

The mean score for the three vignette cases in the Hospital B was 56% (±9%) compared with 48% (±10%) in Hospital A.

Variation between clinical skill sets was greater in Hospital B (the tertiary care hospital) than in Hospital A (the secondary care hospital). For example, by domain, tertiary care physicians obtained their highest average score for the physical examination domain (55%). Meanwhile, secondary care physicians scored lowest for physical examination skills (41%). See Table 3.

Discussion

Quality of care in secondary- and tertiary care hospitals was quite different, with differences in vignette scores ranging from 10% to 20%. This variation is attributed to physicians' characteristics. For example, according to our study, younger physicians, female physicians, and tertiary care physicians provided—in general—better quality care for their patients.

In the tertiary care Hospital B, physicians responded correctly to the clinical vignettes more likely, especially in the domains of history taking, physical examination, and making a diagnosis. According to vignette scoring in our study, they provide higher quality of clinical care.

Clinical vignettes have an important niche in the overall measurement of quality of health care but their use should be carefully defined with further research. Measuring the quality of health care services with clinical vignettes is a useful instrument for managers and policymakers. Using this tool, health care managers and policymakers can compare the quality of care in different settings to—for example—evaluate management and policy interventions (Peabody et al., 2004b).

In addition, clinical vignettes make an important contribution to health knowledge because they allow for greater understanding of the distribution of physician's competence and practice quality (Dresselhaus et al., 2004).

Our reported validation of vignettes is one step toward their wider application. However, more work is needed to link vignette scores to policy interventions or improvements in patient-related outcomes. While clinical vignettes are able to capture actual practice, control for case mix, measure necessary and unnecessary care, and are easy to administer at a low cost, they have some limitations that must be considered and are currently best used in conjunction with other measures (Kumaranayake et al., 2000; Laing et al., 2001; Quimbo et al., 2011).

References

  1. Dresselhaus, T, Peabody, J, Luck, J, & Bertenthal, D (2004). An Evaluation of Vignettes for Predicting Variation in the Quality of Preventive Care. Journal of General Internal Medicine. 19(10):1013-1018.
  2. Kumaranayake, L, Mujinja, P, Hongoro, C, & Mpembeni R (2000). How Do Countries Regulate the Health Sector? Evidence from Tanzania and Zimbabwe. Health Policy and Planning. 15(4):357-367.
  3. Laing, R, Hogerzeil H, & Ross-Degnan D. (2001). Ten Recommendations to Improve Use of Medicines in Developing Countries. Health Policy and Planning. 16(1):13-20.
  4. Peabody J, Luck J, Glassman P, Dresselhaus T, & Lee M. (2000). Comparison of Vignettes, Standardized Patients, and Chart Abstraction: A Prospective Validation Study of 3 Methods for Measuring Quality. Journal of the American Medical Association. 283(13):1715-1722.
  5. Peabody J, Luck J, Glassman P, Jain S, Hansen J, Spell M, & Lee M. (2004a). Measuring the Quality of Physician Practice by Using Clinical Vignettes: A Prospective Validation Study. Annals of Internal Medicine. 141(10):771-780.
  6. Peabody J, Taguiwalo M, Robalino D, & Frenk J. (2006). Improving the Quality of Care in Developing Countries. pp.1293-1308. In Jamison D, Breman J, Measham A, Alleyne, G, Claeson, M, Evans, D, Jha, P, Mills, A, & Musgrove, P, Editors. Disease Control Priorities in Developing Countries. 2nd ed. Washington, DC: World Bank.
  7. Peabody J, Tozija F, Muñoz J, Nordyke R, & Luck J. (2004b). Using Vignettes to Compare the Quality of Care Variation in Economically Divergent Countries. Health Services Research. 39:1951-1970.
  8. Pham T, Roy C, Mariette X, Lioté F, Durieux P, & Ravaud P. (2009). Effect of Response Format for Clinical Vignettes on Reporting Quality of Physician Practice. BMC Health Services Research. 9:128.
  9. Quimbo S, Peabody J, Shimkhada R, & Solon O. (2011). Evidence of a Causal Link Between Health Outcomes, Insurance Coverage and a Policy to Expand Access: Experimental Data from Children in the Philippines. Health Economics. 20(5):620-630.
  10. Vanderbush R, Kirtley J, & West D. (2005). Evaluating Internet-Based Multimedia Vignettes for Teaching Ophthalmic and Otic Drug Administration Techniques. American Journal of Pharmaceutical Education. 69(4) Article 64.

Tables

Table 1
Table 1: Socio-demographic characteristics of participating physicians
Table 1
Table 1
Table 2
Table 2: Vignette score by socio-demographic characteristics
Table 2
Table 1
Table 3
Table 3: Vignette scores by domains of care
Table 3
Table 1


Abstract

PDF

Headings

Share



..