In pursuit of anchoring vignettes that work: Evaluating generality versus specificity in vignette texts

Objective: Anchoring vignettes appear with growing frequency in surveys of health and aging, but little research investigates how to optimize their wording. This study experimentally tests whether mentioning specific health conditions and/or medical procedures enhances or undermines vignette validity. Methods: Three series of general health anchoring vignettes were fielded to 2,550 respondents in the Wisconsin Longitudinal Study: one mentioning no specific health conditions or procedures, one mentioning heart disease-related ones, and one mentioning diabetes-related ones. Variations on hierarchical ordered probit models were used to test whether vignette wording affected adherence to the key measurement assumptions of vignette equivalence (VE) and response consistency (RC). Results: While all vignette series showed substantial violations of VE, violations were larger (especially by sex and education) when using disease-specific texts. RC violations appeared relatively minor, but somewhat larger in disease-specific texts. Discussion: These findings suggest that more general, universal vignette texts may be preferable to ones describing highly specific conditions/procedures. The common advice to prioritize specificity and concreteness in survey texts may be misguided if sociodemographic groups differ in their familiarity or associations with the presented details. Anchoring vignettes are a potentially useful survey tool, but further efforts are needed to optimize their wording.