Research & Resources
CALNOC Research Articles
Ambulatory Care Nurse-Sensitive Indicators Series: Reaching for the Tipping Point in Measuring Nurse-Sensitive Quality in the Ambulatory Surgical and Procedure Environments
Nursing Economics. MayJune;34(3):147-151
Brown, D. S., Aronow, H., (2016).
The value of the ambulatory care nurse remains undocumented from a quality and patient safety measurement perspective and the practice is at risk of being highly variable and of unknown quality.
The American Academy of Ambulatory Care Nursing and the Collaborative Alliance for Nursing Outcomes propose nurse leaders create a tipping point to measure
the value of nursing across the continuum of nursing care, moving from inpatient to ambulatory care.
As care continues to shift into the ambulatory care environment, the quality imperative must also shift to assure highly reliable, safe, and effective health care
Ambulatory care nursing-sensitive indicators
Nursing Management. June;47(6):16-18
Matlock, A. M., Start, R., Aronow, H., Brown, D. S. (2016).
In the spring of 2013, the American Academy of Ambulatory Care Nursing's (AAACN) board of directors commissioned a task force to identify and define nurse-sensitive indicators specific to the ambulatory care setting. Members included indicator development experts from the Collaborative Alliance for Nursing Outcomes (calnoc) and the National Database of Nursing Quality Indicators(R) (NDNQI(R)), as well as a past president of AAACN who was part of the original work done by the American Nurses Association (ANA) in the late 1990s to identify indicators sensitive to nonacute settings. Members of the task force represented all geographic areas in the United States and a broad range of practice settings.
Improving hospital patient falls: leveraging staffing characteristics and processes of care.
The Journal of Nursing Administration. May;45(5):254-62
Aydin, C., Donaldson, N., Aronow, H., Fridman, M., Brown, D. S. (2015).
OBJECTIVE: Predictive models for falls, injury falls, and restraint prevalence were explored within nursing unit structures and processes of care.
BACKGROUND: The patient care team is responsible for patient safety, and improving practice models may prevent injuries and improve patient safety.
METHODS: Using unit-level self-reported data from 215 hospitals, falls, injury falls, and restraint prevalence were modeled with significant covariates as predictors.
RESULTS: Fewer falls/injury falls were predicted by populations with fewer frail and at-risk patients, more unlicensed care hours, and prevention protocol implementation, but not staffing per se, restraint use, or RN expertise. Lower restraint use was predicted by fewer frail patients, shorter length of stay, more RN hours, more certified RNs, and implementation of fall prevention protocols.
CONCLUSION: In the presence of risk, patient injuries and safety were improved by optimizing staffing skill mix and use of prevention protocols.
Modeling Hospital-Acquired Pressure Ulcer Prevalence on Medical-Surgical Units: Nurse Workload, Expertise, and
Clinical Processes of Care
Health Services Research. April;50(2):351-373
Aydin, C., Donaldson, N., Stotts, N., Fridman, M., Brown, D. S. (2015).
OBJECTIVE: This study modeled the predictive power of unit/patient characteristics,
nurse workload, nurse expertise, and hospital-acquired pressure ulcer (HAPU) preventive
clinical processes of care on unit-level prevalence of HAPUs.
DATA SOURCES: Seven hundred and eighty-nine medical-surgical units (215 hospitals)
STUDY DESIGN: Using unit-level data, HAPUs were modeled with Poisson regression
with zero-inflation (due to low prevalence of HAPUs) with significant covariates as
DATA COLLECTION/EXTRACTION METHODS: Hospitals submitted data on NQF endorsed
ongoing performance measures to CALNOC registry.
PRINCIPAL FINDINGS: Fewer HAPUs were predicted by a combination of unit/patient
characteristics (shorter length of stay, fewer patients at-risk, fewer male patients), RN
workload (more hours of care, greater patient [bed] turnover), RN expertise (more
years of experience, fewer contract staff hours), and processes of care (more risk assessment
CONCLUSIONS: Unit/patient characteristics were potent HAPU predictors yet generally
are not modifiable. RN workload, nurse expertise, and processes of care (risk assessment/
interventions) are significant predictors that can be addressed to reduce HAPU.
Support strategies may be needed for units where experienced full-time nurses are not
available for HAPU prevention. Further research is warranted to test these finding in
the context of higher HAPU prevalence.
The Economics of Preventing Hospital Falls. Demonstrating ROI Through a Simple Model
The Journal of Nursing Administration. January;45(1):50-57
Spetz J., Brown D.S., Aydin C. (2015).
OBJECTIVE: The objective of this study was to assess the cost savings associated with implementing nursing approaches to prevent in-hospital falls.
BACKGROUND: Hospital rating programs often report fall rates, and performance-based payment systems force hospitals to bear the costs of treating patients after falls. Some interventions have been demonstrated as effective for falls prevention.
METHODS: Costs of falls-prevention programs, financial savings associated with in-hospital falls reduction, and achievable fall rate improvement are measured using published literature. Net costs are calculated for implementing a falls-prevention program as compared with not making improvements in patient fall rates.
RESULTS: Falls-prevention programs can reduce the cost of treatment, but in many scenarios, the costs of falls-prevention programs were greater than potential cost savings.
CONCLUSIONS: Falls-prevention programs need to be carefully targeted to patients at greatest risk in order to achieve cost savings.
Improving Medication Administration Safety: Using Naive Observation to Assess Practice and Guide Improvements in Process and Outcomes
Journal for Healthcare Quality. December;36(6):58-65
Donaldson N., Aydin C., Fridman M., Foley, M. (2014).
PURPOSE: To present findings from the Collaborative Alliance for Nursing Outcomes' (CALNOC) hospital medication administration (MA) accuracy assessment in a sample of acute care hospitals. Aims were as follows: (1) to describe the CALNOC MA accuracy assessment, (2) to examine nurse adherence to six safe practices during MA, (3) to examine the prevalence of MA errors in adult acute care, and (4) to explore associations between safe practices and MA accuracy.
METHODS: Using a cross-sectional design, point in time, and convenience sample, direct observation data were collected by 43 hospitals participating in CALNOC's benchmarking registry. Data included 33,425 doses from 333 observation studies on 157 adult acute care units. Results reveal that the most common MA safe practice deviations were distraction/interruption (22.89%), not explaining medication to patients (13.90%), and not checking two forms of ID (12.47%). The most common MA errors were drug not available (0.76%) and wrong dose (0.45%). the overall percentage of safe practice deviations per encounter was 11.40%, whereas the overall percentage of MA errors was 0.32%.
CONCLUSIONS and IMPLICATIONS: Findings predict that for 10,000 MA encounters, 27,630 safe practice deviations and 770 MA errors will occur. A 36% reduction in practice deviation per encounter prevents 4.4% MA errors. Ultimately, reliably performing safe practices improves MA accuracy.
Predictors of Unit-Level Medication Administration Accuracy: Microsystem Impacts on Medication Safety
Journal of Nursing Administration. June;44(6):353-361
Donaldson N., Aydin C., Fridman, M. (2014).
OBJECTIVE: This study tested multivariate models exploring unit-level predictors of medication administration (MA) accuracy.
BACKGROUND: During MA, nurses are both the last line of defense from medication-related errors and a potential perpetrator of error. Direct observation reveals safe practices and the accuracy of medication delivery.
METHODS: Using a direct-observation, cross-sectional design, data submitted by 124 adult patient care units for 15600 medication doses, from January 2009 to April 2010, were studied.
RESULTS: Distractions and interruptions were the most common safe practice deviation. Characteristics of patient care units and RN hours of care affected nurses’ safe practices and MA accuracy. Safe practices predict and mediate MA accuracy. A 5% decrease in safe practice deviations would reduce MA errors by 46% without any change in RN hours of care.
CONCLUSION: Nurses’ adherence to MA safe practices, combined with unit characteristics and staffing factors, has the potential to dramatically improve MA accuracy.
Safety Culture Relationships with Hospital Nursing Sensitive Metrics
Journal for Healthcare Quality. Jul/Aug;35(4):61-74
Brown D.S., Wolosin R. (2013).
Public demand for safer care has catapulted the healthcare industry’s efforts to understand relationships between patient safety and hospital performance. This study explored linkages between staff perceptions of safety culture (SC) and ongoing measures of hospital nursing unit-based structures, care processes, and adverse patient outcomes. Relationships between nursing-sensitive measures of hospital performance and SC were explored at the unit-level from 9 California hospitals and 37 nursing units. SC perceptions were measured 6 months prior to collection of nursing metrics and relationships between the two sets of data were explored using correlational and regression analyses. Significant relationships were found with reported falls and process measures for fall prevention. Multiple associations were identified with SC and the structure of care delivery: skill mix, staff turnover, and workload intensity demonstrated significant relationships with SC, explaining 22–45% of the variance. SC was an important factor to understand in the quest to advance safe patient care. These findings have affordability and care quality implications for hospital leadership. When senior leaders prioritized a safety culture, patient outcomes may have improved with less staff turnover and more productivity. A business case could be made for investing in patient safety systems to provide reliably safe care.
The Value of Reducing Hospital-Acquired Pressure Ulcer Prevalence: An Illustrative Analysis
Journal of Nursing Administration. Apr;43(4):235-241
Spetz J., Brown D.S., Aydin C., Donaldson N. (2013).
OBJECTIVE: The aim of this study was to assess the cost savings associated with implementing nursing approaches to prevent hospital-acquired pressure ulcers (HAPU). BACKGROUND: Hospitals face substantial costs associated with the treatment of HAPUs. Interventions have been demonstrated as effective for HAPU prevention and management, but it is widely perceived that preventative measures are expensive and, thus, may not be a good use of resources. METHODS: A return-on-investment (ROI) framework from the Agency for Healthcare Research and Quality (AHRQ) Quality Indicators Toolkit was used for this study. The researchers identified achievable improvements in HAPU rates from data from the Collaborative Alliance for Nursing Outcomes and measured costs and savings associated with HAPU reduction from published literature. RESULTS: The analysis produced a baseline ROI ratio of 1.61 and net savings of $127.51 per patient. CONCLUSIONS: Hospital-acquired pressure ulcer surveillance and prevention can be cost saving for hospitals and should be considered by nurse executives as a strategy to support quality outcomes.
Eliminating Hospital-Acquired Pressure Ulcers: Within Our Reach
Advances in Skin & Would Care. 26(1):13-18
Stotts, N.A., Brown, D.S., Donaldson, N., Aydin, C., Fridman, M. (2013).
Hospital-acquired pressure ulcers (HAPUs) are a serious nosocomial problem that has been viewed as a ubiquitous consequence of immobility. This article provides data from the Collaborative Alliance for Nursing Outcomes (CALNOC) that shows a significant reduction in HAPUs in adults from 78 acute care hospitals over 8 years (2003-2010).
Using Minimum Nurse Staffing Regulations to Measure the Relationship Between Nursing and Hospital Quality of Care
Medical Care Research and Review, 70(1)
Spetz, J., Mark, B.A., Herrera, C.N., Harless, D.W. (2013).
This study tests whether changes in licensed nurse staffing led to changes in patient safety, using the natural experiment of 2004 California implementation of minimum staffing ratios. We calculated counts of six patient safety outcomes from California Patient Discharge Data from 2000 through 2006, using the Agency for Healthcare Research and Quality Patient Safety Indicators (PSI) software. For patients experiencing nonmortality-related PSIs, we measured mean lengths of stay. We estimated difference-in-difference equations of changes in PSIs using Poisson models and calculated the marginal impact of nurse staffing on outcomes from fixed-effect Poisson regressions. Licensed nurse staffing increased in the postregulation period, except for hospitals in the highest quartile of preregulation staffing. Growth in registered nurse staffing was associated with improvement for only one PSI and reduced length of stay for one PSI. Higher registered nurse staffing per patient day had a limited impact on adverse events in California hospitals.
Hospital Acquired Pressure Ulcer Prevalence in Adults—Trends, Accomplishments, Challenges: Collaborative Alliance for Nursing Outcomes (CALNOC) Benchmarking 1998-2010
In Pressure Ulcers in America: Prevalence, Incidence, and Implications for the Future, 2nd Edition. National Pressure Ulcer Advisory Panel (NPUAP), 189-196.
Stotts, N.A., Brown, D.S., Aydin, C., Donaldson, N. (2012).
Noting that the CALNOC nursing sensitive benchmarking dataset provides a unique longitudinal view of HAPU in a regional subset of American hospitals, this chapter reviews and discusses CALNOC data published between January 2000 and October 2011. Specifically it explores trends in prevalence of PUs, in particular HAPUs, and nurse staffing, as well as the impacts of nursing processes of care measures including PU risk and skin assessment and the presence of a prevention protocol.
Nursing-Sensitive Benchmarks for Hospitals to Gauge High-Reliability Performance
Journal for Healthcare Quality 32(6), 9-17.
Brown, D.S., Donaldson, N., Burnes Bolton, L., Aydin, C.E. (2010).
The purpose of this article is to provide hospitals not currently participating in comparative benchmarking databases with nursing-sensitive data from the Collaborative Alliance for Nursing Outcomes for use in performance improvement processes.
Benchmarking for Small Hospitals: Size Didn’t Matter!
Journal for Healthcare Quality, 32(4), 50-60.
Brown, D.S., Aydin, C.E., Donaldson, N., Fridman, M., Sandhu, M. (2010).
This article reports the examination of hospital size as a proxy characteristic to define "like” hospitals for the purpose of benchmarking outcomes. Findings suggest that optimal classifications into small and large hospital size based on the outcome measures of falls, falls with injury, and hospital acquired pressure ulcers stage 2 or worse (HAPU 2+) were not consistent with historical administrative categories based on average daily census and not consistent by outcome. These data did not support the use of size based categories to define like hospitals for benchmark comparisons.
Quartile Dashboards: Translating large datasets into performance improvement priorities.
Journal for Healthcare Quality, 30(6), 18-30.
Brown, D.S., Aydin, C.E., Donaldson, N. (2008).
Quality professionals are the first to understand challenges of transforming data into meaningful information for frontline staff, operational managers, and governing bodies. To understand an individual facility, service, or patient care unit’s comparative performance from within large datasets, prioritization and focused data presentation are needed to meet this challenge. This article presents a methodology for translating data from large datasets into dashboards for setting performance improvement priorities, in a simple way that takes advantage of tools readily available and easily used by support staff.
How many nurses per patient? Measurements of Nurse Staffing in Health Services Research.
Health Services Research, 43(5), Part I, 1674-1692.
Spetz, J., Donaldson, N., Aydin, C., Brown, D.S. (2008).
This paper compares alternative measures of nurse staffing and assess the relative strengths and limitations of each measure. Primary and secondary data from 2000 and 2002 on hospital nurse staffing from the American Hospital Association, California Office of Statewide Health Planning and Development, California Nursing Outcomes Coalition, and the California Workforce Initiative Survey. The greatest differences in staffing measurement arise when unit-level data are compared with hospital-level aggregated data reported in large administrative databases. Unit-level data collection may be more precise. Differences between databases may account for differences in research findings.
Beyond nursing quality measurement: The nation’s first regional nursing virtual dashboard.
In Henriksen, K., Battles, J.B., Keyes, M.A., Grady, M.L., Eds. Advances in Patient Safety: New Directions and Alternative Approaches, Vol. I Assessment. AHRQ Publication No. 08-0034-1. Rockville, MD: Agency for Healthcare Research and Quality; August 2008, 217-234.
Aydin, C.E., Burnes Bolton, L., Donaldson, N., Brown, D., Mukerji, A. (2008).
This paper describes the data reporting infrastructure of the California Nursing Outcomes Coalition (CalNOC), and examines the system’s capacity to provide participating hospitals with seamless, interactive access to sophisticated reports in a secure environment that authorizes users and controls access privileges. By leveraging the data repository to create both standardized and customized reporting capacity, CalNOC significantly improves the responsiveness and strategic value of the data to members who create query-driven customized reports generated directly from the dataset. The resulting capacity for a virtual dashboard, the first in nursing quality measurement arena, is unique in the field and a model for further study and emulation.
Mandated nurse staffing ratios in California: A comparison of staffing and nurse-sensitive outcomes pre- and post-regulation.
Policy, Politics, & Nursing Practice, 8(4), 238-250.
Burnes Bolton, L., Aydin, C., Donaldson, N., Brown, D., Sandhu, M., Fridman, M., Aronow, H.U. (2007).
This article examines the impact of mandated nursing ratios in California on key measures of nursing quality among adults in acute care hospitals. This study is a follow-up and extension of our first analysis exploring nurse staffing and nursing-sensitive outcomes comparing 2002 preratios regulation data to 2004 postratios regulation data. For the current study we used postregulation ratios data from 2004 and 2006 to assess trends in staffing and outcomes. Findings for nurse staffing affirmed the trends noted in 2005 and indicated that changes in nurse staffing were consistent with expected increases in the proportion of licensed staff per patient. This report includes an exploratory examination of the relationship between staffing and nursing-sensitive patient outcomes. However anticipated improvements in nursing-sensitive patient outcomes were not observed. This report contributes to the growing understanding of the impacts of regulatory staffing mandates on hospital operations and patient outcomes.
Impact of California’s Licensed Nurse-Patient Ratios on Unit-Level Nurse Staffing and Patient Outcomes.
Policy, Politics, & Nursing Practice. 6(3), 198-210.
Donaldson, N., Burnes Bolton, L., Aydin, C., Brown, D., Elashoff, J.D., Sandhu, M. (2005).
This report examines the impact of mandated nursing ratios in California on key measures of nursing quality among adults in acute care hospitals. This study is a follow-up and extension of our first analysis exploring nurse staffing and nursing-sensitive outcomes comparing 2002 pre-ratios regulation data to 2004 post-ratios regulation data (Donaldson et al., 2005b). For the current study we used post-regulation ratios data from 2004 and 2006 to assess trends in staffing and outcomes. Findings for nurse staffing affirmed the trends noted in 2005 and indicated that changes in nurse staffing were consistent with expected increases in the proportion of licensed staff per patient. This report includes an exploratory examination of the relationship between staffing and nursing-sensitive patient outcomes. However anticipated improvements in nursing-sensitive patient outcomes were not observed.
Impact of California's Licensed Nurse-Patient Ratios on Unit-Level Nurse Staffing and Patient Outcomes.
Policy, Politics, & Nursing Practice. 6(3), 198-210.
Donaldson, N., Burnes Bolton, L., Aydin, C., Brown, D., Elashoff, J.D., Sandhu, M. (2005).
This article presents the first analysis of the impact of mandated minimum-staffing ratios on nursing hours of care and skill mix in adult medical and surgical and definitive-observation units in a convenience sample of 68 acute hospitals participating in the California Nursing Outcomes Coalition Project. Findings, stratified by unit type and hospital size, reveal expected changes as hospitals made observable efforts toward regulatory compliance.
Leveraging Nurse-related Dashboard Benchmarks to Expedite Performance Improvement and Document Excellence.
Journal of Nursing Administration. 35(4), 162-171.
Donaldson, N., Brown, D.S., Aydin, C.E., Burnes Bolton, M.L., & Rutledge, D.N. (2005).
Using nursing quality benchmarks in operational dashboards and translating those data to drive performance excellence is a strategic imperative. Since access to unit-level, hospital generated nurse-related benchmarks is an emerging arena, the authors provide an overview of aggregated trends and benchmarks gleaned from the California Nursing Outcomes Coalition acute care database for 2 established nurse-related quality indicators-patient falls incidence and hospital-acquired pressure ulcer prevalence.
Creating and analyzing a statewide nursing quality measurement database.
J Nurs Schol. 2004;36(4):371-378.
Aydin CE, Donaldson Bolton LB, Donaldson N, Brown DS, Buffum M, Elashoff JD, Sandhu M.
The purpose of the study was to present a replicable methodology for designing and analyzing a large, ongoing quality database to examine nurse staffing and patient care outcomes in acute care hospitals. The quality database gathered staffing data from 842 units in 134 acute hospitals, making it the largest statewide effort of its kind in the U.S. The database included clinical outcome information on 34,262 reported patient falls, pressure ulcer prevalence data on 41,982 patient observations, and service outcome data on patient satisfaction from 26,461 patients.
Outcomes of adoption: measuring evidence uptake by individuals and organizations.
Worldviews on Evidence-Based Nursing, Third Quarter (Suppl.); S41-S51.
Donaldson, N.E., Rutledge D. N., Ashley, J. (2004).
The translation and diffusion of findings into health care validates the potential of evidence-based innovation to improve clinical practice and affirms the benefits of society's investment in advancing science. This manuscript briefly reviews key concepts in the knowledge use process, considers theoretical implications for measuring outcomes and uptake of innovation, discusses issues to consider in planning for measurement of adoption and provides an example of confronting those challenges from a project now in progress. Conclusions: Efforts to change practice in order to enhance evidence-based patient care must integrate monitoring and evaluation of specific target outcomes of adoption as the basis for validating the impact of the change.
Nurse staffing and patient perceptions of nursing care.
J Nurs Admin. 2003;33(11):607-614.
Burnes Bolton L, Aydin CE, Donaldson N, Brown DS, Nelson MS, Harms D.
The purpose of the study was to examine the relationship between nurse staffing and patient perceptions of nursing care in a convenience sample of 40 California hospitals. Nurse staffing alone showed a significant but weak relationship to patient perceptions of their care, indicating that staffing is likely only one of several relevant measures influencing patient perceptions of their nursing care. This research contributes data to the body of knowledge regarding nurse staffing. It is essential that nurse executives integrate results from this and other studies in developing strategic and tactical staffing plans that yield positive patient care outcomes.
A response to California's mandated nursing ratios.
Journal of Nursing Scholarship; 33(2). 179-184.
Burnes Bolton, L.., Jones, D., Aydin, C.E., Donaldson, N., Brown, D.S., Lowe, M., McFarland, P. L., Harms, D. (2001).
The purpose of the study reported here was to explore the need for evidence-based health policy as illustrated by the mandatory staffing bill passed by the California state legislature in 1999. The authors conclude that the evidence that currently exists is insufficient to determine minimal RN staffing requirements.
Hospital nursing benchmarks: The California Nursing Outcome Coalition Project Experience.
Journal for Healthcare Quality, 23(4), 22-27.
Brown, D., Donaldson, N., Aydin, C., Carlson, N. (2001).
The California Nursing Outcomes Coalition (CALNOC) project is an initiative that has become the largest ongoing nursing quality measurement repository in the nation. Launched in 1996 by California nursing leaders concerned with trends in hospital care, CALNOC has created reliable quality benchmark data to define patient safety thresholds in California. This article describes CALNOC's effort, which aligns with the strategy of the National Quality Forum for measuring and reporting healthcare quality. By tracing the evolution of the CALNOC project and its future potential, we hope to encourage other grassroots efforts to build the database repositories needed for healthcare quality measurement in the 21st century.
Nurse staffing in California hospitals 1998-2000: Findings from the California Nursing Outcomes Coalition Database Project.
Policy, Politics, & Nursing Practice, 2(1). 19-28.
Donaldson, N.E., Brown, D.S., Aydin, C.E., Bolton, L.B., (2001).
This article describes nurse staffing in 330 critical care, medical-surgical, and step-down units in 52 acute care California hospitals that was reported over nine quarters between April 1998 and June 2000. These data, representing more than 3 million patient days of care, comprise the largest prospective descriptive sample of nurse staffing, using standardized measures, reported to date. These data are especially timely as the profession, policy makers, and regulators in California and the nation respond to the legislative mandate to establish nurse-to-patient staffing ratios that ensure patient safety and develop methods to monitor the impact of ratios on the quality and outcomes of patient care. Findings reveal relative stability over the nine calendar quarters, no significant differences between groups of hospitals stratified by using average daily census to cluster by hospital size, and wide variation in staffing across hospitals within the same unit type categories.