Evaluation and Learning

A rigorous evaluation is a central part of our Community Health Initiatives effort at Kaiser Permanente. We use the information gathered through the evaluation to improve our programs and the impact they have. We also contribute to the field by building the evidence base for what works and sharing that with others.

Our 5-point evaluation philosophy values program improvement, organizational learning, and community participation in the evaluation process:

Use data to improve, not just to prove.
Measure the impact of our investments to drive effective interventions.
Engage community partners as participants and stakeholders.
Develop new methods and share with the field.
Leverage Kaiser Permanente’s unique assets.

Evaluation Approach

Kaiser Permanente’s goal for its place-based Community Health Initiatives is ambitious: to improve the health of every resident living in the 40+ participating communities. These “population-level” improvements focus on achieving outcomes such as increased physical activity, healthier diets, reductions in obesity, and improvements in obesity-related health issues such as high blood pressure and diabetes.

The design principles that form the foundation for Community Health Initiatives provide a roadmap for achieving population-level outcomes, which is summarized in the Community Health Initiatives logic model. The overarching idea is that collaborative approaches to changing community policies and the environment will lead to sustainable changes that promote healthy eating and active living, and ultimately, improvements in health outcomes.

The evaluation process measures the success of an initiative by following these actions:

Documenting whether sustainable community changes are made (e.g. Are more children physically active during the school day? Are local grocery stores offering more healthful items?)
Assessing the impact of those changes on behavior.
Measuring long-term population-level outcomes using surveys of youth and adults living in the communities, Kaiser Permanente’s electronic health records and other data.

The Impact of Dose on Population Health

An interesting learning to date has been around the concept of “dose.” Dose entails estimating the number of people exposed to a particular health intervention and the impact that the intervention has on changing behaviors to improve health outcomes.

The following tools provide a good overview of the dose methodology and how it has been used to inform Kaiser Permanente's Community Health Initiatives:

Using the Concept of “Population Dose” in Planning and Evaluating Community-level Obesity Prevention Initiatives
(study published in the American Journal of Evaluation)
Measuring and Increasing the Dose of Community Health Interventions
(an overview)
Community Health Initiatives Population Dose (powerpoint)

The CHI evaluation approach is summarized in the following publications and reports:

Cheadle A, Schwartz PM, Rauzon S, Beery WL, Gee S, Solomon L. The Kaiser Permanente Community Health Initiative: Overview and Evaluation Design. Am J Public Health 2010 100: 2111-2113. [Presents the design framework we use for our place-based Community Health Initiatives, with findings from our Northern California sites]
Cheadle A, Samuels S, Rauzon S, Yoshida SC, Schwartz PM, Boyle M, Beery WL, Craypo L, Solomon L. Approaches to Measuring the Extent and Impact of Environmental Change in Three California Community-Level Obesity Prevention Initiatives Am J Public Health 2010 100: 2129-2136. [Presents our methodology with findings from our Northern California sites.]
2010 Summary of Evaluation Approach [Highlights the overarching strtegy and design framework for our 40+ collaboratives with results for 26 sites across California and Colorado]

The Center for Community Health & Evaluation has been leading the evaluation work around our Community Health Initiatives at Kaiser Permanente. For more information, visit the CCHE webpage.

Evaluation Results to Date

Kaiser Permanente is currently working with more than 40 communities across the country to improve population health through Healthy Eating Active Living collaboratives. We compile and report evaluation information in several ways, ranging from detailed individual community case studies to overall summaries across the Community Health Initiatives program. Because each collaborative was launched at varying times and because each community effort is implemented over multiple year, data collection that includes follow-up population-level surveys has been completed in only a few sites to date. You can find our results presented in the following reports.

Stay tuned to this section as we continue to update our findings.

For more information on evaluation methods and findings, contact Natl-CB-Communications@kp.org.