Learning + Evaluation

Barr Klarman Massachusetts Arts Initiative

Insights

We’re committed to ongoing strategic learning.

Through learning and evaluation, we seek meaningful and actionable insights to inform our strategies and the field.

At Barr, we support learning activities at multiple levels. The mission for our team is to support a culture and practice of learning alongside our grantees, using evidence to inform our decisions, evaluating our work, advancing equity, and holding ourselves accountable.

Our Learning + Evaluation Team partners across Barr to:

Help program teams ground their strategies in evidence and build on existing knowledge.
Help colleagues articulate what success looks like and to integrate learning and evaluation into their strategies.
Facilitate ongoing, data-informed, collaborative learning and adaptation with individual grantees, and with larger cohorts or initiatives.
Deepen understanding of our context and test assumptions through activities such as focus groups or other methods of listening to communities, landscape scans, or public opinion polls.
Collaborate with other funders and evaluation consultants to share what we’re learning through multiple communications methods.

We intentionally put “learning” first in our team’s name to reflect our primary purpose: to make continual strategic learning an integral part of Barr’s culture. We support learning and evaluation activities that are meaningful, actionable, manageable, and equitable.

Knowledge Center

The Knowledge Center contains a collection of relevant learnings from our partners and in our program areas. These reports and evaluations are free to view and download.
Visit our knowledge center

Meaningful

Our learning and evaluation activities aim to produce relevant information on questions that support foundation and grantee learning and impact, and advance understanding in our fields.

We emphasize learning and evaluation approaches that recognize the complexity of systems change work and the reality that the greatest impact comes when we listen, learn, and adjust along the way, such as formative or developmental evaluation. This way, Barr and our grantees learn together about what is working well and when a new approach is needed. We also invest significantly in outcome measurement to better understand the impact of our work, gauge whether conditions are changing, and gain insight into potential factors leading to, or impeding positive change.

Much of Barr’s grantmaking is for general operating support, where grants don’t define specific outcomes. We strive to provide flexibility and to minimize reporting requirements, so that grantees can focus on doing their work, rather than measuring and reporting to us. This has led us to shift learning and evaluation practices to ensure that insights are meaningful for our grantees.

GreenRoots

Actionable

We focus on questions and information that can be applied directly to increase the impact of an intervention or strategy. As part of our strategic learning process, each program team has identified a set of medium- and long-term outcomes that they believe would indicate meaningful progress toward their strategy goals. We collect data on these to shed light on the progress they are making, and to inform adjustments to our approach. Recent examples of actionable information include Climate polling on natural gas and the Education program’s use of Indicators of School Quality.

In major initiatives where we engage evaluation consultants to help us assess our progress, our grantees are our partners in learning. Such evaluations (like this one related to high school transformation) are designed to provide insight to Barr and its grantees about our collective progress, to capture key lessons, and to note opportunities for improvement. In these multi-year partnerships, taking action based on feedback from our grantees is essential to ensuring that Barr’s support is effective and responsive to sector needs.

Chelsea Opportunity Academy

Manageable

Our activities are structured to maximize learning and minimize burdens on our partners and staff. The most important step here is to spend time up front to think deeply about the most critical few pieces of information we need (vs. the universe of data that would be “nice” to have), for what purposes and audiences. Too often, evaluation and measurement start with trying to anticipate all the possible information one might want, resulting in costly, complex, and difficult processes.

One of the key ways we aim to maximize learning at the grant level is by linking grant outcomes to the grantee reporting process. This is a collaborative process; in the grant application phase, many grantees co-design outcomes with their program officer, and these outcomes ultimately shape reporting requirements. We strive to ensure that these outcomes are meaningful, realistic to measure within the grant time frame, and focused on what is important to grantees and not just the foundation.

For more information and examples, explore this resource on creating outputs and outcomes, Kerri Hurley’s blog on our grant application process and CEP’s blog on removing grantee burdens.

GreenRoots

Equitable

We continually examine our learning and evaluation practices through an equity lens. As a Practice Partner with the Equitable Evaluation Initiative, we recognize our responsibility as a funder to shift and influence the way in which evaluation is practiced and used, meaning that we seek to align our evaluation practices with Barr’s commitment to equitable grantmaking, and that we use evaluation as a tool for advancing equity, in alignment with the Equitable Evaluation Framework.

Our Learning and Evaluation team also supports our grantmaking teams in analyzing the type and level of funding dedicated to organizations led by people of color and organizations with an explicit focus on racial equity and sharing those results, such as through this blog post from our Director of Climate.