Greater NY Evaluation Evolution
In 2021, at 12 years, Greater NY has grounded its evaluation framework around three core ways we think about program success – satisfaction, goal attainment and development of cross-sector leadership practices. This section lays out how this evaluation framework evolved with lessons learned and decisions made as we evaluated the program to understand its real impact.
City Hall – Strengthening Organizations (2009-2014)
Greater NY was launched as a Mayor’s Office program to strengthen nonprofits who were contracted to deliver critical services and whose financial sustainability was at risk in the aftermath of the 2008 crisis. During its time in City Hall, Greater NY evaluation focused on the impact of partnerships on three elements of nonprofit sustainability: 1) increased financial stability 2) strengthened Board management and 3) better communications. Each element was evaluated against four or five indicators that determined outcome attainment.
As data was collected, stronger data emerged about Greater NY’s impact on leadership than on organizational sustainability and qualitative data pointed to the uniqueness of the cross-sector advisory relationship between Greater NY partners.
Key Learning: A partnership that meets an hour a month may not move the needle for an organization but can move the needle for a leader especially when focused on a particular issue.
Leveraging Private Sector Advice (2016-2018)
After leaving City Hall and with latitude to re-design the evaluation framework, Greater NY deepened its exploration of the impacts of cross-sector partnership. An assumption when the program was launched was that it would build “hard” skills for nonprofit leaders – budgeting, forecasting etc. But what emerged was an almost universal focus on the “soft” skills of partnership – relationship management, culture building, talent development. In the first two years post City Hall, Greater NY tried to understand the type of private sector advice that was most helpful in developing these “soft” skills.
From the earliest days of this approach, there was significant pushback from corporate and nonprofit partners on the notion that it was the private sector nature of the advice that was impactful. Participants urged staff to look beyond sector expertise and to look more closely at the unique nature of the Greater NY relationship to understand what makes this relationship one where so much is learned and gained from each other.
Key Learning: Greater NY’s core innovation is the partnership relationship and understanding the dimensions of this relationship’s impact would strengthen outcomes.
Cross-Sector Leadership Practices (2018 - present)
In 2018, with two cohorts launched post-City Hall and the understanding that looking for the impact of private sector advice did not capture the program’s true impact and did not represent the equity in partnerships, staff began to think of different ways to look across partnerships and understand the program’s broad impact. As partnership numbers grew, staff observed similar experiences and impacts particularly in the relational impact of the partnership. Evaluation work turned towards understanding the contours of the Greater NY relationship, identifying the key places partnerships were most successful and building an approach that would enable staff to measure and manage partnerships towards a few points of shared success. With this approach, staff hoped to be able to coach every partnership towards great outcomes and offer every participant the opportunity to get the most out of the Greater NY partnership relationship.
Staff undertook a multi-step process to evaluate the program and identify outcomes. This process included: 1) a deep dive into qualitative data gathered across the program’s first 7 years and 60 completed partnerships 2) tagging all 60 completed partnerships for key outcomes 3) identifying clusters of outcomes 4) deploying an evaluation tool (survey) for the class of 2019 to test a hypothesis that Greater NY partnership impact generally fell in four key leadership areas for both corporate and nonprofit partners 5) gathering and analyzing data and conducting extensive follow up interviews to better understand the data collected 6) refining leadership areas and coming up with descriptive names for each and 7) developing final indicators for the identification of these areas, which were identified not as skills but as practices.
Key learning: The activity of Greater NY partnership develops practices,
not qualities, that are critical to external facing leadership, cross-sector leadership, and are transferable to other, non-Greater NY relationships.
When Greater NY began developing its evaluation there were no existing frameworks for cross-sector executive partnership programs. We needed to come up with something on our own.