Organizational Capacity Building

Candace LaRue and Associates

Promise neighborhoods panel live blog post #CICSUMMIT

on November 15, 2012

Michael McAfee

Measuring Performance for Promise Neighborhoods: Collecting Data and Reporting Results

Promise Neighborhoods is a current opportunity to work on a core set of results over a period of time. It is grounded in 10 results and 15 indicators. The implementation grant competition is built from a results framework. Indicators are focused on education, communities, and supports. The intention is to use the neighborhood as a unit to advance guiding principles: working at scale (serving all of the children in the neighborhood) and creating a culture of accountability. Many of these principles are similar to the Harlem Children’s Zone. There are right now 5 implementation Grantees working over 5 years. All PN Grantees are following a cradle-college-career strategy. He do we weave systems of support to address the complete continuum. We understand what data will tell us if we don’t learn from past experience. So what? PN is not a new idea, nor is it a silv bullet. It should not be divisive in your community. promise Neighborhoods can hold all of that. If we are doing good cradle to career work, we will be building organizations that can deliver. It is easy to ask for results and to move indicators, but it is hard to ask if we are insetting in organizations to be able to deliver.

The framework: define the neighborhood, identify partners, develop contracts for accountability, capitalize on partnerships to really deliver over the long term. It is difficult for organizations to fund administrative capacity such as data system and days system management. It is important to think about what it really takes to do this work. W have to include all the stake holders, including children, parents, teachers, service providers, and policy makers. It is important to think early on about how each partner will contribute. How do non-profits do the work? How should funders be funding it? And how can policy makers remove variables? There is nothing special about PN…it only becomes special when we do something about it. What makes it special is what we do with it in communities.

The PN institute at PolicyLink was established to provide technical support to PN Grantees. This includes getting the federal appropriation every year, and possibly to make it permanent. They are working to build communities of practice by bringing Grantees together at least 3 times per year. We provide technical assistance – one on one coaching, reports, gathering resources together. How do we build national infrastructure so we can mature as a field? We should not be building new databases and governance structures every time there is a new opportunity. We should be building capacity so these things don’t have to be recreated. Some communities were struggling with data systems, especially those who did not have high capacity regarding data. PNI purchased licenses for Grantees from Social Solutions, so they could hold some of that capacity in their organization. The data system has allowed there to be a common conversation. The data needs to be aggregated to track data at the national level. The PN communities are using a common data platform, working on the same results and indicators, and using the same framework for action. How do communities develop the capacity to do this work at this scale? Some of the capacity needs to be held as national infrastructure. A cradle to career strategy suggests a 20 year commitment – but where is the 20 year investment? If we can invest in PN strategies over the long term then we can achieve results and scale up the work that needs to be done to move an indicator, and build the capacity of organizations so they can move beyond failed models. In congress, people have read all about what non-profits want to do, but where are the results? This is an opportunity to build capacity, to learn to have different conversations, and move beyond this. Promise Neighborhoods is a funding stream and strategy, and it should be used with all the other good work being done in a community.

Jennifer Comey: Data Technical Assistance for Promise Neighborhoods

She is part of the National Indicators Project in DC, and has worked with the DC Promise Neighborhood initiative for 2 years (now moving forward to implementation). The Urban Institute was awarded a contract to provide national technical assistance.

Goals of presentation:
– review efforts to assist PN implementation sites with data definition and data collection
– describe required performance measurement for PN
– Lost potential sources of data
– challenges for data collection and evaluation (place-based initiative)

Task 1: data definitions
– common data definition and methods of collection
— project and program indicators
— implementation indicators
— neighborhood indicators
— various levels of observation: individual, school, and neighborhood
– technical working group

The department of education did not provide a data system or a specific set of requirements around data indicators. The Grantees were able to identify what they thought about data collection, and the UI worked to create commonality among Grantees. Grantees were required to set up a longitudinal data system, which became a case management system. Grantees also need to understand how the neighborhood is changing over time.

20121115-111307.jpg

Goals
1. Enter kindergarten ready
2. proficient in core subjects
3. successful transition from Ms to HS
4. graduate from HS
5. high school graduates obtain post secondary degree, citric action, or credential (without the need for remediation)

20121115-111533.jpg

Goals
6. Students are healthy
7. Students feel safe at school and in their community
8. Students live in stable communities
9. Families and community members support learning in PN schools
10. Students have access to 21st century learning tools

Potential Sources for GPRA Indicators
1. Individual administrative data
2. Aggregated administrative data
3. Neighborhood survey
4. School climate survey

Urban Institute will be publishing a guidance document. At the beginning for UI, the goal was individual level data, but that wouldn’t necessarily allow Grantees to measure the broader neighborhood rather than just people participating in programs.

Case Management Data System
– track participants, services received, and programs on goals
– coordinate and communicate between programs and solutions
– determine effective solutions and make changes

All PN sites are a partnership of multiple organizations, state, local, and non-profit. It is very difficult to create a unified case management data system due to confidentiality, sensitivity, etc.

Task 2: National Promise Neighborhood TA
– TA for sites to collect GPRA indicators and other programmatic performance data
– TA for local data systems

This includes TA on issues of surveys, IRB issues, etc.

Task 3: National PN efforts
– collect grantee data and prepare are restricted – use data files for researchers
– developing code book and documentation of files
– risk disclosure analysis

Task 4: National PN Efforts
– annual performance reports
— develop databae to collect GPRA indicator data
— TA for forms
__ analysis of baseline data and eventual performance in GPRA indicators in aggregate

Challenges
1. Tension between place-based vs project-based initiative
2. era of school choice
3. people move – do the successful families succeed and move out?
4. penetration rates are expected to increase
5. formative evaluation as opposed to a random experimental design

Alison Churilla and Paul Mattessich: Promise Neighborhood, Strive, and MN compas: Lessons from the Nascent Implementation of Data-Driven Cradle to Career Initiatives

Michael started with a conceptual framework, and Jennifer presented a methodological framework. How do you get this to work in a community? How do you work with multiple languages, or competing political interests, or when people believe the computers can do everything for them? wilder has worked with 2 Promise Neighborhoods projects and a STRIVE project, navigating these challenges.

20121115-113040.jpg

Data is meant to tell a story, and MN compass is more than just an indicator project – these are people, individuals, and communities. MN compass provides unbiased, credible information curated by the topic area advisement groups. For example, there is an advisement group for Education with a variety of stakeholders to tell them what information they need to do their job and improve life. MN Compass provides a set of key measures that tell an initial story and the “tip of the iceberg” on these measures. The data is “cut” by different demographics, I.e. race and ethnicity, age, gender, and income. MN Compass also provides additional resources/information – I.e.evidence-based programs.

Information is kept as up to date as possible, as a we based tool – MN Compass

There are three types of PN indicators – Community Outcome Indicators, Promise Neighborhood Outcome Indicators, and Process indicators

COI – education level of workforce, crime rates, property rate, etc
PNI – achievement scores, graduation rates, etc
PI – use of navigators, participation in out-of-school time activities, etc (longitudinal database)

Putting together a team that can gather all these types of data require different skill sets.

Creating indicators is science, politics, and art.

Note: these are my notes from the presentation, not my opinion. In the case of Jennifer Comey, the notes follow her power point very closely (as did her presentation).

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: