See the Checklist Now!

Are you collecting apples or oranges?

What do fruits have to do with a sector-wide  data project? In 2010, when we started working on HomeKeeper and the HomeKeeper National Data Hub, we envisioned creating a shared data hub of aggregated program data to help programs measure  social impact and benchmark their success to their peers. Creating a  national data  hub already seemed like a  daunting task, but it was complicated by the fact that  comparing one program’s outcomes to another was  a lot like comparing apples to  oranges, as the saying goes.

The shared  road to shared data

One our first tasks  in building a shared measurement system  was to develop a common set of inputs and standard data collection practices with our earliest pilot members.  This would allow us to create common social impact metrics for the sector. Fortunately, three organizations in the Northwest, OPAL Community Land Trust, Homestead Community Land Trust and Kulshan Community Land Trust, had already begun  aligning  the way they tracked their program intake forms. But when we started looking at the existing data tracking spreadsheets from all the  organizations  in our pilot program, no two programs had the same list.

Some groups were tracking age, others were tracking date of birth.  Some never asked if their applicants were first-time homebuyers and most programs couldn’t tell  whether or not homeowners were moving on to market rate homeownership when they sold their deed-restricted home. Tracking income and area median income (AMI) was even more tricky since some  programs were only tracking the eligibility income, while others relied only on  gross household income.    While every program agreed that tracking the property address  and number of bedrooms  was important, some programs had never tracked  any appraised market values, the subsidies attached to those homes, or even the year the home was built.

In the beginning it was clear that transitioning to a common standard was going to mean change and compromise for everyone. It was a thoughtful process, with a lot of discussion and input over a series of  webinar conference calls. Eventually the discussion yielded some consensus.  For example, we decided  we weren’t  going to track congressional district, because it’s not found on an appraisal and we can probably map that later, we figured.  And we decided to let programs define “first-time homebuyer” themselves because there is  more than one official definition if you ask around.  We agreed  that everyone should start tracking  date of birth of all household members, so we could count children and seniors housed dynamically. And  that everyone should have  an exit survey for outgoing homeowners in place.  Some very good metrics were proposed, but did not make it on the list, like the energy ratings of homes and commute times before and after purchase. If  programs wanted to track more information, they certainly had the flexibility to do so, but our task at hand was to identify what the group  thought  every  program should always track at a very minimum. The data had to be reasonably accessible and of interest to the majority of the programs  to make the cut.

Setting a new standard

Eventually, everyone was convinced it was well worth the effort.    This was despite the fact that  it meant asking  more applicant details on the application and incorporating additional processes, like capturing appraisal information and reviewing  HUD-1 settlement statements after close. By the end of our first year, we not only had buy-in on a common list of fields and practices, but the process had inspired programs  to  suggest new metrics that everyone wanted to add,  like employment and educational status of individual household members.  People  were literally asking us to require them to input more data!

As a result of this year-long process working with program staff and stakeholders  from across the country, along with researchers from the Urban Institute, we ultimately  assembled  a standardized list of data fields and data management practices for  measuring  program performance and impacts. These standards  represents what  the  Cornerstone Partnership and the HomeKeeper community  believe homeownership programs need  to tell a more complete story about the impact of their  programs and our sector as a whole.  As we discover and learn  how to improve the ways we  measure  program outcomes, these standards will no doubt  be refined and expanded.  But for now, they represent our best attempt at identifying and defining the data every program should track.

Join us!

If you manage a homeownership program that preserves affordability over the long term, we  invite you to align your program documents and procedures with this  data collection checklist  developed for you by your peers. This is intended to be used  by all homeownership programs regardless of whether or not you are a HomeKeeper user. By going through the process of reviewing and updating your forms and procedures, your program will reflect many program design best practices. With a community of practitioners collecting the same data in the same way, we can more accurately measure the impact of the sector at national level and you can compare your program metrics to your peers knowing  that you’re making  a true apples to apples comparison.