Introduction

Outcomes measures systems have two major components:

1. Major outcomes and indicators: The substantive content of the system.
2. Data collection systems and procedures: The procedures and technical systems used to collect, compile, analyze, and report the data.

Developing the outcomes measures and systems that will meet your program’s particular needs is best accomplished through an inclusive process that incorporates the input of staff from all units of the program that will be involved in the collection and use of the data. Two simple sets of questions should guide this process:

1. What do you want data for? How do you want to use data that you may want to collect (or are already collecting)?
2. How cost-effective will it be to collect and analyze specific types of data? Will the value and benefits of having different types of data be worth the costs required to collect and analyze them (e.g., staff time required to collect and analyze the data, financial costs of establishing and maintaining dataset)?

Learn about the process BRLS conducted to develop its outcomes data systems or the process Cleveland Legal Aid Society (CLAS) conducted. Other legal services programs that developed and implemented outcomes measures systems may also provide useful information about this topic. View a list of some of these programs.

Developing Outcome Measures

Your program’s mission and goals should provide the basic framework for the development of outcomes measures. The most fundamental outcomes measures relate to the benefits of your program’s casework. Outcomes from extended services cases can be collected and analyzed most easily (learn more about this).

Management and advocates can take the lead role in identifying the outcome measures. The following questions can help frame this process:

• How does your program’s case work relate to its mission and goals?
• What does your program wish to accomplish with its case work overall and in each substantive law area?
• What data do you need to effectively know and evaluate what the program’s case work is (and is not) accomplishing, and to identify ways this work can be improved?
• Aside from the data you are required to report to funders, what data would enable you to more effectively demonstrate to funders the value of your program’s services and the benefits they provide clients?

As discussed here, the outcomes most widely used by legal services programs are the results of cases in different substantive law areas (“problem codes”) of the LSC Case Service Report (CSR) system (e.g., Consumer, Housing, Family, Public benefits. Programs collect non-financial “main benefit” outcomes and financial outcomes for these cases. Programs also collect outcomes data related to success rates.

Developing Data Collection Systems and Procedures

After your program has identified the types of outcomes measures it would like to collect and use, it must develop and implement the procedures and technical systems that will enable it to collect these data most effectively and efficiently. The development of this component of the program’s outcomes data system must be framed by a fundamental question: What are the relative costs and benefits of collecting different types of data?

Your program will need procedures and data management systems that:

1. Enable case handlers to easily enter these data into their program’s case management systems (CMS's) at the time they close a case.
2. Aggregate these data and generate a range of reports in formats that enable the program to effectively analyze and present data for a variety of purposes.

It may be most appropriate for the program’s technology staff to lead this effort. The program must assess the extent to which their case management system and other data management systems have the capacity to perform these functions effectively and efficiently. They may find that the data systems they currently use have these functionalities or that the necessary functionalities can be added to these systems at a relatively low cost.

The vendors of the different CMSs used by the large majority of LSC grantees (e.g., LegalServer, Kemps Case Works, Legal Files, Pika) have informed LSC that their systems have the necessary functionalities for collecting outcomes data or that these capacities can be incorporated into systems.

Programs have employed different approaches to develop the outcomes systems they use. CLAS determined that the program’s case management system lacked the necessary functionalities, so the program’s staff worked with staff of the Ohio Legal Assistance Foundation to develop these functionalities.

Blue Ridge Legal Services and other legal services programs in Virginia partnered with the Legal Services Corporation of Virginia (LSCV), the state’s IOLTA program, to develop the outcomes collection and reporting system used by LSCV’s grantees. (Read more about their process.) LSCV ultimately purchased a case management system for its grantees to improve their effectiveness and efficiency and facilitate their collection and reporting of required outcomes data.