I’m headed to Vancouver in two weeks to present at the revamped CARF Canada Advanced Outcomes Training.  That training event includes a discussion about the use of client goals to measure program outcomes, so I thought I would get my rant about the limits of that approach out of the way ahead of time!

The overwhelming majority of programs and services I’ve been involved in evaluating over the past several years use some form of client goal achievement to measure their success. I get the attraction. It’s a ‘two-fer’ for many programs! Staff have to define goals for the work they do with clients as part of case management and program accountability expectations (e.g., accreditation), so why not get some extra mileage by using them for outcomes measurement? But the devil is in the detail. Most of the programs I’ve worked with have taken advantage of software that has some form of goal scaling built in. Many software programs (and most in-house solutions) simply require users to indicate whether a goal has been fully achieved, partly achieved, or not achieved at some point in time after the goal is set. Some provide opportunity to indicate why it was achieved or not achieved. There are few (if any) parameters around what achievement means or what a reasonable timeframe for full achievement might be. The system then produces a report counting how many goals are achieved (or not) and links that to program level outcome statements based on categories of goal type that the worker chooses when entering the goal. [Read more…]