Star data and assigning monetary value to outcomes

Outcomes Star & Cost-Benefit Analysis (CBA) #

Cost–benefit analysis (CBA) is an approach to comparing costs and benefits for stakeholders of a specific intervention or service to those of alternatives/no intervention. Stakeholders are those that experience material change because of the service – e.g. service users, local council, central government (benefits for individuals and the state are usually measured separately).

While traditional CBA has emphasised the economic costs and benefits, alternative types of CBA such as social CBA and Social Return on Investment (SROI) have been introduced to complement the conventional method. SROI is more in line with the Outcomes Star approach because it is oriented towards internal learning and demonstrating value across a wide range of outcomes, emphasising what matters to service users and other stakeholders based on questionnaires or semi-structured interviews/workshops. For a fuller description of these approaches see NEF (2013). We have explored the use of the Outcomes Star as a measure in CBA with clients previously, and learned some things to consider when using the Star in this context:

Scope/stakeholders impacted: The Outcomes Star focuses on benefits for the person completing it, though wider benefits could be extrapolated (e.g. the impact of employment or independent living on relatives, the community or society)

Defining the impact of the service: The outcome areas and Journey of Change in the Outcomes Star are informed by what is important to practitioners and service users involved in the development as well as the research literature and other consultation. However, as mentioned above, in a true SROI, outcomes should be defined by stakeholder consultation within the particular project in question

Assigning financial values: Value banks are used to assign financial proxies to particular outcomes and indicators– for example, the cost saving from better health, with the indicator of avoiding an A&E visit, or better mental health linked to a 1-point improvement on a wellbeing measure. When the Outcomes Star is used in this context, there are some important considerations:

1. Outcome areas sometimes cover more than one issue, which can make it harder to be certain about the status of each individual issue from the reading. For example, in an area including employment, training, education and other meaningful use of time, even when someone progresses to 10 they may or may not be in employment.
2. The Journey of Change is not purely measuring severity so thresholds at which severity reductions are achieved, or things change in a material sense need to be identified (e.g. when there is progress from 1-3 to 4 or 5) – or probability of outcomes such as employment estimated at each Journey of Change stage.
3. The Journey of Change assesses engagement with services so costs of the service may initially increase as people accept help. The benefit period is therefore worth considering.
4. More than one outcome area may link to the same indicator so care must be taken not to double count.

In conclusion, the Stars are not developed or tested with CBA in mind and the features described above can present challenges. Nevertheless, with careful consideration and transparency about the assumptions and caveats, Star data can be a useful part of assessing benefits in this context.