Creating evaluation reports using Outcomes Star data #
Below are sections you might usually find in an evaluation report and suggestions on how you can use your Star Data to support your evaluation.
Background and context
What to include:
- The aims of the service – what change the service intends to create for the people being supported
- Who the service supports – including their needs and characteristics
- How the support is delivered – duration, intensity etc.
- Summary of theory of change (how the support leads to outcomes)
Good practice:
- Link aims to measurable outcomes
- Be explicit about assumptions
Method
What to include:
- Details of reporting period
- Details of cohort included (size, demographics)
- Can use Snapshot report of first Stars here to demonstrate needs of cohort
- Description of the Outcomes Star and its use with the cohort.
Good practice:
- Be transparent about any data limitations
- Comment on data quality.
Expected outcomes
What to include:
- What the service expected to see in the data:
- The percentage of people expected to progress in Outcome Areas
- Which Outcome Area most progress should be seen in
- Outcome Areas where it is expected people will maintain or drop back
- Differences between different groups (e.g. based on demographics, geographical areas, support needs etc).
- Key evaluation questions:
- Did people make progress, which Outcome area saw the most progress, maintenance and drop-back, where difference seen between groups?
Good practice:
- Base on evidence, previous experience, knowledge of cohort or prior data.
- Avoid hindsight bias, set expectation prior to data analysis.
Present the data
What to include:
- Snapshot report of 1st Stars to show where people started and their baseline needs
- Distance-travelled report to show the progress they made
- Snapshot report of last Stars to show where people got to.
Good practice:
- Highlight areas where there is strong improvement as well as areas where limited change or drop back
- Segment data to compare different cohorts.
Explain the findings
What to include:
- A clear narrative explaining what the data shows and why
- Comparison with expectations
- Include qualitative evidence (e.g. case studies, practitioner insights, contextual information).
Consider whether:
- Outcomes can be attributed to support provided, or influenced by external factors
- Some Outcome areas are harder for people to make progress in.
Next steps
What to include:
- What has been learnt from the data
- Insights about service delivery
- What changes might be made going forward in response to the data.
Good practice:
- Focus on continuous improvement
- Make actions specific and measurable
- Link findings to actions.