Humanising Impact Measurement in Probation Services
With a team of service designers, I supported Elect Her in its role as a systems convener, bringing together 30+ organisations across England working toward gender parity in UK politics. The focus was on strengthening the ecosystem itself — aligning vision, reducing duplication, and optimising collective impact within a resource-constrained landscape.
Project Partners:
6 months, London
Tools & methods used: 1:1 interviews, focus groups, co-design workshops, prototyping workshops, user personas, literature reviews, systems mapping, insight systhesis
TL;DR
The Problem
Catch22 needed a more meaningful way to evidence personal wellbeing progress for men on probation — one that reflected lived experience while satisfying Ministry of Justice requirements.
The Process
Through desk research, interviews with frontline staff and probationers, and co-design workshops, we uncovered tensions between relational practice and rigid reporting structures.
The Intervention
Co-created a conversation-based progress framework that translated tacit practitioner knowledge into structured, evidence-ready outcomes — seamlessly embedded within existing session routines and digital platforms, without adding administrative burden or retraining demands.
The Impact
Preserved practitioner–service user trust by reframing measurement as structured dialogue rather than form-filling. Improved the consistency and integrity of outcome data for Ministry of Justice reporting, translating tacit frontline insight into clearer, policy-recognisable evidence.
Context
Catch22 is a UK social business delivering public services across justice, education, employment, and wellbeing. Within the criminal justice system, their Personal Wellbeing Service supports people on probation as they transition from prison back into the community. Funded by the Ministry of Justice, the service is required to evidence its impact through measurable outcomes.
The organisation approached us with a core challenge:
How might they meaningfully measure the progress of service users in a way that quantifies impact for the Ministry of Justice — while reflecting the lived realities of both practitioners and the men they support?
The Current Situation
At the time, progress was measured through a self-scoring tool completed at the first and final sessions. Service users rated themselves against a set of lengthy evaluation statements on a 0–10 scale, with the difference in scores used to demonstrate impact.
However, the tool was widely viewed as ineffective.
1. Defined Outcomes (MoJ-set goals)
Service users were referred against predefined outcome categories that did not always reflect their individual journeys or context.motional progress.
2. Evaluation Statements (Catch22-designed prompts)
Lengthy and abstract statements required high levels of self-awareness and interpretation, often needing practitioners to translate or explain them.
3. 0–10 Self-Scoring Scale
Service users rated themselves numerically, producing singular scores that lacked nuance and were frequently completed without meaningful reflection.
Separating the tool and the method
Beyond the tool itself, the method of measurement was also ineffective.
To understand why the measurement approach was failing, we conducted initial interviews across three key stakeholder groups:
-
Practitioners delivering the Personal Wellbeing Service
-
Men on probation with lived experience of similar assessment tools
-
Catch22 senior management, responsible for reviewing data and reporting to the Ministry of Justice
This allowed us to examine the issue from frontline, lived experience, and institutional perspectives.
Across stakeholders, we heard a shared frustration:
The tool failed to reflect real progress.
What does progress really mean in the context of an individual's personal wellbeing journey?
Through a series of focused interviews with the practitioners and service users, we uncovered some key observations. The practitioners were spending time building trust and creating a bond with the probationers, making sure they truly benefited from the service.
They were observing the progress being made by each of their service users but this existed mostly only as tacit knowledge, sometimes being recorded in case notes.
Progress was already being observed — it simply wasn’t being captured.
The Big Idea
In this context, small shifts in behaviour were not minor — they were meaningful indicators of progress. Practitioners were already noticing subtle changes: increased eye contact, asking for help, taking initiative, showing up consistently, expressing emotion. These seemingly insignificant moments often signalled growth in confidence, resilience, and self-awareness.
Our insight was that progress was already being observed — it simply wasn’t being captured. By leveraging practitioners’ tacit knowledge and structuring these micro-shifts into measurable growth categories, we could translate relational insight into quantifiable, evidence-ready outcomes.
Crucially, the framework needed to align with the Ministry of Justice’s required outcome measures.
To ensure this, we facilitated a co-creation workshop with practitioners to surface commonly observed behavioural shifts and cluster them into defined “growth categories.”
These categories were then directly mapped to existing MoJ outcome frameworks — preserving frontline nuance while ensuring institutional accountability.
The New Tool
Rather than introducing a separate assessment tool, we integrated the framework directly into the case notes practitioners were already completing at the end of each session. Practitioners draw on their in-session observations and score six agreed “growth categories” — such as confidence, resilience and communication — on a 1–5 scale.
This creates a consistent, session-by-session record of movement, allowing progress to be tracked incrementally rather than only at the start and end of the service. The framework translates relational, qualitative observations into structured data without disrupting the natural flow of sessions.
The result is a tool that captures progress as it unfolds — making behavioural change visible, defensible and aligned with reporting requirements.
Low fidelity prototype developed to test with practitioners
The logic of the framework operates across three levels:
1. Behavioural Observation (Session Level)
Practitioners observe small, tangible behavioural shifts during sessions — for example, a service user initiating difficult conversations, demonstrating improved emotional regulation, or following through on agreed actions. These observations are scored across six cross-cutting growth categories (e.g. confidence, resilience, communication, initiative, commitment, self-awareness)
.
2. Growth Categories (Development Level)
Each growth category represents a foundational capability required to achieve formal rehabilitation outcomes. Rather than mapping isolated behaviours directly to Ministry of Justice (MoJ) outcomes, we identified these categories as underlying drivers of progress across outcome domains.
3. MoJ Outcome Domains (Reporting Level)
The growth categories were co-created and validated with practitioners, then systematically mapped against MoJ outcome areas (e.g. family relationships, responsible parenting, compliance with programmes).
For example:
-
Increased communication and confidence contribute to FS03 (ability to develop positive intimate relationships).
-
Improved resilience and self-awareness align with FS04 (positive coping strategies during relationship breakdown).
-
Demonstrated commitment and initiative support FS05 (compliance with voluntary or mandatory programmes).
Over time, consistent upward movement across relevant growth categories provides evidence of progression toward specific MoJ outcomes.
This layered structure ensures that relational, qualitative insight can be translated into structured, defensible data without reducing complex human change to a single end-point score.
Sample visualisation of the report generated
Impact
Time Saved Per Session
The previous tool required separate completion outside case notes, taking 10–15 minutes per session and duplicating documentation. The redesigned framework is embedded within existing case notes and requires only 2–3 minutes of structured scoring.
This reduces administrative time by approximately 8–12 minutes per session, redirecting practitioner time back into frontline engagement rather than reporting.
Service-Wide Integration & Scalability
Designed for service-wide integration, the framework standardises progress measurement across all frontline practitioners without requiring new systems or retraining. Its structure is transferable and can be adopted across other Catch22 services, including financial wellbeing and youth programmes, enabling consistent impact reporting organisation-wide.
Improved Data Quality
The previous method captured only start and end-point assessments, missing incremental progress. The new framework records session-by-session behavioural scoring, generating longitudinal trend data aligned to Ministry of Justice outcomes. This improves accuracy, consistency and defensibility of impact reporting.
Protecting Trust
By embedding measurement within structured conversation rather than standalone assessment, the framework avoids extractive practices and preserves practitioner–service user trust — strengthening engagement while evidencing progress.