How to design effective, low-friction stakeholder surveys

The problem is rarely a lack of effort. It is usually a lack of focus. This article sets out how to design surveys that are proportionate, honest, and genuinely useful for tracking progress on the outcomes that matter.

Start with your outcomes, not your questions

Before writing a single question, go back to your theory of change or outcomes framework. Which outcomes are central to your SROI case? How will you know whether you have achieved the change you set out to make? Your survey should follow directly from that logic, not run in parallel to it.

A useful discipline is to write down the three to five outcomes you most need to evidence, and then ask: what is the minimum data I need to demonstrate change against each one? That constraint is productive. It forces you to prioritise.

If you have not yet mapped your outcomes in a structured way, our guidance on building a theory of change is a useful starting point before designing your data collection tools.

Keep it short enough to get answered honestly

Survey fatigue is real and it affects data quality. Respondents who are partway through a long survey become less considered in their answers. Later questions tend to attract lower completion rates and less reliable responses.

As a general guide, a beneficiary survey for a single programme should rarely exceed twelve to fifteen questions. If you find yourself writing more, consider whether you are trying to do too much in one instrument.

It is often better to use multiple lighter touchpoints across a programme's duration than to front-load a single long survey. A short baseline check, a mid-point prompt, and a follow-up at the end will typically give you richer data than one comprehensive form that nobody completes carefully.

For commissioners and procurement leads, the same logic applies to supplier reporting. Our article on proportionate social value reporting covers how to calibrate evidence requirements to contract size and complexity.

Make space for negative impact

Most surveys are designed with an implicit assumption that the programme has been beneficial. The questions are positive, the framing is affirming, and there is rarely an easy way for a respondent to say that things have got worse, or that the intervention has had unintended consequences.

This is a significant gap. Negative impacts are a formal part of a robust SROI analysis, and ignoring them does not make them go away. It simply means your evidence base is incomplete.

Some practical approaches:

• Use neutral framing. Instead of 'How much has this service improved your confidence?', ask 'How has your confidence changed since taking part?'
• Include at least one open-response question that invites people to describe anything that has not worked well for them.
• Where possible, give respondents the option to respond anonymously, particularly for more sensitive outcomes such as mental health or financial wellbeing.
• Consider whether peer researchers or independent interviewers might surface concerns that a provider-administered survey would not.

Include the deflation questions that honest measurement requires

One of the most important differences between a credible SROI analysis and a simple output report is the treatment of deflation factors. These are the adjustments that account for the fact that not all of the change you observe can be attributed to your work, and not all of the change persists over time. SVE's approach to SROI methodology applies these rigorously, and your survey design should support that.

The five factors to consider are:

• Deadweight: how much of the change would have happened anyway, even without the programme? A question such as 'How likely do you think it is that you would have achieved this without the support you received?' is direct and serviceable.
• Attribution: where multiple organisations have contributed to an outcome, how much of the change is genuinely attributable to your work? Questions that ask respondents to name other support they have received, or to estimate the relative contribution of different services, can help calibrate this.
• Displacement: has the benefit to one group come at the cost of another? This is particularly relevant in employment programmes, where placing one person in a role may mean another candidate does not get it.
• Drop-off: does the impact last? Asking the same question at different points in time, or including a follow-up survey some months after programme completion, gives you evidence on whether outcomes are sustained.
• Leakage: do the benefits accrue outside the intended population or geography? This is more common in place-based or community programmes, where knock-on effects are harder to track but can be significant.

A well-evidenced SROI that accounts honestly for deflation is considerably more credible to commissioners and funders than one that presents unmodified change as direct impact.

From survey to credible SROI

Survey data is only valuable if it connects clearly to your valuation. Before finalising your questions, check that each one maps to a specific outcome and, where applicable, to the financial proxies you are using to calculate your social return. If a question does not connect to either, it is probably not earning its place.

Share your draft survey with a small group of intended respondents before deploying it. A brief cognitive test, asking participants to talk through what each question means to them and how they are deciding on an answer, will surface ambiguities that you will not catch by reading it yourself.

Good survey design is the foundation on which a defensible, meaningful SROI case is built.