Why I’m sceptical about surveys
Last week I explained why I think it’s better for managers to understand morale by talking to people than by sending out a staff survey. Morale (and other things we care about, like performance) involve messy real world things like ‘feelings’ and ‘relationships’ that you can understand better through experience than through measurement.
I think surveys are in general an over-used tool1, and are not as helpful as we think they are.
A colleague of mine drafted a survey question that said:
“How often would you like to see us running events?”
And gave people a range of options from fortnightly to quarterly. I pointed out that it’s very easy to say you’d like to see two events a month - and perhaps it would make you feel good in some way to know they were happening. But events are only really worthwhile if someone turns up. So a better question might be:
“How many events would you realistically go to?”
This raises the stakes a bit.
But even then, it’s still too easy to over-commit. You might think you’d go to one event every two months. It’s easy to imagine a theoretical future version of yourself to doing that. But when you eventually become the future version of yourself, you discover that you’re always busy2, and you go once every six months at most.
Put simply: We do not know what we want. At least, not reliably and completely.
Surveys assume generally assume that we do - and are willing to talk about it. I think these are rather shaky assumptions.3
This is why the most effective ways of working4 put emphasis on building things incrementally. It’s so you can try them out in the real world and let people show you how (and if!) they want to use them. It’s in real situations that you discover people’s real preferences.
This is also why it’s so important to make full use of the surface you have in contact with your customers. Every time someone in your organisation has an interaction with a customer, you might learn something valuable about what they want5.
Very often, the people in most frequent contact with customers are your more junior employees. So then the challenge becomes how you, the organisation’s leaders, can learn from them. They’re out there, talking to customers, and getting input through all their senses. Since they’re a person, they cannot help but be impacted by other people. They have an experience of other people’s experience. That experience leaves them full of information that goes beyond their ability to put it into words.
If you want access to the rich seam of information, then you have to go and talk to your junior team members. This is not just so you can listen to what they to say. It’s so that you can be impacted by them, just as they were impacted by your customers6.
If you have interactions like this, with junior staff, you will not only learn about your customers, you will learn about the people you work with. You will have less need to do staff surveys, because you will already have a feel for morale. And, as a final thought, consider the impact on your network of relationships of every customer story you discover in this way:
“Telling a story - in any format - is about building a relationship”
- Mark Edwards, ‘Best Story’
I include in this the ubiquitous “net promoter score”.
As beautifully and wittily captured by Amanda Palmer’s song In My Mind.
We all have survey fatigue. Buy any product online, and you can expect a barrage of requests to “tell us what you think”. When another one comes along, how much enthusiasm are we likely to have for completing it? And what implication does that have on the quality of the findings?
Which you’ve probably heard as buzzwords (“agile”, “MVP”, “lean”, etc).
Even the customer might not have been aware of their wishes previously.
There is kind of an ‘oral tradition’ going on here. This is another reason that I like to think of the unit of impact data as ‘the story’.