Aim for approximately right

How much effort should a really effective organisation put into gathering data?

I’ve written previously about the trap of becoming over-reliant on data, and about how measurement is not always a good idea. But I don’t think that these things are never useful, and I’m not arguing that we should make every decision based on gut feel. So, is there a sweet spot?

Let’s take a step back for a second and remind ourselves that actually getting hold of reliable data is hard - and then knowing what it means is even harder. This is why we have a replication crisis in science. A whole generation of scientific research thought that it had reliable data with a clear meaning, but on closer inspection it turned out there were a lot of flights of fancy. And the people involved were scientists, trained to do research. Most people using data in companies are not.

So let’s acknowledge our limitations and instead aim for data that is approximately right.

Data doesn't make the decision. You make the decision. Instead of putting a lot of effort into ever finer levels of precision in data that will never quite eliminate the ambiguity of decision making - aim for something useful and timely, that helps you (the human decision maker) orient to the situation.

Case in point: time tracking. If you are a professional services organisation, face up to the fact that time tracking is a fiction. Thinking is not really linear enough for knowledge work to be measured in this way. But it can be a useful fiction, provided you don’t waste too much energy trying to make it precise. Just ask people to estimate once a month the proportion of their time they spent on each project. This should be enough to give you an overview.

The other key think about ‘approximately’ write is: Data gathering is not most people’s value generating job. Do not put the cart (management reporting) before the horse (doing real value adding work for customers).