Sometimes It Counts To Count -- And Sometimes It Doesn't

A couple of very good posts on the uses of qualitative vs. quantitative data have come my way over the past few weeks. Both of these are well worth looking into:

Andrew Hargadon, author of How Breakthroughs Happen: The Surprising Truth About How Companies Innovate, wrote this on his blog about “the virtues of qualitative research”:

“Qualitative research is, at its heart, an attempt to understand how people (or fish) interpret their reality and as a result make it. Anyone who has both looked at manufacturing statistics and wandered the factory floor knows that you can learn a lot by watching and talking to the workers about their work and their lives. And so, when you decide you want your company to be more innovative--and you decide to reward those who are ‘innovative’--you need to be very careful how you are measuring innovation.”

In an unrelated post, Bob Sutton, author of The No-Asshole Rule, discusses on his blog the current focus of many management gurus (including Sutton himself), “evidence-based management”:

“Managers and the business press seem to automatically assume that quantitative evidence is always the best evidence....The message seems to be that evidence-based management means management by quantitative data. I reject that thought, and have always believed that there are times when qualitative data are more powerful, valid, and useful for guiding action than quantitative data.”

Sutton goes on to describe three areas he feels it essential that companies use qualitative data:

1. When you don’t know what to count.

My take: This is what we often use qualitative techniques for in market research. If you don’t know what specific kinds of answers you're looking for, you can’t even construct a questionnaire with closed-end questions. So, often the first step is a qualitative study that will allow you to understand what’s important and what even begin to understand what can and should be measured.

2. When you can count it, but it doesn’t stick.

My take: It seems to me that this is the least compelling of Sutton's reasons, if only because so many companies seem to want numbers of some kind before they’ll make a decision. But it is true that compelling stories and images, which can only come from qualitative research, can be very persuasive.

3. When what you can count doesn’t count.

I don’t have anything to add here that’s better than Sutton’s observation: “In the hunt for and obsession with what can be counted, the most important evidence is sometimes overlooked. As Einstein said, ‘Not everything that counts can be counted, and not everything that can be counted counts.’ ”