Back away from the Research Methodologies (and leave them to the experts)

Lately, in magazines, blogs, and e-newsletters, I’ve seen several articles in which a few market research methodologies are explored usually through the same three facets: how each methodology works, its advantages, and its disadvantages. This “research methodologies for dummies” influx seems to point out the recent rise in the misuse of methodologies and their results.

The other day I witnessed a study being conducted at a local movie theater in which a slew of research rules were being violated: preset answers were being suggested by the moderator, exact wording was being modified and recorded however the moderator deemed appropriate, and at one point, two respondents’ answers were compiled to fill a single survey (just to name a few).

When methodologies aren’t used correctly, research data deteriorates. Implementing invalid research data more than likely yields poor results. Getting poor results, in this case is a result of methodology malpractice NOT the research field in general. However, since many marketers are unaware of the importance of the methodology chosen and how it will influence the gathered data, they leave behind not only their poorly predicted results and their budget for research, but also a cloud of unmerited doubt around whether research really works or not.

To make matters worse, we also have a surge of people who like to not only criticize the industry but do so poorly with flawed, illogical arguments and often offer no solution in return. Seth Godin attempts to argue that “our personal outlook is a lousy indicator of what works for anyone else” in his post titled “How do I persuade you?”

Here’s the main problem with this question and the rest of his post: the definition of the word “you.” Now, I hate to get all Bill Clinton-y here but I do think this is a very valid point in this context. Godin specifically labels the population to which he is referring as “human beings.” Could he be more right in this context? I don’t think anyone is arguing that the entire human race makes decisions in the same way or even in a similar manner. Decision processes don’t transcend across the entire world’s population, or even the US for that matter.

What I think many would argue is that when you limit that population down to a specific demographic, whether it’s based on geography, age, sex, and/or what brand they purchase, you may be able to narrow this defined population’s decision making process down to a common thread.

Now, John Windsor’s comment in response to Godin’s post somewhat validates my perspective but perhaps needs a bit of sustenance to back it up. Yes, “we need to listen to those we hope to influence, and then adapt our approach accordingly,” but again, to regurgitate Windsor’s original question, “Now what?” There are a couple questions you need to be sure you are answering correctly before you “adapt your approach accordingly.” They are:

1. What is it you want to hear about?
2. What is the most efficient/accurate way to listen to it?

All of the previously listed problems above could be solved by recognizing the inherent relationship between these two questions. Determining which methodology will be the most accurate and efficient all depends on what you are trying to hear. Common desired results include pain points, reactions to a new design, or how satisfied people are with each with its own best-methodology-to-use solution. If you just want a quick rundown of a few methodologies and the types of data they can produce, this is a good start.

So, if you are looking to do research in the near future but are not completely sure which methodology would be the best way to go about gathering your desired results, please ask and find out to so we can fight research methodology abuse and bring it to an end.