Research 101

A friend just shared this Penn & Teller video with me. WARNING - NOT SAFE TO PLAY ON THE LOUD SPEAKERS AT WORK. While not completely academically correct, or a thorough two-sided investigation (not that there are ever many of those), it does quickly and easily explain some basics of survey writing, analysis and how it can all go wrong. And, it is entertaining. Some very basic things to watch when writing a  survey:

1. Non-response bias. Who does not answer can sometimes be more compelling than who does. Quick example: Do a phone survey Monday night on TV matching habits. The results? Hardly anyone watches football anymore! No, not really - the people watching football did not answer the phone. You have a big non-response problem. That is why, for all surveys you want to manage replicates (how much sample you release), contact design (how many times and over how many days you try to reach a record before you call it dead) and always take into account non-response bias.
2. Scales: Likert scales (good Wikipedia overview here) are the standard. But, even how you use these varies widely. A few quick comments:
1. Using 4-point, 5-point, 7-point, 10-point scales. Always a hot debate here, quite enthralling and very sexy. But, in a nutshell I favor 5-point for several reasons. When you use 10-point, you typically roll up "top-box" as 9+10, so why not just go with 5 and make it easier? 10-point causes more respondent fatigue - larger survey questions (actually twice as long), more options, etc. Additionally, what is the difference between and 8 and 9? Moving from 4 to 5 seems a lot clearer form an analysis standpoint and form a respondent standpoint.
2. End-labeled or label all points? If you can, always label all points. The human mind is much better and more consistent at deciding between "somewhat satisfied" and "very satisfied" more so than "3" vs. "4". This is another argument to keep scales short - you can't anchor every point on a 10-point scale, but you can on a 5-point scale.
3. Neutral point? Typically I say no. The idea of using a scale is to differentiate respondents; to differentiate the most satisfied or dissatisfied - to find those levers you can pull as a business to make a difference. Allowing a neutral typically wastes a scale point. Instead I prefer to do a positively weighted 5-point scale (e.g., not at all satisfied, not very satisfied, satisfied, very satisfied, extremely satisfied). We want to make that "top box" the cream of the crop; hard for people to check, but if someone checks it you know you want to figure out how to get more of them. The goal is to move your 3s and 4s to this 5 spot. And, to figure out how to limit the bleeding with your 1s and 2s.
3. Don't Know: The inclusion of "Don't Know" as an answer choice can drastically skew survey results. The classic example is in voter polls. If you only give the option of the two candidates with now "undecided" option your survey results can be drastically different than if you allow for an "undecided" option. Last minute voter turn-out drives, the results of primary races, the overall propensity for "undecided" voters to lean towards a given party - all throw off results by the inclusion or exclusion of this one "little" answer choice.

There are plenty of other ways that survey design can dramatically change how data looks upon completion. You have to take into account cultural differences in scale usage (including colors - there are some great survey platforms that allow visual selections and provide visual cues, but remember while red means stop or bad in the US, it is a positive color in China), respondent propensity to check "Don't Know" (much higher in Japan than the US) and a myriad of other little factors.

Okay, here is the darn video - enjoy!

Design Thinking Applied To New iPad App

I came across this good, quick read on design thinking and applying it to launching a new business - in this case the iPad app Pulse from Alphonso Labs. You can read it here. In a nutshell here are the 5 steps:
1)Empathize: Research and understand the end-user - their needs, pains and usage scenarios.
2)Define: Focus on a core niche and then research that niche to define the why and how.
3)Ideate: Brainstorm sessions and innovation.
4)Create Prototype: These can be rough and even Post-It notes. But create something to narrow and code from.
5)Test: Usability test in the real world.

I would add a 6th one here which is to get Steve Jobs to present your app at the launch of iPad...which happened for Pulse. The result of good design or luck? Probably both, but you won't get lucky with bad design!

Lessons from Hawaii

At the end of June through the beginning of July, I took advantage of Sentient’s sabbatical benefit and took off to Hawaii for two weeks in the great outdoors.  Unplugging for a full two weeks was a little bit of a challenge – I went through email withdrawal for the first couple of days.  Once I did unplug, I was able to regain some perspective outside of my computer and smartphone.

Lesson #1: You have to get dirty to find the best views.

On Maui, we drove the road to Hana and saw LOTS of waterfalls.  By far the most fun we had at one was in Pua’a Ka’a State Park.  However NOT at the lower falls that you just stroll to on the park’s walkway; instead, the upper falls aboe that one.

Lower Falls

Upper Falls

 Our guidebook, Maui Revealed by Andrew Doughty (which we highly recommend), clued us in:

“There’s an awkward trail on the right of the upper falls. It first leads to a short path to the top of the falls, but if you go past it for 5-10 squishy minutes (it’s usually muddy), there’s a much heavier untapped falls and pool just above the diversion ditch that’s taking much of the lower falls’ water. When the trail gets to the elevated waterway (viaduct), you have to walk along it (which those afraid of heights will hate), then across.  Only 100 more feet upstream is your prize.”

Awkward, squishy and muddy are all understatements – the whole trail was practically a slip-n-slide.  At one point my hiking shoes were completely submerged in mud. These were new shoes when I left for Hawaii, and after this hike, they looked like this.

There were at least 3 different times where I wanted to turn back – crossing the viaduct was one of them.

But being the only person swimming in the waterfall’s pool was definitely worth all the work.

Takeaway: Don’t be content with the status quo – find new trails in your work.  From a research perspective, this could mean:

• Using a new technique to understand the desirability of a design described in the paper Measuring Desirability: New Methods for Evaluating Desirability in a Usability Lab Setting (WORD DOC) by Joey Benedeck and Trish Miner of Microsoft
• Trying a new platform from 20|20 Research with a mobile population that lets you reach respondents via text message
• Reading this month’s edition of Quirks to learn more about social media research (PDF)

Lesson #2:  The human mind tries to relate new things to things it knows.

We also visited Mount Haleakala, the dormant volcano on Maui.  Going up the 10,023 feet, I saw terrain and vegetation that reminded me of the alpine climate I have seen in the Colorado Rocky Mountain National Park.  However, there were some plants, like the silversword, that I had never seen before.  We were also lucky enough to see one that was blooming – something that they only do once in their lifespan of 50 years before dying.

And then there was the crater.  This was terrain that I had never seen before, yet my brain still tried to relate it to something – namely what I imagine the terrain of Mars to look like.

Takeaway: When presenting something new, make it relevant.  For example, relate new research findings to things that your audience might be familiar with – previous research, industry news, etc.

Lesson #3: You’re more interesting when you’re moving.

Starting to wonder if I spent any time in the ocean?  Well yes, we went on a snorkel trip to Molokini.  We saw all sorts of coral and fish.

On the way back to Maui, some dolphins started swimming around our boat.  The captain stopped and the dolphins came to check us out.  They played a bit, and then started swimming away.  When the boat started moving again, the dolphins were immediately back, in front of the bow, and racing the boat – we were much more interesting when we were moving.

Takeaway: MOVE on your findings.  At Sentient, we are always customizing and finding creative and actionable ways to interpret and present research that allows our clients to:

• Know the why behind the what
• Easily distribute market intelligence throughout the organization
• Take action and build consensus based upon market research
• Have complete confidence in the analytics and validity of implications

Ideas On Usability Presentations

So, this could be considered a lazy blog post - a blog post about another blog post. And, this other blog post is about other posts and presentations on usability, so this could be considered extremely lazy on my part.

However, click here for the Useful Usability blog and get links to 5 Radical Ideas From Usability Presentations. There is stuff in here for UX professionals, iPhone developers and tips on presenting research findings. All good stuff.Share it this post around and let us know what radical ideas you have for presenting usability research.

Here is a quick link to the first presentation covered:

Usability 2.0

View more presentations from Greg Bell.

The Power of Mobile Market Research

As advertising to the public via mobile phones becomes more widespread, it will be interesting to see what the future holds for other marketing activities – including market research. Market research will need to keep pace with location-based marketing services like FourSquare, wantANDfound, Gowalla, Loopt, Whrrl, Placecast and many others. Emerging as the latest medium for reaching consumers, how can mobile research technologies be utilized to gain deeper and richer insights into behavior and drive innovation? Think about it, what if customers could give feedback anytime, anywhere to improve products and services – right when the issue happens or the idea arises!

A relatively new tool we have come across is QualAnywhere from 20/20 Research, Inc. QualAnywhere allows you to build panels or do one-off recruiting and then interact with respondent via their mobile phone for surveys or qualitative chat. A few examples 20/20 Research gives as ways to utilize this methodology includes:

• Conducting studies about eating habits DURING the lunch hour
• Conducting studies on shopping WHILE participants are likely to be at the grocery store

Interacting with respondents at the POD (Point-Of-Decision) is a rarity, but when available adds to the research validity and actionability. How do you conduct research at the POD? I would love to hear about it and how it helped your research.