As a customer experience researcher, I like online, task-based research for a variety of reasons, most importantly the insights I gain from the users are rich, articulate, honest, and sometimes very surprising. I know why users drop off process flows, what works well, what needs improving, and what is missing, and I have a better understanding of what UI elements users interact with naturally.

Online, task-based user research is great because it combines the best of market research, usability labs, and web analytics. Like surveys, this methodology uses statistically significant sample sizes – 200 to several thousand depending on the goals of the study. It is very similar to usability studies conducted one-on-one because it is a task-based methodology. In addition, navigation paths and behavior are tracked while participants are interacting with the Web site.

Online, task-based, user research is very powerful for understanding not only what visitors are doing on your site, but ‘why’. Surveys are good at collecting user preferences and opinions. However, if you want to know what users do at your site – you need to capture the clickstreams and other behavioral data. Knowing why people drop off process flows, such as a purchase checkout, you gain insights into their attitudes which can lead directly to improvements in the design of the site. Using web analytics alone tells you where users are going and what they are doing, but not ‘why’. Using task-based, online user research helps us understand the gap between what people say they are doing versus what people are actually doing.

As the senior director responsible for the successful delivery of all Keynote’s user research, I like the flexibility this methodology gives me and my team. I have team members in San Francisco, New York, Boston, Bozeman, and London. We can be located anywhere and be able to conduct research for our clients. All I need is good internet connection and a phone to meet with my clients. Recruiting for these studies is easy as well. If I intercept visitors from a web site, then there are no panel costs. I have not provided incentives for intercepted panelist, except in a few cases, for years. People are very brand loyal and will participate in a 25-35 minute study just because they want to help you improve the web site. I can also use third party market research firms or client customer lists to recruit panelists.

I use Keynote’s WebEffective application to conduct online user research. It has a very powerful and easy to use test scripter so I can design a variety research studies. The application also has fully programmed study templates to help new users get started.

The participant view of the application is localized into 17 languages and can be easily customized for your company.

The online reporting tool is my favorite. I don’t have to do any data prep like I do with other tools and methodologies. The application does it all for me. When I close my study, I download all the quantitative questions to PowerPoint—automatically. I don’t have to create PowerPoint graphs anymore. If I want to segment or filter the data, I can download the filtered data to PowerPoint and have charts with multiple bars generated for me.

WebEffective captures thumbnail images of the pages participants go to and I can see aggregated and individual click paths. The application collects all actions when working with sites developed with Web 2.0 technologies. I know where participants hover, click, how much they scroll, whether they are scrolling up, down, left, or right, and if they are interacting with flash elements on the site.

What user research methodologies do you prefer and what tools do you use?