Re: Beta Testing For Usability?

On Mon, Nov 2, 2009 at 12:58 PM, doug.gorman@...
<doug.gorman@...> wrote:
> Hello, everyone!
>
> We're about to roll out a few applications to a beta test group and I was wondering what techniques you've all used to get useful feedback about usability during beta test phases.

Nice question.

> Have you used other approaches or techniques during betas to get great "real world" feedback on usability? What techniques had the highest yield?

Depending on the type of application, a true real-world indicator of
the effectiveness of various UI changes would be to A/B test them
(http://en.wikipedia.org/wiki/A/B_testing).  Assuming you're working
with a web application, you should be able to send a certain
percentage of users (possibly pre-screened and filtered) through the
beta page(s) to determine the overall affect on the target metrics. If
a single change positively effects the target metrics, like conversion
or lead generation, then it's probably a good change and likely makes
the application more usable.

--
Kaleb Pederson

Blog - http://kalebpederson.com
Twitter - http://twitter.com/kalebpederson