Should an e-commerce design agency test the usability of its own designs?

[Paul Bryan-2]

Recently I was on an e-commerce strategy project. I received a
usability test report that the previous agency had produced after
testing their own design work. I went back to the source tapes and
there seemed to be a dramatic difference between the level of
problems users were having in the sessions, and the resulting report.


I know it's convenient for e-commerce site owners to get an
integrated package, esp. when large MSA's are in place. And trying
to keep ahead of Agile cycles puts strain on the schedule and number
of partners. But I'm just wondering if readers of this list feel
like there is an inherent conflict of interest, or if testing is
viewed as a normal component of a design partner relationship.  

Paul Bryan
Usography (http://www.usography.com)
Blog: Virtual Floorspace (http://www.virtualfloorspace.com)
Linked In: http://www.linkedin.com/in/uxexperts



________________________________________________________________
Welcome to the Interaction Design Association (IxDA)!
To post to this list ....... discuss@...
Unsubscribe ................ http://www.ixda.org/unsubscribe
List Guidelines ............ http://www.ixda.org/guidelines
List Help .................. http://www.ixda.org/help

[Kristen Connor]

Paul, I think your example speaks more to a lack of professionalism on
the part of the previous agency than an inherent conflict of interest.

At my firm, Lofthouse Design, we have always conducted usability
testing on the sites and applications we design. In doing so, we find
opportunities to improve the product we deliver to our clients -- and
we do not take any criticism as an attack on our design skills.

For example, we recently tested an online enrollment application with
a sample of Medicare-age seniors. Although no users complained, we
observed first-hand that users struggled to use standard form
controls like radio buttons. They spent unnecessary time and effort
trying to click on a small target.

Based on this observation, we identified an opportunity to improve
the usability of basic form controls. We created a set of custom UI
elements that visually highlight whenever the user mouses over any
part of the radio button or its label, creating a much bigger target
for the user to click.

Had we not conducted the user testing ourselves, we may not have seen
such a subtle usability problem -- nor could we have come up with an
innovative design solution as we did in this case.

Hope this example helps.

 




. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Posted from the new ixda.org
http://www.ixda.org/discuss?post=47287


________________________________________________________________
Welcome to the Interaction Design Association (IxDA)!
To post to this list ....... discuss@...
Unsubscribe ................ http://www.ixda.org/unsubscribe
List Guidelines ............ http://www.ixda.org/guidelines
List Help .................. http://www.ixda.org/help

[Jesse Zolna]

I agree that this sounds like a lack of professionalism.  Maybe they
did not spot the same issues that you saw?  If they saw the issues, I
am not sure why they would not have taken those learnings to heart and
used them to improve the design.  In the end, better design would make
them look better than some report %u201Cvalidating%u201D the usability
of their design.  

Better design (however you define it) should be the ultimate goal of
every project, and every and project within the project should
support that goal.  Right?  I don't know why else they would have
done usability testing, except to bill more hours, I suppose.


. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Posted from the new ixda.org
http://www.ixda.org/discuss?post=47287


________________________________________________________________
Welcome to the Interaction Design Association (IxDA)!
To post to this list ....... discuss@...
Unsubscribe ................ http://www.ixda.org/unsubscribe
List Guidelines ............ http://www.ixda.org/guidelines
List Help .................. http://www.ixda.org/help

[Alan Salmoni]

It sounds a little like an own goal here, but only work done by
someone else would show this up. If their usability testing doesn't
show up defects in their design, then I would be asking what are they
being paid for? (except for being able to tick the relevant box)

I guess it's one of those difficult questions for companies to ask
themselves when considering employing someone: if they are expert
enough in a field, why employ someone else? And if not, how do they
know who are the real experts?

This explains why a list of previous clients can be the best selling
point though I've known of products and services that have gained a
good client list based upon the strength of their client list ("they
must be good, look at the big names that use them!").



. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Posted from the new ixda.org
http://www.ixda.org/discuss?post=47287


________________________________________________________________
Welcome to the Interaction Design Association (IxDA)!
To post to this list ....... discuss@...
Unsubscribe ................ http://www.ixda.org/unsubscribe
List Guidelines ............ http://www.ixda.org/guidelines
List Help .................. http://www.ixda.org/help

[mark schraad-2]

I think there is great benefit to having an independent (person or  
group) do the testing of a design. I also think  there is benefit to  
having product managers generate the use cases designers work to  
solve for. Additionally, having the back end development team do the  
QA is troublesome.

Some level of self fulfilling prophesy is likely to find its way in.  
Isn't that why we go to someone else for things as simple as prof  
reading? What I don't think, is that it is necessarily cause for  
inditement. They may very well have had separate staff do the  
testing. If you understand how agencies work, then you know that it  
is very much about billables... and with that you must have  
deliverables. It is very rare for an agency to recommend another  
group in that situation.

Mark


On Nov 9, 2009, at 11:35 AM, Paul Bryan wrote:

> Recently I was on an e-commerce strategy project. I received a
> usability test report that the previous agency had produced after
> testing their own design work. I went back to the source tapes and
> there seemed to be a dramatic difference between the level of
> problems users were having in the sessions, and the resulting report.
>
>
> I know it's convenient for e-commerce site owners to get an
> integrated package, esp. when large MSA's are in place. And trying
> to keep ahead of Agile cycles puts strain on the schedule and number
> of partners. But I'm just wondering if readers of this list feel
> like there is an inherent conflict of interest, or if testing is
> viewed as a normal component of a design partner relationship.
>
> Paul Bryan
> Usography (http://www.usography.com)
> Blog: Virtual Floorspace (http://www.virtualfloorspace.com)
> Linked In: http://www.linkedin.com/in/uxexperts
>
>
>
> ________________________________________________________________
> Welcome to the Interaction Design Association (IxDA)!
> To post to this list ....... discuss@...
> Unsubscribe ................ http://www.ixda.org/unsubscribe
> List Guidelines ............ http://www.ixda.org/guidelines
> List Help .................. http://www.ixda.org/help

________________________________________________________________
Welcome to the Interaction Design Association (IxDA)!
To post to this list ....... discuss@...
Unsubscribe ................ http://www.ixda.org/unsubscribe
List Guidelines ............ http://www.ixda.org/guidelines
List Help .................. http://www.ixda.org/help