Design Values

[John Schrag-3]

I've written a post for my company's external design blog, that might  
be of interest to this list.

"...my co-workers and I spent several hours considering the question  
of UX design practice.  We weren't considering the question of what  
makes good design;  there are innumerable books on that topic.  We  
were considering the question of what separates a healthy, effective  
design practice from the horror stories we hear about when talking to  
others in the industry...."

The full post is here:  http://dux.typepad.com.   Feel free to comment.

-john schrag
interaction designer
autodesk

[Jared Spool]

Hi John,

This is interesting, but seems to me to be a bit more complex than it  
needs to be.

Sounds to me that what you're saying in the blog post (http://is.gd/1uMjH 
) is "Well-researched user experiences over group-think-produced  
feature requirements."

Is there more to it?

Jared



On Jul 11, 2009, at 8:39 AM, John Schrag wrote:

>
>
> I've written a post for my company's external design blog, that  
> might be of interest to this list.
>
>
> "...my co-workers and I spent several hours considering the question  
> of UX design practice.  We weren't considering the question of what  
> makes good design;  there are innumerable books on that topic.  We  
> were considering the question of what separates a healthy, effective  
> design practice from the horror stories we hear about when talking  
> to others in the industry...."
>
> The full post is here:  http://dux.typepad.com.   Feel free to  
> comment.
>
> -john schrag
> interaction designer
> autodesk
>
>

[John Schrag-3]

There is more to it --- specifically, there's a particular horror  
story or two for each item on the list, which will be talked about in  
future blog posts by members of my team.  I think those articles will  
be more fun to read, once they're written.

I can't really disagree with the accuracy of your simplification, but  
I don't know how useful it is.   People who know what "well-
researched" means don't need to be told, and people who don't often  
think that they do.

-john


On 11-Jul-09, at 9:50 AM, Jared Spool wrote:

> Hi John,
>
> This is interesting, but seems to me to be a bit more complex than  
> it needs to be.
>
> Sounds to me that what you're saying in the blog post (http://is.gd/1uMjH 
> ) is "Well-researched user experiences over group-think-produced  
> feature requirements."
>
> Is there more to it?
>
> Jared
>
>
>
> On Jul 11, 2009, at 8:39 AM, John Schrag wrote:
>
>>
>>
>> I've written a post for my company's external design blog, that  
>> might be of interest to this list.
>>
>>
>> "...my co-workers and I spent several hours considering the  
>> question of UX design practice.  We weren't considering the  
>> question of what makes good design;  there are innumerable books on  
>> that topic.  We were considering the question of what separates a  
>> healthy, effective design practice from the horror stories we hear  
>> about when talking to others in the industry...."
>>
>> The full post is here:  http://dux.typepad.com.   Feel free to  
>> comment.
>>
>> -john schrag
>> interaction designer
>> autodesk
>>
>
>
>

[juliebooth36-2]

John, thanks for this.  I am raptly interested in your real-world experience; as you say, I've heard a lot from people who know what "well-researched" and who can speak academically.  I don't identify so much as I do with real-world horror stories.  I've just signed on with a large freight shipping company as one of only two UX types in the business unit responsible for developing all the software support for the pickup to delivery lifecycle.  Just as I have joined, the development teams are switching to Agile.  Everything is new and exciting here!  I look forward to reading more.  

Peace,

jb

--- In agile-usability@..., John Schrag <john@...> wrote:

>
>
> There is more to it --- specifically, there's a particular horror  
> story or two for each item on the list, which will be talked about in  
> future blog posts by members of my team.  I think those articles will  
> be more fun to read, once they're written.
>
> I can't really disagree with the accuracy of your simplification, but  
> I don't know how useful it is.   People who know what "well-
> researched" means don't need to be told, and people who don't often  
> think that they do.
>
> -john
>
>
> On 11-Jul-09, at 9:50 AM, Jared Spool wrote:
> > Hi John,
> >
> > This is interesting, but seems to me to be a bit more complex than  
> > it needs to be.
> >
> > Sounds to me that what you're saying in the blog post (http://is.gd/1uMjH 
> > ) is "Well-researched user experiences over group-think-produced  
> > feature requirements."
> >
> > Is there more to it?
> >
> > Jared
> >
> >
> >
> > On Jul 11, 2009, at 8:39 AM, John Schrag wrote:
> >
> >>
> >>
> >> I've written a post for my company's external design blog, that  
> >> might be of interest to this list.
> >>
> >>
> >> "...my co-workers and I spent several hours considering the  
> >> question of UX design practice.  We weren't considering the  
> >> question of what makes good design;  there are innumerable books on  
> >> that topic.  We were considering the question of what separates a  
> >> healthy, effective design practice from the horror stories we hear  
> >> about when talking to others in the industry...."
> >>
> >> The full post is here:  http://dux.typepad.com.   Feel free to  
> >> comment.
> >>
> >> -john schrag
> >> interaction designer
> >> autodesk
> >>
> >
> >
> >
>

[Russell Wilson-2]

I think this is great!  Ironically, we are just kicking off our own effort
to document our
design guidelines to frame how we work both as a team and with other
depts/teams.
I'm going to use John's list to seed our thinking... (or to compare to after
the fact).


--------
Russell Wilson
Vice President of Product Design, NetQoS
Blog: http://www.dexodesign.com
LinkedIn: http://www.linkedin.com/in/russwilson


On Sat, Jul 11, 2009 at 9:38 AM, John Schrag <john@...> wrote:

>
>
>
> There is more to it --- specifically, there's a particular horror story or
> two for each item on the list, which will be talked about in future blog
> posts by members of my team.  I think those articles will be more fun to
> read, once they're written.
>
> I can't really disagree with the accuracy of your simplification, but I
> don't know how useful it is.   People who know what "well-researched" means
> don't need to be told, and people who don't often think that they do.
>
> -john
>
>
> On 11-Jul-09, at 9:50 AM, Jared Spool wrote:
>
> Hi John,
>
> This is interesting, but seems to me to be a bit more complex than it needs
> to be.
>
> Sounds to me that what you're saying in the blog post (http://is.gd/1uMjH)
> is "Well-researched user experiences over group-think-produced feature
> requirements."
> Is there more to it?
>
> Jared
>
>
>
> On Jul 11, 2009, at 8:39 AM, John Schrag wrote:
>
>
>
> I've written a post for my company's external design blog, that might be of
> interest to this list.
>
> "...my co-workers and I spent several hours considering the question of UX
> design practice.  We weren't considering the question of what makes good
> design;  there are innumerable books on that topic.  We were considering the
> question of what separates a healthy, effective design practice from the
> horror stories we hear about when talking to others in the industry...."
>
> The full post is here:  http://dux.typepad.com.   Feel free to comment.
>
> -john schrag
> interaction designer
> autodesk
>
>
>
>
>  
>

[Jared Spool]

On Jul 11, 2009, at 10:38 AM, John Schrag wrote:

> I can't really disagree with the accuracy of your simplification,  
> but I don't know how useful it is.   People who know what "well-
> researched" means don't need to be told, and people who don't often  
> think that they do.

I think that's true of all the items on your list though:

     Validated Data over Expert Opinion
     Quality of Data over Ease of Data Collection
     Complete Workflows over Long Feature Lists
     Achieving Results over Writing Reports
     Collaborative Design over Design by Referendum or Design by Fiat
     Ease of Use over Ease of Coding
     Well-designed Critical and Common Workflows over Complete  
Coverage of Every Possible Workflow

In my experience, people who think their opinions are right believe  
they've validated them. People who think easy data collection is  
quality data collection don't understand that it isn't. People how  
believe in thier long lists of features imagine the workflows behind  
them. People who like the reports they've written think they're  
achieving results. Etcetera, etcetera, etcetera.

So, maybe the problem is that you can see something clearly that the  
people you need to work with can't. This list helps you, but would it  
help them? I'm not really so sure.

In my opinion, the nice thing about focusing on experience is that you  
can (a) measure it easily and (b) use the results of those measures to  
drive change effectively.

Jared

[Larry Constantine]

Jared introduced into this thread the simplification:

 

"Well-researched user experiences over group-think-produced feature
requirements."

 

Later the list was expanded with other dichotomous distinctions:

 

    "Validated Data over Expert Opinion

    Quality of Data over Ease of Data Collection

    Complete Workflows over Long Feature Lists

    Achieving Results over Writing Reports

    Collaborative Design over Design by Referendum or Design by Fiat

    Ease of Use over Ease of Coding

    Well-designed Critical and Common Workflows over Complete Coverage of
Every Possible Workflow"

 

As a licensed troublemaker and certified outlier, I am bothered by the
continued virulence of this sort of "versus virus" that infects us so
frequently. Even valid or useful dichotomies are constraining, limiting the
purview to a single dimension or distinction when the world is, thankfully,
ever so much messier and interesting than that. And false dichotomies, such
as most of those listed, are for more insidious in the way they can poison
thought and practice. In my own design work and teaching, I try to avoid
them like the plague they are, to preach and to practice "both/and" thinking
over the appealing but so often wrongheaded "either/or" thinking
encapsulated by polar opposites that may be neither poles nor opposites and
so often are little more than theological pronouncements in disguise: one
pole is the "obvious" good and the other is the implied evil, to be avoided.

 

For instance, quality of data and ease of data collection need not be in
opposition. The brain breakthrough that is needed is to look at the methods
and processes by which we gather data and think about what might be done to
make them easier to execute and more efficient in the generation of quality
data. Just because everybody does contextual inquiry does not mean it is the
only or the best or the most efficacious way to build validated
understanding. When you start thinking both/and you begin to ask questions
like: What is inefficient about method A? What about it so tightly ties the
quality of data to the effort expended? Maybe there are better ways that
could yield rich and useful data that is easily gathered.

 

Or consider Jared's opening offering. It implies that group-think (clearly
and self-evidently evil) is the alternative to well-researched user
experiences. I would never presume to suggest that user experience research
is bad (although it is over-rated and even over-used, but I would never say
that because anyone who does is clearly deranged or evil or both), but
group-think is not the sole or necessary alternative and group-think itself
has been unfairly tarred by bad group think. I have worked with groups who
thought collectively and well, whose group thinking was creative, on target,
and useful. It all depends on the group, their thinking abilities, and how
they group their thoughts.

 

Or continue with the next false dichotomy of valid data versus expert
opinion. It ignores the issue of who is the expert in question and what is
the basis of the expert's opinion. Experts become experts by virtue of much
experience, learning, and reflection. In many cases, expert opinion IS
validated data that has been distilled through years of practice and
understanding and has been embodied in a person or persons.

 

I have worked in spaces in which the organization, political, practical, and
fiscal constraints have made field research essentially impossible and
validated data has been unavailable. This has not prevented us from going
forward and designing good products, even outstanding successes. Field
research and validation are always desirable but the simplistic implication
that these are the only acceptable bases for design or design decisions is
simply not true. The quality of the group thinking and the quality of the
experts and their opinion are just as important as the quality of the user
experience data and the research that generated it. Here in Madeira, we just
wrapped up a week-long workshop with Donald Norman as Invited Scholar for
the Madeira Usability and Software Encounters. Frankly, I would bet on his
expert opinion almost any day over the sort of "validated data" that I have
seen so many groups produce and rely on.

 

Equally important is how all the expert opinion, validated data, and group
thought are used to inform the design process. Looking closely at actual
practice, you find that the norm is to gather enormous quantities of
information, distill it down into a series of maps or affinity clusterings
of one sort or another offering a variety of complex and largely unconnected
views, and then pretty much ignore most of what is on the many flipcharts
and post-it notes. The data gathering and distillation are often fun and
engaging exercises yielding walls covered with pictures and diagrams and
narratives, after which the magic begins, largely disconnected from the
masses of "validated data."

 

For my part, however, I have learned, painfully, it is best not to question
certain enshrined truths and techniques regarding user research and user
testing or experts and data, so I won't.

 

--Larry Constantine, IDSA, ACM Fellow

  Director, Lab:USE Laboratory for Usage-centered Software Engineering
(www.labuse.org)

  Professor, Department of Mathematics & Engineering

  University of Madeira | Funchal, Portugal

[Todd Zaki Warfel]

On Jul 12, 2009, at 7:37 AM, Larry Constantine wrote:

> For instance, quality of data and ease of data collection need not  
> be in opposition.

I took Jared's dichotomy list as "strive for A instead of B." While  
quality data may not always be difficult, the focus should be on  
obtaining quality data regardless of how easy or difficult it is to  
get. The goal isn't "just get any data quickly." The goal is "get the  
highest quality data we can w/in the constraints (e.g. time, money,  
resources) we have."

And frankly, in corporate environments where attitudes are often "I  
need to do something to validate my job", the focus is on doing  
something/anything, rather than something of quality. This is one of  
the primary contributors to products and services ladened with  
unnecessary and low-value features that have little if any benefit  
whatsoever.




Cheers!

Todd Zaki Warfel
Principal Design Researcher
Messagefirst | Designing Information. Beautifully.
----------------------------------
Contact Info
Voice: [phone]
Email: todd@...
AIM: twarfel@...
Blog: http://toddwarfel.com
Twitter: zakiwarfel
----------------------------------
In theory, theory and practice are the same.
In practice, they are not.

[Todd Zaki Warfel]

On Jul 12, 2009, at 7:37 AM, Larry Constantine wrote:

> I would never presume to suggest that user experience research is  
> bad (although it is over-rated and even over-used, but I would never  
> say that because anyone who does is clearly deranged or evil or both)
> [...]

This clearly shows you have a misconception of what user experience  
research is. Unfortunately, there's a lot of bad research going on in  
the UX field by people who are misinformed and untrained. The fact of  
the matter is quite the opposite of what you propose—user experience,  
which I prefer to refer to as design research, is not only not over-
rated, but in fact typically under-used.

I am curious why you think it is over-used (not to be an instigator,  
but really am curious). In my experience working w/user experience  
practitioners, business people, and software developers I frequently  
hear "we don't have time for user research." This is bullocks. There's  
always time. You just have to know what method to use given your  
constraints.

In the example of Don Norman, yes, some people like Don Norman,  
Jonathan Ives, Jared Spool and others have been doing this so long  
that their expert input is quite valid and can serve as A data point.  
But I'm sure if you speak to Jared (I use him as I know him  
personally) he'll tell you that he's continually learning things as  
times are changing. And frankly, if Don Norman and Jonathan Ives don't  
think they're continually learning, then I wouldn't trust their  
"expert opinion." SMEs are only one data point. Relying solely on them  
is dangerous and rarely produces innovation.

BTW, Apple's innovation comes from design exploration and not  
listening to "the experts." So, I would consider Ives an expert who  
doesn't listen to "expert opinion."

Finally, you state that "I have worked in spaces in which the  
organization, political, practical, and fiscal constraints have made  
field research essentially impossible and validated data has been  
unavailable. This has not prevented us from going forward and  
designing good products, even outstanding successes."

First, how do you determine good products and success? If you don't  
have some type of base research to determine what it is suppose to be  
(based on user need, not some bogus marketing requirements document),  
what do you measure against to determine you've met your goal?  
Further, if it was outstandingly successful, imagine how  
astronomically successful it would have been if it had been informed  
by some good design research.


Cheers!

Todd Zaki Warfel
Principal Design Researcher
Messagefirst | Designing Information. Beautifully.
----------------------------------
Contact Info
Voice: [phone]
Email: todd@...
AIM: twarfel@...
Blog: http://toddwarfel.com
Twitter: zakiwarfel
----------------------------------
In theory, theory and practice are the same.
In practice, they are not.

[John Schrag-3]

Larry, I'm sorry that you didn't read the original article that  
started this discussion.  It would have saved you some time.

The list of "dichotomies" came first, from the article, and Jared  
later offered his simplification as a way of encapsulating everything  
on the existing list.

Plus, they article makes it very clear that these are not dichotomies  
(true or false).  Everything on the list is of value;  the list  
reflects how our team resolves differences in cases of conflict.  For  
example, I value expert opinion very highly --- but at times when the  
facts on the ground contradict it, I'm going with the facts.  Not  
every team does this.

john


Larry Constantine wrote:

> Jared introduced into this thread the simplification:
>
> "Well-researched user experiences over group-think-produced feature  
> requirements.
>
> Later the list was expanded with other dichotomous distinctions
>
>   ...
>
> As a licensed troublemaker and certified outlier, I am bothered by  
> the continued virulence of this sort of “versus virus” that infects  
> us so frequently. Even valid or useful dichotomies are constraining,  
> limiting the purview to a single dimension or distinction when the  
> world is, thankfully, ever so much messier and interesting than  
> that. And false dichotomies, such as most of those listed, are for  
> more insidious in the way they can poison thought and practice.

[Jared Spool]

On Jul 12, 2009, at 7:37 AM, Larry Constantine wrote:

> Jared introduced into this thread the simplification:
>
>
>
> "Well-researched user experiences over group-think-produced feature  
> requirements."
>
>
>

And, for the record, I wasn't stating that I agreed with this  
statement. I was just checking to see if I understood what John had  
originally tried to say in his blog post.

Jared

[Larry Constantine]

I did indeed write:

> I would never presume to suggest that user experience research is bad
(although it is over-rated and even over-used, but I would never say that
because anyone who does is clearly deranged or evil or both) <

And Todd Warfel responded:

> This clearly shows you have a misconception of what user experience
research is. <

How quickly it becomes ad hominem. :-(

> Unfortunately, there's a lot of bad research going on in the UX field by
people who are misinformed and untrained. <

Of course. It's Sturgeon's Law.

> The fact of the matter is quite the opposite of what you propose-user
experience, which I prefer to refer to as design research, is not only not
over-rated, but in fact typically under-used. <

Well, I made no proposal, so I am not sure what is the opposite or how to
argue against "the fact of the matter" or what is true "in fact." Nor can
either of us tell whether what you refer to as design research is what I or
anyone else refers to as user experience research. So whether anyone has
misconception will have to remain undetermined.

Your reaction to my indirect reference and tongue-in-cheek parenthetical
disclaimer is precisely why I "would never presume" and "would never say"
those things. The reactions to any challenging of the shibboleths or
questioning of the received truths is met with immediate and even sometimes
violent reaction.

User experience research (or design research, if you prefer), is an absolute
and unmitigated good (if done well by good people), so no one should ever
suggest there might be disadvantages. So I won't do that.

Todd Warfel continued:

> I am curious why you think it is over-used (not to be an instigator, but
really am curious). In my experience working w/user experience
practitioners, business people, and software developers I frequently hear
"we don't have time for user research." This is bullocks. There's always
time. You just have to know what method to use given your constraints. <

I also hear this all the time. And, I agree, often it is just an excuse.
However, it is not true that there is always time and only a matter of
choosing the right method. The real issue in situations of extreme time
constraints is always how best to use your time, and sometimes the answer is
not to do any user research, but to use your extremely limited time for
something else that, given that limitation, will yield better return on
investment. (But I am not proposing that or arguing it or even saying it,
because it would be dismissed out of hand as wrong and would brand me as a
heretic.)

Todd went on:

> In the example of Don Norman, yes, some people like Don Norman, Jonathan
Ives, Jared Spool and others have been doing this so long that their expert
input is quite valid and can serve as A data point. But I'm sure if you
speak to Jared (I use him as I know him personally) he'll tell you that he's
continually learning things as times are changing. And frankly, if Don
Norman and Jonathan Ives don't think they're continually learning, then I
wouldn't trust their "expert opinion." SMEs are only one data point. Relying
solely on them is dangerous and rarely produces innovation. <

I'm an expert, too, and learning all the time, too. Both Jared and Don are
friends and lifelong learners and strong advocates of user research, so I
would not want them to be tarred with the same brush as used on me. But, I
think it is a misdirection to try equating design research with learning.
They are unrelated. You can do scads of design research and learn nothing
from it, and you can learn from many different sources by many different
means. SMEs and design experts are not "only one data point"; a single user
or single diary or single factoid or single contextual interview is one data
point. If you don't give more credence to what your expert tells you, then
you are not using your expert well (or you might have the wrong expert or
just be someone who does not give credence to expertise, which can't be
helped). In fact, an hour with good SMEs and design experts could save you
days of field inquiry, depending on how you spend that hour and how you use
what you learn. The interesting cases are where the data and the experts are
at odds, but even there, it is not as clear cut as the evangelists would
make it out to be. In general, I would say trust the data, but I don't work
in general; I always work in specifics, and I have known a number of cases
in which the data and/or its interpretation and/or the drawn conclusions
were wrong and the expert was right.

What I see far more often than most people are prepared to admit or even
talk about is groups spending far more time on user research than they
actually needed to (for varied reasons: sometimes out of insecurity,
sometimes because it is seductive, and in a few cases because they just had
more time and money than was good for them) and then becoming hostage to
"the data," allowing it to override their own intuition, experience, and
even common sense or allowing it to dictate designs that are less than they
are capable of. I am not saying this is the norm or the majority of cases,
but it is a real phenomenon. What I find intriguing is that the so many
people will not even consider that user research could be over done or that
it might have a downside as well as an upside. Which is one reason that I
would never dare say anything about these issues in a public forum, least of
all here.

And Todd added:

> BTW, Apple's innovation comes from design exploration and not listening to
"the experts." So, I would consider Ives an expert who doesn't listen to
"expert opinion." <

I don't know Ives, so I have no idea to what extent he does or does not
listen to expert opinion and I would guess neither do you. As to Apple's
successful design innovation, it is rooted in many contributing factors and
I would not reduce it to such a simple notion.

And Todd finished by quoting me and then parrying:

> Finally, you state that "I have worked in spaces in which the
organization, political, practical, and fiscal constraints have made field
research essentially impossible and validated data has been unavailable.
This has not prevented us from going forward and designing good products,
even outstanding successes."

First, how do you determine good products and success? If you don't have
some type of base research to determine what it is suppose to be (based on
user need, not some bogus marketing requirements document), what do you
measure against to determine you've met your goal? Further, if it was
outstandingly successful, imagine how astronomically successful it would
have been if it had been informed by some good design research. <

There are many measures of success, many of which depend not an iota on
baseline research. Microsoft software is widely criticized, not the least by
Apple advocates, but it is clearly a gigantic market success. Apple's
computers are worshipped by a community of true believers, yet it is in one
sense a market failure at a paltry 4%. (For the record, I think both
companies do good things and stupid things and I have used them both, so
please don't turn this into another Mac versus Windows sideshow.)

I was going to not rise to the bait, because the structure of your inquiry
suggests to me that no matter what criteria I offer as measures of success,
they will only be dismissed. But, I'll naively proceed anyway. In one case I
have in mind, with extremely limited user research (many would not even
consider what we did as user research), we produced an award-winning product
that was dramatically easier to use than previous versions, that beat the
competitors hands down, that was a commercial success and became a paradigm
for future software from the company. In another case, with limited up front
user research and way too modest on-going input, we produced a product that
cut training time by a factor 15 and increased productivity by 50 some
percent. Users absolutely loved it and it went on to become the most
successful and profitable software product in the company's history. Both of
these examples have at least one thing in common: ongoing access to in-depth
subject-matter expertise that was viewed through the lens of a systematic
model-driven approach to interaction design.

As to your last sentence, I have no reason to believe that so much as ten
minutes or ten weeks more of user research would have resulted in a
dramatically better product in the cases I know within the constraints of
time. It is always about tradeoffs: every minute you focus on one thing is a
minute you are not focusing on something else. All I am saying is that
sometimes it is a better use of time to do less user research and more of
something else. I have my own views based on experience of what I might do
instead, but I would not expect you to agree with me or maybe even believe
me. Most projects I have ever worked on as a designer have had too little
time and too little money, and I am not saying a little too little, I am
saying whopping too little. In every case, if we had had more time or more
money, we probably would have done more field work, but we also would have
done more of other things, and the split would not necessarily be down the
middle.

--Larry Constantine, IDSA, ACM Fellow
  Director, Lab:USE Laboratory for Usage-centered Software Engineering
(www.labuse.org)
  Professor, Department of Mathematics & Engineering
  University of Madeira | Funchal, Portugal
  Chief Scientist, Constantine & Lockwood Ltd

[Jared Spool]

On Jul 12, 2009, at 7:29 PM, Larry Constantine wrote:

> Both Jared and Don are
> friends and lifelong learners and strong advocates of user research,  
> so I
> would not want them to be tarred with the same brush as used on me.  
> But, I
> think it is a misdirection to try equating design research with  
> learning.

Oh, go ahead and tar me. I haven't had a good tarring in a really long  
time.

:)

Jared

[Larry Constantine]

John Shrag wrote:

> Larry, I'm sorry that you didn't read the original article that started
this discussion.  It would have saved you some time. <

I'm sorry, too, but then I don't always have the time, opportunity, or
bandwidth to read everything I should or could or would like to.

And John continued:

> The list of "dichotomies" came first, from the article, and Jared later
offered his simplification as a way of encapsulating everything on the
existing list.

Plus, they article makes it very clear that these are not dichotomies (true
or false).  Everything on the list is of value;  the list reflects how our
team resolves differences in cases of conflict.  For example, I value expert
opinion very highly --- but at times when the facts on the ground contradict
it, I'm going with the facts. <

They are, in both cases, dichotomies, not "dichotomies" or for that matter
"true or false," but this is getting into semantic quibbles that are not
terribly important or interesting to me. The punch-line is in your closing
sentence. In another post I said "In general, I would say trust the data,
but..." The big "but" is that user research is NOT in practice always right
or valid, even when validated, any more than expert opinion is always right
or valid, even when based on extensive experience. In truth, when so-called
facts and so-called opinion contradict, sometimes the opinion is right. What
I try to teach my students and counsel my clients is that user research
should never become a substitute for thinking, that it CAN be misleading or
misunderstood, and that scads of user research can become a cover behind
which to hide a lack of conviction or confidence. I have seen people do
things that they know in their heart of hearts is wrong but they are afraid
to contradict "the data." Basing work--or at least maintaining the
appearance of basing work--on validated user research data is the safe and
secure route to take. Whatever is delivered can be justified and defended on
the basis of the extensive data that was carefully gathered and extensively
analyzed and thoroughly checked. So clearly, extensive user experience
research, thoroughly validated, is THE way and the ONLY way to do good
design, and anyone who suggests otherwise is demented.

In abject and fully acknowledged cowardice I have glossed over the
unpleasant possibility that most user experience research might be only
remotely related to anything deserving of the term "facts on the ground,"
but that is material for another post that I may or may not ever dare to
make.

--Larry Constantine, IDSA, ACM Fellow
  Director, Lab:USE Laboratory for Usage-centered Software Engineering
(www.labuse.org)
  Professor, Department of Mathematics & Engineering
  University of Madeira | Funchal, Portugal
  Chief Scientist, Constantine & Lockwood Ltd

[Larry Constantine]

Thanks and good for you Jared.

 

--Larry Constantine, IDSA, ACM Fellow

  Director, Lab:USE Laboratory for Usage-centered Software Engineering
(www.labuse.org)

  Professor, Department of Mathematics & Engineering

  University of Madeira | Funchal, Portugal

  Chief Scientist, Constantine & Lockwood Ltd

 

  _____  

From: agile-usability@...
[mailto:agile-usability@...] On Behalf Of Jared Spool
Sent: Sunday, July 12, 2009 10:10 AM
To: agile-usability@...
Subject: Re: [agile-usability] Design Values

 





 

On Jul 12, 2009, at 7:37 AM, Larry Constantine wrote:

 

Jared introduced into this thread the simplification:

 

"Well-researched user experiences over group-think-produced feature
requirements."

 

And, for the record, I wasn't stating that I agreed with this statement. I
was just checking to see if I understood what John had originally tried to
say in his blog post.

 

Jared

[Ron Jeffries-3]

Hello, Todd.  On Sunday, July 12, 2009, at 7:32:41 AM, you wrote:

> On Jul 12, 2009, at 7:37 AM, Larry Constantine wrote:

>> I would never presume to suggest that user experience research is  
>> bad (although it is over-rated and even over-used, but I would never  
>> say that because anyone who does is clearly deranged or evil or both)
>> [...]

> This clearly shows you have a misconception of what user experience
> research is.

It seems to me very unlikely that Larry Constantine is under a
misconception about what user experience research is. I'm submitting
a defect report on you for this one.

Ron Jeffries
www.XProgramming.com
www.xprogramming.com/blog
To be on the wire is life. The rest is waiting.  --Karl Wallenda



------------------------------------

Yahoo! Groups Links

<*> To visit your group on the web, go to:
    http://groups.yahoo.com/group/agile-usability/

<*> Your email settings:
    Individual Email | Traditional

<*> To change settings online go to:
    http://groups.yahoo.com/group/agile-usability/join
    (Yahoo! ID required)

<*> To change settings via email:
    mailto:agile-usability-digest@...
    mailto:agile-usability-fullfeatured@...

<*> To unsubscribe from this group, send an email to:
    agile-usability-unsubscribe@...

<*> Your use of Yahoo! Groups is subject to:
    http://docs.yahoo.com/info/terms/

[Todd Zaki Warfel]

On Jul 12, 2009, at 8:02 PM, Ron Jeffries wrote:

> It seems to me very unlikely that Larry Constantine is under a  
> misconception about what user experience research is. I'm submitting  
> a defect report on you for this one.

Herein lies one of the reasons that Agile has such a bad name in the  
UX community—the expressed attitude that user research (what I refer  
to as design) research provides little if any value. That perspective  
is in fact deeply flawed.

User research or design research, when conducted correctly, provides  
considerably more value than the time and effort it costs. Design  
research can come in a variety of flavors. One of the most valuable is  
contextual research (ethno-graphic based interviews). Go watch people  
and talk to them in their native environments. You'll be amazed at  
what you'll learn by talking to a human vs. looking at weblogs, or  
survey data (two other common design research methods).

Yes, SMEs can be used as part of the data collection process, but  
they're only one data point in the process.

One of the reasons I got involved with the agile community in the  
first place was to try and show UX practitioners that agile methods  
can help their UX practice and to show agile practitioners the value  
of user/design research.

The self-stated fact that Larry cannot imagine user research  
contributing to making the cited projects any better shows that his  
perspective is flawed from the outset.

I agree with Larry that you have to determine the best use of your  
resources given your constraints. And I'm not advocating 6 month long  
investigations. I'm talking about conducting in some cases guerilla UX  
research that will not only inform your design, but give you ammo to  
fend off those derailing stakeholder discussions that steer toward  
scope creep and missed deadlines.

Further, over reliance on SMEs is just as dangerous as doing no  
research or doing too much. They're an important data point, but I  
wouldn't be my farm on them.

Finally, while I don't know Ives personally, I interpret Apple's not  
following SMEs as being something that's evident by their design model—
they ignore analysts, industry experts, and what others are doing.  
They march to their own drum often in direct opposition to SMEs.


Cheers!

Todd Zaki Warfel
Principal Design Researcher
Messagefirst | Designing Information. Beautifully.
----------------------------------
Contact Info
Voice: [phone]
Email: todd@...
AIM: twarfel@...
Blog: http://toddwarfel.com
Twitter: zakiwarfel
----------------------------------
In theory, theory and practice are the same.
In practice, they are not.

[George Dinwiddie]

Todd Zaki Warfel wrote:

> On Jul 12, 2009, at 8:02 PM, Ron Jeffries wrote:
>
>> It seems to me very unlikely that Larry Constantine is under
>> a misconception about what user experience research is. I'm
>> submitting a defect report on you for this one.
>
> Herein lies one of the reasons that Agile has such a bad name in the UX
> community—the expressed attitude that user research (what I refer to as
> design) research provides little if any value. That perspective is in
> fact deeply flawed.

Are you equating Larry Constantine with the Agile community?  That
really is quite funny.

Todd, I get the feeling that you know nothing about Larry's writings on
usability.  Your responses to and about him are making little sense to me.

  - George

--
  ----------------------------------------------------------------------
   * George Dinwiddie *                      http://blog.gdinwiddie.com
   Software Development                    http://www.idiacomputing.com
   Consultant and Coach                    http://www.agilemaryland.org
  ----------------------------------------------------------------------



------------------------------------

Yahoo! Groups Links

<*> To visit your group on the web, go to:
    http://groups.yahoo.com/group/agile-usability/

<*> Your email settings:
    Individual Email | Traditional

<*> To change settings online go to:
    http://groups.yahoo.com/group/agile-usability/join
    (Yahoo! ID required)

<*> To change settings via email:
    mailto:agile-usability-digest@...
    mailto:agile-usability-fullfeatured@...

<*> To unsubscribe from this group, send an email to:
    agile-usability-unsubscribe@...

<*> Your use of Yahoo! Groups is subject to:
    http://docs.yahoo.com/info/terms/

[Todd Zaki Warfel]

On Jul 12, 2009, at 10:02 PM, George Dinwiddie wrote:

> Are you equating Larry Constantine with the Agile community?  That  
> really is quite funny.
>
> Todd, I get the feeling that you know nothing about Larry's writings  
> on usability.  Your responses to and about him are making little  
> sense to me.

I'm equating Larry's expressed perspective with an expressed  
perspective from the agile community and one of the reasons so many UX  
practitioners snuff off agile. It's a fundamentally flawed perspective.

And yes, I'm familiar with Larry's background and writings on usability.


Cheers!

Todd Zaki Warfel
Principal Design Researcher
Messagefirst | Designing Information. Beautifully.
----------------------------------
Contact Info
Voice: [phone]
Email: todd@...
AIM: twarfel@...
Blog: http://toddwarfel.com
Twitter: zakiwarfel
----------------------------------
In theory, theory and practice are the same.
In practice, they are not.

[Larry Constantine]

Todd,

I don't know where to begin. No one here in this forum, and certainly not
me, has ever implied (as far as I can recall) that user research provides
little if any value. The fact that you keep reading into and exaggerating
what is in fact a nuanced position that I have tried to express with some
precision (leavened with some humor, too) is exactly the problem that my
peppering of flippancy addresses. The perspective you attribute to the agile
community is, I would agree, deeply flawed, but it is not one I hear from
the agilistas.

> The self-stated fact that Larry cannot imagine user research contributing
to making the cited projects any better shows that his perspective is flawed
from the outset. <

In the interest of ongoing civility in this forum I will only repeat that
you completely and repeatedly misrepresent my perspective. On the assumption
that you misrepresent it because you don't understand it rather than as a
matter of choice, let me say it one more time in simple statements: (1) user
research is generally a good idea; (2) it is not infinitely good; (3) it has
a downside as well as an upside; (4) there always tradeoffs and choices, and
sometimes the best way to spend scarce resources is on something other than
user research.

Going back to your earlier prod:

> imagine how astronomically successful it would
have been if it had been informed by some good design research <

My exact response, again, was:

> I have no reason to believe that so much as ten
minutes or ten weeks more of user research would have resulted in a
dramatically better product in the cases I know within the constraints of
time. <

The operant clauses there are: "have no reason to believe" not "can't
imagine"; "resulted in a dramatically better" not "contributing to"; and
"within the constraints of time."

Indeed, if you go back and read my original post, you'll see that in both
cases we did do some "user experience research," but not too much. (It was
of course, "good" user research; I wouldn't have it any other way. :-) You
do not have to sell me on face-to-face interaction with real people who
might be or become users, which I always do.

If you truly want to help the agile community appreciate UX and the UX
community benefit from agile thinking, then I would suggest you address what
members of these communities are actually saying rather than exaggerated
attributions that are easily dismissed but also not representative.

As for me, I am not really a member of either community, although I work and
teach and write in both spheres. The fact that I am an equal opportunity
detractor and have found fault with some of the assumptions and practices on
both sides may explain why no one will claim me. ;-)

Yours in nuanced distinctions,

--Larry Constantine, IDSA, ACM Fellow
  Director, Lab:USE Laboratory for Usage-centered Software Engineering
(www.labuse.org)
  Professor, Department of Mathematics & Engineering
  University of Madeira | Funchal, Portugal
  Chief Scientist, Constantine & Lockwood Ltd