How competent must people be to use the Web?

[Morten Tollefsen]

Hi!

We have a small Norwegian pre project, and I hope it's OK to send info. to this list. Please do not hesitate to forward the message or contact me.

Experience and testing are necessary to create user-friendly Web sites. WCAG is a set of guidelines to ensure that the Web can also be used by the disabled. User competence is however required - even when sites are accessible and usable. MediaLT would like to define user competence, and we are very interested
to hear from others working on similar approaches to this challenge.

MediaLT has been working with user testing of Web sites for a number of years. Despite focus on Universal Design when Web sites are created, in our experience many users struggle to use these Web sites. To give good advice during the development process and evaluation of existing sites, we need to define necessary
user skills. This is also desirable in other research projects where a more precise definition of competency requirements would be useful for the implementation of user testing and / or participatory design. With the support of several Norwegian research and user organizations we have started a pilot project with the following main objective:

To define necessary user competence to use WCAG 2.0 Web sites and to develop a framework for training and testing required skills.

There are various certification schemes and competency requirements for mastery of the PC. The most well known in Europe is the E/ICDL (European/International computer driving license) Such a standard is good, but does not entirely define the necessary competency requirements for all users. The EDCL includes requirements which are not
a prerequisite for using the Web in an appropriate manner. For example in the curriculum (version 4.0): "7.1.1.2 Define and understand the terms: HTTP, URL, hyperlink, ISP, FTP." It is of course useful to understand these concepts, but they are not a prerequisite for using the Web. On the other hand, some
groups of users must master functionality that is not described in a standard curriculum. Blind users must for example be familiar with how the screen reader can move between headings and display a link list.

Our intention is to offer training courses relating to the defined competency requirements. It will moreover be possible to arrange tests to ensure that users participating in the evaluation of Web sites have the necessary skills.

We do not aim to cover all assumptions and requirements in this pilot project. Two main groups have been selected: severely visually impaired users and those with cognitive disabilities.

Online article (more or less the same as this mail):
http://medialt.no/news/en-US/how-competent-must-people-be-to-use-the-web/634.aspx

Project web pages (in English, but with some Norwegian words - sorry for this):
http://medialt.no/universal-user-competence-uuc/733.aspx


Best regards,

Morten Tollefsen
MediaLT: www.medialt.no
Phone: [phone]
Mobile: [phone]
Address: Jerikoveien 22, N-1067 Oslo, Norway
Twitter: www.twitter.com/mortentollefsen
MSN: mortentollefsen@...

[David Woolley (E.L)]

Morten Tollefsen wrote:
>
> To define necessary user competence to use WCAG 2.0 Web sites and to
* develop a framework for training and testing required skills.

I don't see that this is a useful skill level, as it doesn't include the
skills in recognizing abstractions, like links, in the heavily disguised
forms in which they often occur in real world web sites.  In my view, it
is the need for many years of experience doing this, and a lack of
practice in using the constructs in easily recognizable forms, that are
the biggest blockers for elderly users.

(For elderly learners, if you can't summarise the rules on a single
sheet of paper, the design is too complex.  I would venture to suggest
that, if the WCAG guidelines have been drafted properly, a conversion
course from basic use of Windows Notepad (including printing) to using
the WCAG 2.0 web sites should fit on one page, with a summary of about
one quarter of a page.)

>
> There are various certification schemes and competency requirements
* for mastery of the PC. The most well known in Europe is the E/ICDL
* (European/International computer driving license) Such a standard is

 From what I've seen of ECDL book in the book shops, ECDL aims rather
higher than passive use, and at how to write your own vanity web pages.
In the examples I saw, it was actually doing accessibility a
dis-service, as it was teaching people how to produce visual results on
IE, not how to produce proper HTML.


--
David Woolley
Emails are not formal business letters, whatever businesses may want.
RFC1855 says there should be an address here, but, in a world of spam,
that is no longer good advice, as archive address hiding may not work.

[David Woolley (E.L)]

David Woolley wrote:
> that, if the WCAG guidelines have been drafted properly, a conversion
> course from basic use of Windows Notepad (including printing) to using
> the WCAG 2.0 web sites should fit on one page, with a summary of about

Where some tuition may be needed is in adapting what one has learnt on
Windows to kiosk systems, which often use track balls and touch screens.

--
David Woolley
Emails are not formal business letters, whatever businesses may want.
RFC1855 says there should be an address here, but, in a world of spam,
that is no longer good advice, as archive address hiding may not work.

[Michael S Elledge]

Hi Morten--

It seems to me that the question isn't how competent your users are, but
how usable (and accessible) a website is for them. If participants in
user testing reflect your target audience, they will naturally have a
representative set of skill levels. In most cases that will include
novices as well as more expert users of adaptive technology. We collect
information about how often people use the Internet, the AT they use,
and (less often) how they would characterize their ability. Once testing
is completed, it is fairly easy to characterize them as novices,
intermediate or expert based on how much difficulty they have with a
site or application and the techniques they use.

Perhaps I've missed your point, but I think the key is not to determine
which level of competence is necessary for people to use websites built
to WCAG 2.0 guidelines, but to identify which implementations of those
guidelines work best for persons with disabilities.

Mike Elledge
Assistant Director
Usability & Accessibility Center
Michigan State University

David Woolley wrote:

> Morten Tollefsen wrote:
>>
>> To define necessary user competence to use WCAG 2.0 Web sites and to
> * develop a framework for training and testing required skills.
>
> I don't see that this is a useful skill level, as it doesn't include
> the skills in recognizing abstractions, like links, in the heavily
> disguised forms in which they often occur in real world web sites.  In
> my view, it is the need for many years of experience doing this, and a
> lack of practice in using the constructs in easily recognizable forms,
> that are the biggest blockers for elderly users.
>
> (For elderly learners, if you can't summarise the rules on a single
> sheet of paper, the design is too complex.  I would venture to suggest
> that, if the WCAG guidelines have been drafted properly, a conversion
> course from basic use of Windows Notepad (including printing) to using
> the WCAG 2.0 web sites should fit on one page, with a summary of about
> one quarter of a page.)
>
>>
>> There are various certification schemes and competency requirements
> * for mastery of the PC. The most well known in Europe is the E/ICDL
> * (European/International computer driving license) Such a standard is
>
> From what I've seen of ECDL book in the book shops, ECDL aims rather
> higher than passive use, and at how to write your own vanity web
> pages. In the examples I saw, it was actually doing accessibility a
> dis-service, as it was teaching people how to produce visual results
> on IE, not how to produce proper HTML.
>
>

[David Woolley (E.L)]

Michael S Elledge wrote:

> representative set of skill levels. In most cases that will include
> novices as well as more expert users of adaptive technology. We collect

I don't think he was particularly talking about adaptive technology
users.  I think he was talking about people who don't belong to the
computer using generation, and maybe also young people with cognitive
disabilities.

As well as physical disabilities age, in particular, brings a difficulty
in learning new technology, a fear of breaking things, and a lack of
prior experience of that technology.

>
> Perhaps I've missed your point, but I think the key is not to determine
> which level of competence is necessary for people to use websites built
> to WCAG 2.0 guidelines, but to identify which implementations of those
> guidelines work best for persons with disabilities.

I'd agree that, for web sites designed for accessibility, the need for
education is a negative quality indicator.


--
David Woolley
Emails are not formal business letters, whatever businesses may want.
RFC1855 says there should be an address here, but, in a world of spam,
that is no longer good advice, as archive address hiding may not work.

[Morten Tollefsen]

Hi!

Thanks for several comments (both to me directly and on the list).

Communication is art. I'm not very good in English, and some of the comments I've got are not understandable to me. What I'm writing is probably both inperfect and not clear enough, sorry for this!

When testing sites it is of course good practice to include users with various background. This means different computer skills, disabilities, ... However even if this is the case my experience is that it is very often important to be able to have users with sertain skills. And when training users it is also good to have a goal for the courses.

ECDL (European computer driving licence) is one standard user level typically used in Norway. We use the same education for visually impaired users of course. The level Is the same, but not methods: e. g. blind users need to know some screen reader functionality even if web sites are accessible (wcag 2.0 compatible). ECDL has a syllabus typically meant to be used in the labor market, but I think the Internet module is suitable also for modern private web browsing. The tests are measuring both the ability to perform different tasks (e. g. to use a search engine) and the time used. Here a blind user typically need to know screen reader functionality to work fast enough: link list, search using virtual cursor, navigate by headings, jump to the first form field, ...

I'll try to give an example: A page with a form which is coded according to wcag 2.0 (<labels> are used etc. etc.). A Jaws user needs to know how to use Form mode to be able to fill in text fields. Exactly these things are what we'll try to state in our "needed skills document".

Another example: A table with <TH>-tags etc. A VoiceOver user will not be able to read the table if hedo not know the keyboard command to move into the table.

A third example: In Window Eyes you need a specific shortcut key to perform an internal page search. E. g. in Internet Explorer you'll not use the standard Explorer-shortcut for this.

In other words: even if pages/sites are accessible blind users (and other users typically not using mouse/screen) need some "specific" skills to work efficiently.

The reason for focusing on wcag 2.0 compatible sites in our project is to avoid accessibility barriers. Experienced blind users will also be able to use several poorly designed pages, but our goal is to figure out exactly what's needed when web content is accessible (by definition:-)).

I have used the web myselves from the very beginning. I did design one of the first sites in Norway (for the research department in Norwegian Telecom). As a blind researcher I'm trying to help disabled users of course (some of the comments I've got are a bit aggressive). Therefore I have some ideas about what is most important, but this has to be documented. We have some knowledge, e. g. the Webaim screen reader survey, but for me it is very desirable to get feedback from other experts, users and developers (or others of course)!

- Morten


-----Opprinnelig melding-----
Fra: w3c-wai-ig-request@... [mailto:w3c-wai-ig-request@...] På vegne av David Woolley
Sendt: 3. november 2009 22:30
Til: w3c-wai-ig@...
Emne: Re: How competent must people be to use the Web?

Michael S Elledge wrote:

> representative set of skill levels. In most cases that will include
> novices as well as more expert users of adaptive technology. We collect

I don't think he was particularly talking about adaptive technology
users.  I think he was talking about people who don't belong to the
computer using generation, and maybe also young people with cognitive
disabilities.

As well as physical disabilities age, in particular, brings a difficulty
in learning new technology, a fear of breaking things, and a lack of
prior experience of that technology.

>
> Perhaps I've missed your point, but I think the key is not to determine
> which level of competence is necessary for people to use websites built
> to WCAG 2.0 guidelines, but to identify which implementations of those
> guidelines work best for persons with disabilities.

I'd agree that, for web sites designed for accessibility, the need for
education is a negative quality indicator.


--
David Woolley
Emails are not formal business letters, whatever businesses may want.
RFC1855 says there should be an address here, but, in a world of spam,
that is no longer good advice, as archive address hiding may not work.

[Michael S Elledge]

Hi Morten--

I meant to follow-up on this but forgot. Your English is great--don't
worry about it. We've found that user knowledge of screen reader
functionality varies a great deal. Some people only tab and arrow
through an application; others use shortcut keys like h, b, p, and f;
know to turn off the virtual cursor when they encounter poorly
performing widgets; identify data cells from row and column headings, etc.

We haven't encountered the problems you mentioned (we work primarily
with JAWS users and so far haven't had users who don't know about forms
mode). But if there are basic "show-stoppers" you could either tell
testers the correct keystrokes if they get stuck, or pre-screen them to
ensure they will know what to do.

Even WCAG 2.0-level functionality is supposed to be basic to screen
reader users; for example, widgets should be functional using the tab,
arrow, space and enter keys. If they do not follow the typical
methodology (tab into a widget, move within widget using arrow keys, set
focus using space, move using control + arrow keys and activate using
enter, for example--I think this is pretty established), there should be
an explanation.

If you truly want to establish a base level of competence, you may want
to look at the keystrokes recommended for various widgets in the ARIA
Best Practices document, and identify the ones that are most prevalent.
You could then provide that information ahead of time for people using
an application if they are unfamiliar with interactivity.

Hope that helps.

Mike

Morten Tollefsen wrote:

> Hi!
>
> Thanks for several comments (both to me directly and on the list).
>
> Communication is art. I'm not very good in English, and some of the comments I've got are not understandable to me. What I'm writing is probably both inperfect and not clear enough, sorry for this!
>
> When testing sites it is of course good practice to include users with various background. This means different computer skills, disabilities, ... However even if this is the case my experience is that it is very often important to be able to have users with sertain skills. And when training users it is also good to have a goal for the courses.
>
> ECDL (European computer driving licence) is one standard user level typically used in Norway. We use the same education for visually impaired users of course. The level Is the same, but not methods: e. g. blind users need to know some screen reader functionality even if web sites are accessible (wcag 2.0 compatible). ECDL has a syllabus typically meant to be used in the labor market, but I think the Internet module is suitable also for modern private web browsing. The tests are measuring both the ability to perform different tasks (e. g. to use a search engine) and the time used. Here a blind user typically need to know screen reader functionality to work fast enough: link list, search using virtual cursor, navigate by headings, jump to the first form field, ...
>
> I'll try to give an example: A page with a form which is coded according to wcag 2.0 (<labels> are used etc. etc.). A Jaws user needs to know how to use Form mode to be able to fill in text fields. Exactly these things are what we'll try to state in our "needed skills document".
>
> Another example: A table with <TH>-tags etc. A VoiceOver user will not be able to read the table if hedo not know the keyboard command to move into the table.
>
> A third example: In Window Eyes you need a specific shortcut key to perform an internal page search. E. g. in Internet Explorer you'll not use the standard Explorer-shortcut for this.
>
> In other words: even if pages/sites are accessible blind users (and other users typically not using mouse/screen) need some "specific" skills to work efficiently.
>
> The reason for focusing on wcag 2.0 compatible sites in our project is to avoid accessibility barriers. Experienced blind users will also be able to use several poorly designed pages, but our goal is to figure out exactly what's needed when web content is accessible (by definition:-)).
>
> I have used the web myselves from the very beginning. I did design one of the first sites in Norway (for the research department in Norwegian Telecom). As a blind researcher I'm trying to help disabled users of course (some of the comments I've got are a bit aggressive). Therefore I have some ideas about what is most important, but this has to be documented. We have some knowledge, e. g. the Webaim screen reader survey, but for me it is very desirable to get feedback from other experts, users and developers (or others of course)!
>
> - Morten
>
>
> -----Opprinnelig melding-----
> Fra: w3c-wai-ig-request@... [mailto:w3c-wai-ig-request@...] På vegne av David Woolley
> Sendt: 3. november 2009 22:30
> Til: w3c-wai-ig@...
> Emne: Re: How competent must people be to use the Web?
>
> Michael S Elledge wrote:
>
>  
>> representative set of skill levels. In most cases that will include
>> novices as well as more expert users of adaptive technology. We collect
>>    
>
> I don't think he was particularly talking about adaptive technology
> users.  I think he was talking about people who don't belong to the
> computer using generation, and maybe also young people with cognitive
> disabilities.
>
> As well as physical disabilities age, in particular, brings a difficulty
> in learning new technology, a fear of breaking things, and a lack of
> prior experience of that technology.
>
>  
>> Perhaps I've missed your point, but I think the key is not to determine
>> which level of competence is necessary for people to use websites built
>> to WCAG 2.0 guidelines, but to identify which implementations of those
>> guidelines work best for persons with disabilities.
>>    
>
> I'd agree that, for web sites designed for accessibility, the need for
> education is a negative quality indicator.
>
>
>  

[Manser, Erich W.]

Good Morning:
Please pardon the intrusion, though asking if anyone with blindness or low-vision is willing to take a brief 5-7 question survey relating to assistive technology, I would greatly value your feedback.

Please reply to me directly, and I'll forward questions in a separate email.

Thanks so much!

Erich Manser