RE: Error in (approved) test 0009

[Hausenblas, Michael]

Dan,

As always a pleasure to discuss with you :)

> I would need to sync up with Ralph Swick and/or Steven
> Pemberton before I ask you to re-do some work, but...

Steven is on holiday AFAIK, Ralph will be on the call on Thursday,
hopefully
- how about you join us at the RDFa Telecon and tell us how best to
proceed, there?

> I don't know how to approve "intention and semantics"; I only
> know how to approve sequences of bytes.

Now that is a petty - I thought we're on the Semantic Web, at last ;)

> In the GRDDL WG, sometimes the test suite editors were given
> discretion to change some bytes after a test case was approved...

Ok. But we are not in the GRDDL WG, but in the RDFa TF ...

> I was a little uncomfortable with that, but as long as it was
> stuff that was really orthogonal to GRDDL (things like
> license text in XML comments, as I recall), I let it go.

Again. 'My' chair is Ben; whatever he tells me will be fine.

> But in the case of 0009, the change is clearly substantive
> and not editorial. The TF records show a decision to approve
> a SPARQL query that said ... WHERE { X P Y }; it's quite a
> stretch to say that decision applies to a SPARQL query ...
> WHERE { Y P X }.

Yes. It is substantive. From *broken* to *what the TC should be about*.
But again - we can and should discuss this one again; even if it's
just to fulfil some droll 'rules' - BTW, where are they?

I tried hard to prepare myself for the TC (e.g. [1]),
but - and I did not want to state it this clear until your
recent comments - I have to say: regarding testing (support)
I really feel left alone within W3C. There are some activities
here (QA) and there, but NO common framework, no best practice,
nothing in the process document AFAIK - how the heck should I know :(

So, thanks again for your comments. And in case you want to *really*
change the system (and a little bird told me that you happen to be in
this position) please take care that the QA WG/IG or who ever is
responsible
writes something down. For me. For all people doing TC. For the World.
Amen.

Cheers,
        Michael

[1]
http://lists.w3.org/Archives/Public/public-rdf-in-xhtml-tf/2007Jan/0049.
html

----------------------------------------------------------
 Michael Hausenblas, MSc.
 Institute of Information Systems & Information Management
 JOANNEUM RESEARCH Forschungsgesellschaft mbH
 Steyrergasse 17, A-8010 Graz, AUSTRIA
----------------------------------------------------------

> -----Original Message-----
> From: Dan Connolly [mailto:connolly@...]
> Sent: Monday, August 06, 2007 8:13 PM
> To: Hausenblas, Michael
> Cc: Ivan Herman; W3C RDFa task force
> Subject: RE: Error in (approved) test 0009
>
> On Mon, 2007-08-06 at 19:44 +0200, Hausenblas, Michael wrote:
> > Dan,
> >
> > Sooo many question for a simple-minded person, as I am :)
> >
> > Ok. One step at a time.
> >
> > > If you changed the content of the test, the status should
> revert to
> > > unapproved, right? i.e. a new decision should be made to
> approve the
> > > new contents.
> >
> > I'm not a W3C-Test-Professional, but down-to-earth: When we simply
> > *fix* a broken TC -- which intention and semantics has been
> approved
> > -- we do not actually *change* but (ehm what was the word
> again? ah!)
> > *fix* it ;)
> >
> > So, if you prefer it, I'd happily propose to review this one again,
> > from a efficiency POV I'd keep it approved.
>
> I would need to sync up with Ralph Swick and/or Steven
> Pemberton before I ask you to re-do some work, but...
>
> I don't know how to approve "intention and semantics"; I only
> know how to approve sequences of bytes.
>
> In the GRDDL WG, sometimes the test suite editors were given
> discretion to change some bytes after a test case was approved...
> I was a little uncomfortable with that, but as long as it was
> stuff that was really orthogonal to GRDDL (things like
> license text in XML comments, as I recall), I let it go.
> But in the case of 0009, the change is clearly substantive
> and not editorial. The TF records show a decision to approve
> a SPARQL query that said ... WHERE { X P Y }; it's quite a
> stretch to say that decision applies to a SPARQL query ...
> WHERE { Y P X }.
> Perhaps you can get away with that once or twice, but if it
> happens dozens of times, the evidence that the test suite was
> reviewed and approved by the TF gets much less compelling.
>
> > > Hmm... I'm assuming this test suite is organized roughly like the
> > > RDF Core, OWL, SPARQL, and GRDDL test suites, where approval of a
> > > test is always traceable to a recorded decision. Is that the way
> > > this test suite is organized?
> >
> > Roughly, yes. We're still in a kind of learning phase - so any
> > improvement suggestions welcome :)
> >
> > However, I'm very happy you asked this; till now I assumed either
> > everybody would understand the system -- or nobody would
> really care
> > ;)
> >
> > So, here is the idea:
> >
> > Every section in [1] has a title 'Review and Approval YYYY-MM-DD'
> > with YYYY-MM-DD representing the date the review took place
> (in case
> > of a telecon, or when it was closed in case of a mailing
> list review).
>
> Ah... I had missed that... so test cases are grouped by date. Hmm...
> what if a test case is discussed multiple times? Does it show
> up in multiple places? I suppose I can see how this works.
>
> Hmm... do test cases get URI names? What's the full URI of test 0009?
>
> Ah... per the manifest, it seems to be:
>
> http://www.w3.org/2006/07/SWD/RDFa/testsuite/xhtml1-testcases/rdfa
> -xhtml1-test-manifest.rdf#Test0009
>
>
> I don't see links in the manifest to the approval decision records.
>
> You have explained quite a bit about how this works, but I'm
> still not quite sure I understand. It's possible that you
> just have to be at the teleconferences to understand the
> process. That's fine, for now, as long as it's eventually
> written up in a way that's clear enough for consumers of the tests.
>
> --
> Dan Connolly, W3C http://www.w3.org/People/Connolly/ 

[Dan Connolly]

On Mon, 2007-08-06 at 21:17 +0200, Hausenblas, Michael wrote:
[...]
> Ok. But we are not in the GRDDL WG, but in the RDFa TF ...

Very well.

[...]

> Yes. It is substantive. From *broken* to *what the TC should be about*.
> But again - we can and should discuss this one again; even if it's
> just to fulfil some droll 'rules' - BTW, where are they?
>
> I tried hard to prepare myself for the TC (e.g. [1]),
> but - and I did not want to state it this clear until your
> recent comments - I have to say: regarding testing (support)
> I really feel left alone within W3C. There are some activities
> here (QA) and there, but NO common framework, no best practice,
> nothing in the process document AFAIK - how the heck should I know :(
>
> So, thanks again for your comments. And in case you want to *really*
> change the system (and a little bird told me that you happen to be in
> this position) please take care that the QA WG/IG or who ever is
> responsible
> writes something down. For me. For all people doing TC. For the World.
> Amen.

Each group ultimately makes its own rules.

What survives the group is the tech reports:

http://www.w3.org/TR/rdf-testcases/
http://www.w3.org/TR/owl-test/
http://www.w3.org/TR/grddl-tests/
(the SPARQL test cases aren't released as a tech
report, as far as I know)

The OWL test cases doc has a fairly explicit write-up, including
a section on modification.
http://www.w3.org/TR/owl-test/#testProcess

I'm trying to understand
the way this TF is working, but it might not be worthwhile to get
me in sync at this point. It might be easier for me to just stay
tuned until more things become clear.

--
Dan Connolly, W3C http://www.w3.org/People/Connolly/

[Karl Dubost]

Hi Michael,

Le 7 août 2007 à 04:17, Hausenblas, Michael a écrit :
> regarding testing (support)
> I really feel left alone within W3C. There are some activities
> here (QA) and there, but NO common framework, no best practice,
> nothing in the process document AFAIK - how the heck should I know :(


What do you need in terms of best practices/framework for Testing?


The documents available for now

* The QA Handbook
   The QA Handbook (QAH) is a non-normative
   handbook about the process and operational
   aspects of certain quality assurance practices
   of W3C's Working Groups, with particular
   focus on testability and test topics.

* Building a Test Suite
   http://esw.w3.org/topic/QA

* Test Development FAQ
   http://www.w3.org/QA/WG/2005/01/test-faq

* Test Metadata
   The definition and provision of metadata has
   proved helpful in a variety of ways during
   the test development and test execution
   processes. This document defines a minimal
   set of metadata elements that can usefully
   be applied to tests that are intended for
   publication within a test suite.
   http://www.w3.org/TR/test-metadata/

* Policies for Contribution of Test Cases to W3C
   http://www.w3.org/2004/10/27-testcases


--
Karl Dubost - http://www.w3.org/People/karl/
W3C Conformance Manager, QA Activity Lead
   QA Weblog - http://www.w3.org/QA/
      *** Be Strict To Be Cool ***

[Hausenblas, Michael]

Karl,

Thanks for these pointers. If you carefully read one of
my previous posts [1], I already said that in trying to
become familiar with testing in W3C, I found some useful
stuff - first reported in [2].

I read (did not say understand :) most, if not all of the
stuff you mention. We do use for example the Test Metadata
(cf. [3]) etc.

However, what is missing IMHO are clear instructions how
to actually do the test *review*. If I understand Dan correctly
this is what he is worried about. I know, testing is no rocket
science (please do not beat me up, Shane), but anyway, I think
we are all after a transparent, unambiguous process and finally
want people to implement and use RDFa ...

Hope that this discussion might trigger a renewal of the
TC-related stuff at the QA level ... but might also be the case
that I am too inexperienced with W3C processes and the like :)

Cheers,
        Michael

[1] http://lists.w3.org/Archives/Public/public-rdf-in-xhtml-tf/2007Aug/0042.html
[2] http://lists.w3.org/Archives/Public/public-rdf-in-xhtml-tf/2007Jan/0049.html
[3] http://www.w3.org/2006/07/SWD/RDFa/testsuite/Overview.html#ManifestFiles

----------------------------------------------------------
 Michael Hausenblas, MSc.
 Institute of Information Systems & Information Management
 JOANNEUM RESEARCH Forschungsgesellschaft mbH
 
 http://www.joanneum.at/iis/
----------------------------------------------------------
 

>-----Original Message-----
>From: Karl Dubost [mailto:karl@...]
>Sent: Monday, August 06, 2007 11:49 PM
>To: Hausenblas, Michael
>Cc: Dan Connolly; Ivan Herman; W3C RDFa task force; www-qa@...
>Subject: How to handle a Test Suite at W3C Re: Error in
>(approved) test 0009
>
>Hi Michael,
>
>Le 7 août 2007 à 04:17, Hausenblas, Michael a écrit :
>> regarding testing (support)
>> I really feel left alone within W3C. There are some activities
>> here (QA) and there, but NO common framework, no best practice,
>> nothing in the process document AFAIK - how the heck should I know :(
>
>
>What do you need in terms of best practices/framework for Testing?
>
>
>The documents available for now
>
>* The QA Handbook
>   The QA Handbook (QAH) is a non-normative
>   handbook about the process and operational
>   aspects of certain quality assurance practices
>   of W3C's Working Groups, with particular
>   focus on testability and test topics.
>
>* Building a Test Suite
>   http://esw.w3.org/topic/QA
>
>* Test Development FAQ
>   http://www.w3.org/QA/WG/2005/01/test-faq
>
>* Test Metadata
>   The definition and provision of metadata has
>   proved helpful in a variety of ways during
>   the test development and test execution
>   processes. This document defines a minimal
>   set of metadata elements that can usefully
>   be applied to tests that are intended for
>   publication within a test suite.
>   http://www.w3.org/TR/test-metadata/
>
>* Policies for Contribution of Test Cases to W3C
>   http://www.w3.org/2004/10/27-testcases
>
>
>--
>Karl Dubost - http://www.w3.org/People/karl/
>W3C Conformance Manager, QA Activity Lead
>   QA Weblog - http://www.w3.org/QA/
>      *** Be Strict To Be Cool ***

[Ivan Herman-2]

This discussion is really useful for the general picture. But let us not
overadministrate ourself. The bug in this test is an absolute clear case
of misspelling, so to say, it is not _at all_ a technically
controversial case and I would not be particularly worried about
changing the test without further notice in that case.

But the situation is not always that easy. My report on test #29:

http://lists.w3.org/Archives/Public/public-rdf-in-xhtml-tf/2007Aug/0033.html

_does_ require some more disciplined procedure indeed.

Ivan



Hausenblas, Michael wrote:

> Karl,
>
> Thanks for these pointers. If you carefully read one of
> my previous posts [1], I already said that in trying to
> become familiar with testing in W3C, I found some useful
> stuff - first reported in [2].
>
> I read (did not say understand :) most, if not all of the
> stuff you mention. We do use for example the Test Metadata
> (cf. [3]) etc.
>
> However, what is missing IMHO are clear instructions how
> to actually do the test *review*. If I understand Dan correctly
> this is what he is worried about. I know, testing is no rocket
> science (please do not beat me up, Shane), but anyway, I think
> we are all after a transparent, unambiguous process and finally
> want people to implement and use RDFa ...
>
> Hope that this discussion might trigger a renewal of the
> TC-related stuff at the QA level ... but might also be the case
> that I am too inexperienced with W3C processes and the like :)
>
> Cheers,
> Michael
>
> [1] http://lists.w3.org/Archives/Public/public-rdf-in-xhtml-tf/2007Aug/0042.html
> [2] http://lists.w3.org/Archives/Public/public-rdf-in-xhtml-tf/2007Jan/0049.html
> [3] http://www.w3.org/2006/07/SWD/RDFa/testsuite/Overview.html#ManifestFiles
>
> ----------------------------------------------------------
>  Michael Hausenblas, MSc.
>  Institute of Information Systems & Information Management
>  JOANNEUM RESEARCH Forschungsgesellschaft mbH
>  
>  http://www.joanneum.at/iis/
> ----------------------------------------------------------
>  
>
>> -----Original Message-----
>> From: Karl Dubost [mailto:karl@...]
>> Sent: Monday, August 06, 2007 11:49 PM
>> To: Hausenblas, Michael
>> Cc: Dan Connolly; Ivan Herman; W3C RDFa task force; www-qa@...
>> Subject: How to handle a Test Suite at W3C Re: Error in
>> (approved) test 0009
>>
>> Hi Michael,
>>
>> Le 7 août 2007 à 04:17, Hausenblas, Michael a écrit :
>>> regarding testing (support)
>>> I really feel left alone within W3C. There are some activities
>>> here (QA) and there, but NO common framework, no best practice,
>>> nothing in the process document AFAIK - how the heck should I know :(
>>
>> What do you need in terms of best practices/framework for Testing?
>>
>>
>> The documents available for now
>>
>> * The QA Handbook
>>   The QA Handbook (QAH) is a non-normative
>>   handbook about the process and operational
>>   aspects of certain quality assurance practices
>>   of W3C's Working Groups, with particular
>>   focus on testability and test topics.
>>
>> * Building a Test Suite
>>   http://esw.w3.org/topic/QA
>>
>> * Test Development FAQ
>>   http://www.w3.org/QA/WG/2005/01/test-faq
>>
>> * Test Metadata
>>   The definition and provision of metadata has
>>   proved helpful in a variety of ways during
>>   the test development and test execution
>>   processes. This document defines a minimal
>>   set of metadata elements that can usefully
>>   be applied to tests that are intended for
>>   publication within a test suite.
>>   http://www.w3.org/TR/test-metadata/
>>
>> * Policies for Contribution of Test Cases to W3C
>>   http://www.w3.org/2004/10/27-testcases
>>
>>
>> --
>> Karl Dubost - http://www.w3.org/People/karl/
>> W3C Conformance Manager, QA Activity Lead
>>   QA Weblog - http://www.w3.org/QA/
>>      *** Be Strict To Be Cool ***

--

Ivan Herman, W3C Semantic Web Activity Lead
Home: http://www.w3.org/People/Ivan/
PGP Key: http://www.ivan-herman.net/pgpkey.html
FOAF: http://www.ivan-herman.net/foaf.rdf

[Hausenblas, Michael]

Ivan,

>This discussion is really useful for the general picture. But
>let us not overadministrate ourself.

Agree :)

>The bug in this test is an absolute clear case
>of misspelling, so to say, it is not _at all_ a technically
>controversial case and I would not be particularly worried about
>changing the test without further notice in that case.

Agree, again :)

>But the situation is not always that easy. My report on test #29:
>
>http://lists.w3.org/Archives/Public/public-rdf-in-xhtml-tf/2007Aug/0033
.html
>
>_does_ require some more disciplined procedure indeed.

Right. That's why I did not reply so far; that's why it is
still unreviewed [1], and finally that's why we are going to
(at least) review TC29-33 in the upcoming telecon - hopefully :)

Cheers,
        Michael

[1] http://www.w3.org/2006/07/SWD/wiki/RDFaTC#Approval_2007-08-02

----------------------------------------------------------
 Michael Hausenblas, MSc.
 Institute of Information Systems & Information Management
 JOANNEUM RESEARCH Forschungsgesellschaft mbH
 
 http://www.joanneum.at/iis/
----------------------------------------------------------
 

[david_marston]

Hausenblas, Michael wrote:
> However, what is missing IMHO are clear instructions how
> to actually do the test *review*....I think
> we are all after a transparent, unambiguous process...

And Ivan Herman then responded to agree that a "disciplined" process
should be available when needed. To push that discussion further, I
suggest that you look at the test review procedure [1] that the old
OASIS TC on XSLT wrote.
.................David Marston

[1] http://www.oasis-open.org/committees/download.php/1914/reviews.htm

[Christophe Strobbe]

Hi,

At 19:44 7/08/2007, david_marston@... wrote:

>Hausenblas, Michael wrote:
> > However, what is missing IMHO are clear instructions how
> > to actually do the test *review*....I think
> > we are all after a transparent, unambiguous process...
>
>And Ivan Herman then responded to agree that a "disciplined" process
>should be available when needed. To push that discussion further, I
>suggest that you look at the test review procedure [1] that the old
>OASIS TC on XSLT wrote.

Another example of a review process that you may want to look at is
the one by the WAI's Test Samples Development Task Force at
<http://www.w3.org/WAI/ER/tests/process>.

Best regards,

Christophe Strobbe

>.................David Marston
>
>[1] http://www.oasis-open.org/committees/download.php/1914/reviews.htm

--
Christophe Strobbe
K.U.Leuven - Dept. of Electrical Engineering - SCD
Research Group on Document Architectures
Kasteelpark Arenberg 10 bus 2442
B-3001 Leuven-Heverlee
BELGIUM
tel: [phone]
http://www.docarch.be/ 


Disclaimer: http://www.kuleuven.be/cwis/email_disclaimer.htm

[Hausenblas, Michael]

David,

> And Ivan Herman then responded to agree that a "disciplined" process
> should be available when needed. To push that discussion further, I
> suggest that you look at the test review procedure [1] that the old
> OASIS TC on XSLT wrote.
> .................David Marston
>
> [1] http://www.oasis-open.org/committees/download.php/1914/reviews.htm

Very helpful, indeed. Any more pointers to resolve
'reopen' issues?

Cheers,
        Michael

----------------------------------------------------------
 Michael Hausenblas, MSc.
 Institute of Information Systems & Information Management
 JOANNEUM RESEARCH Forschungsgesellschaft mbH
 Steyrergasse 17, A-8010 Graz, AUSTRIA
----------------------------------------------------------

 

[david_marston]

>>I suggest that you look at the test review procedure...

>Very helpful, indeed. Any more pointers to resolve 'reopen' issues?

Well, I wrote something [1] for QAWG which discusses post-review
complaints about test cases, with analogies to errata.

The set of states that a single test case can occupy has been the
subject of much discussion over the years. If your procedure keeps
all submissions, you may need more states (e.g., remanded for fix,
rejected, etc.) to account for tests that are not to be used but also
not "active" in the review workflow. I consider this desirable, if it
stops test case authors from wasting their time writing cases that
someone else already wrote.
.................David Marston

[1] http://lists.w3.org/Archives/Public/www-qa/2005Jan/att-0026/00-part

[Karl Dubost]

Hi Michael

Hausenblas, Michael (7 août 2007 - 15:10) :
> However, what is missing IMHO are clear instructions how
> to actually do the test *review*.

Did you made progress on this, or did you decide of a Process?

--
Karl Dubost - http://www.w3.org/People/karl/
W3C Conformance Manager, QA Activity Lead
   QA Weblog - http://www.w3.org/QA/
      *** Be Strict To Be Cool ***

[Hausenblas, Michael]

Karl,

As a matter of practice, we now have (IMHO) a stable process.

First, whenever someone raises test cases (or proposes
concrete ones, such as done by Ivan [1] or Fabien [2])
the TF puts it on the agenda (see for example [3]).

We then regularly do the review & approval in the TF
(note that there is a tool supporting this process
available at [4]). The decisions (approve/on hold/reject/)
on each TC are finally recorded at [5].

This is the process as practiced and it seems to work fine ;)

Cheers,
        Michael

[1] http://lists.w3.org/Archives/Public/public-rdf-in-xhtml-tf/2007Oct/0001.html
[2] http://lists.w3.org/Archives/Public/public-rdf-in-xhtml-tf/2007Sep/0069.html
[3] http://www.w3.org/2007/10/04-rdfa-minutes#item02
[4] http://sw.joanneum.at/rdfa/rdfadebug/index.php
[5] http://www.w3.org/2006/07/SWD/wiki/RDFaTC 

----------------------------------------------------------
 Michael Hausenblas, MSc.
 Institute of Information Systems & Information Management
 JOANNEUM RESEARCH Forschungsgesellschaft mbH
 
 http://www.joanneum.at/iis/
----------------------------------------------------------
 

>-----Original Message-----
>From: Karl Dubost [mailto:karl@...]
>Sent: Monday, October 08, 2007 11:07 AM
>To: Hausenblas, Michael
>Cc: Dan Connolly; Ivan Herman; W3C RDFa task force; www-qa@...
>Subject: Re: How to handle a Test Suite at W3C Re: Error in
>(approved) test 0009
>
>Hi Michael
>
>Hausenblas, Michael (7 août 2007 - 15:10) :
>> However, what is missing IMHO are clear instructions how
>> to actually do the test *review*.
>
>Did you made progress on this, or did you decide of a Process?
>
>--
>Karl Dubost - http://www.w3.org/People/karl/
>W3C Conformance Manager, QA Activity Lead
>   QA Weblog - http://www.w3.org/QA/
>      *** Be Strict To Be Cool ***
>
>
>
>

[boland]

fyi - there is also a review process for WCAG2.0 Test Samples Development Task
Force [1]

Best, Tim Boland NIST

[1]: http://www.w3.org/WAI/ER/tests/process


Quoting "Hausenblas, Michael" <michael.hausenblas@...>:

>
>
> Karl,
>
> As a matter of practice, we now have (IMHO) a stable process.
>
> First, whenever someone raises test cases (or proposes
> concrete ones, such as done by Ivan [1] or Fabien [2])
> the TF puts it on the agenda (see for example [3]).
>
> We then regularly do the review & approval in the TF
> (note that there is a tool supporting this process
> available at [4]). The decisions (approve/on hold/reject/)
> on each TC are finally recorded at [5].
>
> This is the process as practiced and it seems to work fine ;)
>
> Cheers,
> Michael
>
> [1]
> http://lists.w3.org/Archives/Public/public-rdf-in-xhtml-tf/2007Oct/0001.html
> [2]
> http://lists.w3.org/Archives/Public/public-rdf-in-xhtml-tf/2007Sep/0069.html
> [3] http://www.w3.org/2007/10/04-rdfa-minutes#item02
> [4] http://sw.joanneum.at/rdfa/rdfadebug/index.php
> [5] http://www.w3.org/2006/07/SWD/wiki/RDFaTC 
>
> ----------------------------------------------------------
>  Michael Hausenblas, MSc.
>  Institute of Information Systems & Information Management
>  JOANNEUM RESEARCH Forschungsgesellschaft mbH
>  
>  http://www.joanneum.at/iis/
> ----------------------------------------------------------
>  
>
> >-----Original Message-----
> >From: Karl Dubost [mailto:karl@...]
> >Sent: Monday, October 08, 2007 11:07 AM
> >To: Hausenblas, Michael
> >Cc: Dan Connolly; Ivan Herman; W3C RDFa task force; www-qa@...
> >Subject: Re: How to handle a Test Suite at W3C Re: Error in
> >(approved) test 0009
> >
> >Hi Michael
> >
> >Hausenblas, Michael (7 août 2007 - 15:10) :
> >> However, what is missing IMHO are clear instructions how
> >> to actually do the test *review*.
> >
> >Did you made progress on this, or did you decide of a Process?
> >
> >--
> >Karl Dubost - http://www.w3.org/People/karl/
> >W3C Conformance Manager, QA Activity Lead
> >   QA Weblog - http://www.w3.org/QA/
> >      *** Be Strict To Be Cool ***
> >
> >
> >
> >
>
>

[Hausenblas, Michael]

Tim,

Thanks for the pointer - very interesting stuff!

Cheers,
        Michael

----------------------------------------------------------
 Michael Hausenblas, MSc.
 Institute of Information Systems & Information Management
 JOANNEUM RESEARCH Forschungsgesellschaft mbH
 
 http://www.joanneum.at/iis/
----------------------------------------------------------
 

>-----Original Message-----
>From: boland@... [mailto:boland@...]
>Sent: Monday, October 08, 2007 3:54 PM
>To: Hausenblas, Michael
>Cc: Karl Dubost; Dan Connolly; Ivan Herman; W3C RDFa task
>force; www-qa@...
>Subject: RE: How to handle a Test Suite at W3C Re: Error in
>(approved) test 0009
>
>fyi - there is also a review process for WCAG2.0 Test Samples
>Development Task
>Force [1]
>
>Best, Tim Boland NIST
>
>[1]: http://www.w3.org/WAI/ER/tests/process
>
>
>Quoting "Hausenblas, Michael" <michael.hausenblas@...>:
>
>>
>>
>> Karl,
>>
>> As a matter of practice, we now have (IMHO) a stable process.
>>
>> First, whenever someone raises test cases (or proposes
>> concrete ones, such as done by Ivan [1] or Fabien [2])
>> the TF puts it on the agenda (see for example [3]).
>>
>> We then regularly do the review & approval in the TF
>> (note that there is a tool supporting this process
>> available at [4]). The decisions (approve/on hold/reject/)
>> on each TC are finally recorded at [5].
>>
>> This is the process as practiced and it seems to work fine ;)
>>
>> Cheers,
>> Michael
>>
>> [1]
>>
>http://lists.w3.org/Archives/Public/public-rdf-in-xhtml-tf/2007

Oct/0001.html
>> [2]
>>
>http://lists.w3.org/Archives/Public/public-rdf-in-xhtml-tf/2007
Sep/0069.html

>> [3] http://www.w3.org/2007/10/04-rdfa-minutes#item02
>> [4] http://sw.joanneum.at/rdfa/rdfadebug/index.php
>> [5] http://www.w3.org/2006/07/SWD/wiki/RDFaTC 
>>
>> ----------------------------------------------------------
>>  Michael Hausenblas, MSc.
>>  Institute of Information Systems & Information Management
>>  JOANNEUM RESEARCH Forschungsgesellschaft mbH
>>  
>>  http://www.joanneum.at/iis/
>> ----------------------------------------------------------
>>  
>>
>> >-----Original Message-----
>> >From: Karl Dubost [mailto:karl@...]
>> >Sent: Monday, October 08, 2007 11:07 AM
>> >To: Hausenblas, Michael
>> >Cc: Dan Connolly; Ivan Herman; W3C RDFa task force; www-qa@...
>> >Subject: Re: How to handle a Test Suite at W3C Re: Error in
>> >(approved) test 0009
>> >
>> >Hi Michael
>> >
>> >Hausenblas, Michael (7 août 2007 - 15:10) :
>> >> However, what is missing IMHO are clear instructions how
>> >> to actually do the test *review*.
>> >
>> >Did you made progress on this, or did you decide of a Process?
>> >
>> >--
>> >Karl Dubost - http://www.w3.org/People/karl/
>> >W3C Conformance Manager, QA Activity Lead
>> >   QA Weblog - http://www.w3.org/QA/
>> >      *** Be Strict To Be Cool ***
>> >
>> >
>> >
>> >
>>
>>
>
>
>

[Shadi Abou-Zahra]

Hi Michael,

Any comments and feedback on this review process are welcome, please
send them to <public-wai-ert-tsdtf@...>. You may also be interested
in the metadata that we developed for managing the test suites:
  - <http://www.w3.org/WAI/ER/tests/usingTCDL>

Regards,
   Shadi


Hausenblas, Michael wrote:

>
> Tim,
>
> Thanks for the pointer - very interesting stuff!
>
> Cheers,
> Michael
>
> ----------------------------------------------------------
>  Michael Hausenblas, MSc.
>  Institute of Information Systems & Information Management
>  JOANNEUM RESEARCH Forschungsgesellschaft mbH
>  
>  http://www.joanneum.at/iis/
> ----------------------------------------------------------
>  
>
>> -----Original Message-----
>> From: boland@... [mailto:boland@...]
>> Sent: Monday, October 08, 2007 3:54 PM
>> To: Hausenblas, Michael
>> Cc: Karl Dubost; Dan Connolly; Ivan Herman; W3C RDFa task
>> force; www-qa@...
>> Subject: RE: How to handle a Test Suite at W3C Re: Error in
>> (approved) test 0009
>>
>> fyi - there is also a review process for WCAG2.0 Test Samples
>> Development Task
>> Force [1]
>>
>> Best, Tim Boland NIST
>>
>> [1]: http://www.w3.org/WAI/ER/tests/process
>>
>>
>> Quoting "Hausenblas, Michael" <michael.hausenblas@...>:
>>
>>>
>>> Karl,
>>>
>>> As a matter of practice, we now have (IMHO) a stable process.
>>>
>>> First, whenever someone raises test cases (or proposes
>>> concrete ones, such as done by Ivan [1] or Fabien [2])
>>> the TF puts it on the agenda (see for example [3]).
>>>
>>> We then regularly do the review & approval in the TF
>>> (note that there is a tool supporting this process
>>> available at [4]). The decisions (approve/on hold/reject/)
>>> on each TC are finally recorded at [5].
>>>
>>> This is the process as practiced and it seems to work fine ;)
>>>
>>> Cheers,
>>> Michael
>>>
>>> [1]
>>>
>> http://lists.w3.org/Archives/Public/public-rdf-in-xhtml-tf/2007
> Oct/0001.html
>>> [2]
>>>
>> http://lists.w3.org/Archives/Public/public-rdf-in-xhtml-tf/2007
> Sep/0069.html
>>> [3] http://www.w3.org/2007/10/04-rdfa-minutes#item02
>>> [4] http://sw.joanneum.at/rdfa/rdfadebug/index.php
>>> [5] http://www.w3.org/2006/07/SWD/wiki/RDFaTC 
>>>
>>> ----------------------------------------------------------
>>>  Michael Hausenblas, MSc.
>>>  Institute of Information Systems & Information Management
>>>  JOANNEUM RESEARCH Forschungsgesellschaft mbH
>>>  
>>>  http://www.joanneum.at/iis/
>>> ----------------------------------------------------------
>>>  
>>>
>>>> -----Original Message-----
>>>> From: Karl Dubost [mailto:karl@...]
>>>> Sent: Monday, October 08, 2007 11:07 AM
>>>> To: Hausenblas, Michael
>>>> Cc: Dan Connolly; Ivan Herman; W3C RDFa task force; www-qa@...
>>>> Subject: Re: How to handle a Test Suite at W3C Re: Error in
>>>> (approved) test 0009
>>>>
>>>> Hi Michael
>>>>
>>>> Hausenblas, Michael (7 août 2007 - 15:10) :
>>>>> However, what is missing IMHO are clear instructions how
>>>>> to actually do the test *review*.
>>>> Did you made progress on this, or did you decide of a Process?
>>>>
>>>> --
>>>> Karl Dubost - http://www.w3.org/People/karl/
>>>> W3C Conformance Manager, QA Activity Lead
>>>>   QA Weblog - http://www.w3.org/QA/
>>>>      *** Be Strict To Be Cool ***
>>>>
>>>>
>>>>
>>>>
>>>
>>
>>
>
>

--
Shadi Abou-Zahra     Web Accessibility Specialist for Europe |
Chair & Staff Contact for the Evaluation and Repair Tools WG |
World Wide Web Consortium (W3C)           http://www.w3.org/ |
Web Accessibility Initiative (WAI),   http://www.w3.org/WAI/ |
WAI-TIES Project,                http://www.w3.org/WAI/TIES/ |
Evaluation and Repair Tools WG,    http://www.w3.org/WAI/ER/ |
2004, Route des Lucioles - 06560,  Sophia-Antipolis - France |
Voice: [phone]          Fax: [phone] |