« Return to Thread: pyRDFa Test Suite Reviewer
Re: pyRDFa Test Suite Reviewer
by Ivan Herman-2
::
Rate this Message:
Manu,
I was thinking of something similar, although my strategy was meant to
be a bit different
- I would do the RDFa->RDF conversion using an online service. I guess
it would be possible to send a URI to either Fabien's service or pyRdfa
(well, I know about the latter) or others. In other words, a proper URI
for a test result could be minted. For example,
http://www.w3.org/2007/08/pyRdfa/extract?uri=http%3A//www.w3.org/2006/07/SWD/RDFa/testsuite/xhtml1-testcases/0012.xhtml
is a URI for test #12 with pyRdfa.
- I would create a full SPARQL query using that URI in a 'FROM'
- submit the SPARQL query to an online SPARQL site, like sparqler
http://www.sparql.org/
(I have created a SPARQL library to make such submission easier:
http://ivanherman.wordpress.com/2007/07/06/sparql-endpoint-interface-to-python/
that could come handy here. But there are others.)
This would have, in my view, several advantages:
- the test would be done on the latest versions. At least for pyRdfa I
know for a fact that the version running on the web is ahead of the
tar.gz distribution
- the same test could be done on different implementations easily with a
minor change in the target URI
- the test would rely on an external and, possibly, more reliable sparql
service.
As an example, you can send to sparqler the following query:
ASK
FROM
<http://www.w3.org/2007/08/pyRdfa/extract?uri=http%3A//www.w3.org/2006/07/SWD/RDFa/testsuite/xhtml1-testcases/0012.xhtml>
WHERE {
<http://example.org/node> <http://example.org/property> "chat"@fr .
}
to test pyRdfa and test #12.
Actually... I picked on #12 because it is also an example where the
python SPARQL goes wrong. Indeed, the sparqler result above is 'True'.
However, you test lists it as failing. For reference, here is the result
in Turtle generated by PyRDFa:
[[[
@prefix ex: <http://example.org/>.
ex:node ex:property "chat"@fr.
]]]
Which _is_ correct with the test above:-)
We will have something nice here.
Ivan
Manu Sporny wrote:
> I have had a very hard time reading each RDFa test suite XHTML file,
> SPARQL file and attempting to determine what triples should be generated
> from the given XHTML. We might expect beginning RDFa implementers to
> have the same problem. The test suite is not only good for validation,
> but for understanding how RDFa works.
>
> With the previous problems/thoughts in mind, I hacked together a quick
> and dirty test suite reviewer. The reviewer is attached, along with two
> examples of the output of the tool. It builds upon Ivan Herman's work on
> pyRDFa and Ed Summer's modifications to the tool.
>
> The new tool, tsreview.py, will do the following:
> - Retrieve all approved or unreviewed RDFa test cases
> - Execute the SPARQL queries on the generated graph
> - Dump the graph, in N3 format, to a file.
> - Dump the raw XHTML and raw SPARQL to a file.
> - Generate a test report, containing all the information necessary
> (XHTML, N3, and SPARQL), to understand whether the test case is
> behaving or not.
>
> All of the current N3 and review files are included in the attached
> .tar.bz2, in a directory called "review".
>
> Two of the files that are generated, as a part of each test, are
> important to implementers.
>
> 1. The review file very clearly shows the input, the output in N3, and
> the SPARQL validation code. This will help developers understand
> what is supposed to happen in their parsers.
> 2. The graph N3 serialization is provided. This ensures that developers
> won't have to write a SPARQL engine to test whether or not their
> parser is compliant.
>
> Could we include the auto-generated N3 serialization as part of the RDFa
> test suite (for documentation purposes)?
>
> Could we include the review files (or something similar) as part of the
> RDFa test suite (again, for documentation purposes)?
>
> ------------------------------------------------------------------
> Command line output from the tsreview.py tool:
>
> test-01-approved (__main__.XhtmlTests) ... ok
> test-06-approved (__main__.XhtmlTests) ... ok
> test-07-approved (__main__.XhtmlTests) ... ok
> test-08-approved (__main__.XhtmlTests) ... ok
> test-09-approved (__main__.XhtmlTests) ... ok
> test-10-approved (__main__.XhtmlTests) ... ok
> test-11-approved (__main__.XhtmlTests) ... FAIL
> test-12-approved (__main__.XhtmlTests) ... FAIL
> test-13-approved (__main__.XhtmlTests) ... FAIL
> test-14-approved (__main__.XhtmlTests) ... FAIL
> test-15-approved (__main__.XhtmlTests) ... ok
> test-18-approved (__main__.XhtmlTests) ... ok
> test-19-approved (__main__.XhtmlTests) ... ok
> test-20-approved (__main__.XhtmlTests) ... ok
> test-21-approved (__main__.XhtmlTests) ... ok
> test-23-approved (__main__.XhtmlTests) ... ok
> test-25-approved (__main__.XhtmlTests) ... ok
> test-26-approved (__main__.XhtmlTests) ... ok
> test-27-approved (__main__.XhtmlTests) ... ok
> test-29-approved (__main__.XhtmlTests) ... FAIL
> test-30-approved (__main__.XhtmlTests) ... ok
> test-31-approved (__main__.XhtmlTests) ... ok
> test-32-approved (__main__.XhtmlTests) ... ok
> test-33-approved (__main__.XhtmlTests) ... FAIL
> test-34-approved (__main__.XhtmlTests) ... ok
> test-35-approved (__main__.XhtmlTests) ... ok
> test-36-approved (__main__.XhtmlTests) ... ok
> test-37-approved (__main__.XhtmlTests) ... ok
> test-38-approved (__main__.XhtmlTests) ... ok
> test-41-approved (__main__.XhtmlTests) ... ok
> test-46-unreviewed (__main__.XhtmlTests) ... FAIL
> test-47-unreviewed (__main__.XhtmlTests) ... ok
> test-48-unreviewed (__main__.XhtmlTests) ... FAIL
> test-49-unreviewed (__main__.XhtmlTests) ... ok
> test-50-unreviewed (__main__.XhtmlTests) ... FAIL
> test-51-unreviewed (__main__.XhtmlTests) ... FAIL
> test-52-unreviewed (__main__.XhtmlTests) ... ok
> test-53-unreviewed (__main__.XhtmlTests) ... ok
> test-54-unreviewed (__main__.XhtmlTests) ... ok
> test-55-unreviewed (__main__.XhtmlTests) ... ok
> test-56-unreviewed (__main__.XhtmlTests) ... ERROR
>
> -- manu
>
>
>
> ------------------------------------------------------------------------
>
>
> XHTML
>
> <?xml version="1.0" encoding="UTF-8"?>
> <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML+RDFa 1.0//EN" "http://www.w3.org/MarkUp/DTD/xhtml-rdfa-1.dtd">
> <html xmlns="http://www.w3.org/1999/xhtml"
> xmlns:cc="http://creativecommons.org/ns#">
> <head>
> <title>Test 0008</title>
> </head>
> <body>
> <p>This document is licensed under a
> <a about="" rel="cc:license"
> href="http://creativecommons.org/licenses/by-nc-nd/2.5/">
> Creative Commons
> </a>.
> </p>
> </body>
> </html>
>
>
>
>
> Triples (N3 format)
>
>
> @prefix _5: <http://www.w3.org/2006/07/SWD/RDFa/testsuite/xhtml1-testcases/0008.>.
> @prefix cc: <http://creativecommons.org/ns#>.
>
> _5:xhtml cc:license <http://creativecommons.org/licenses/by-nc-nd/2.5/>.
>
>
>
> SPARQL (PASSED)
>
> ASK WHERE {
> <http://www.w3.org/2006/07/SWD/RDFa/testsuite/xhtml1-testcases/0008.xhtml> <http://creativecommons.org/ns#license> <http://creativecommons.org/licenses/by-nc-nd/2.5/> .
> }
>
>
>
> ------------------------------------------------------------------------
>
>
> XHTML
>
> <?xml version="1.0" encoding="UTF-8"?>
> <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML+RDFa 1.0//EN" "http://www.w3.org/MarkUp/DTD/xhtml-rdfa-1.dtd">
> <html xmlns="http://www.w3.org/1999/xhtml"
> xmlns:foaf="http://xmlns.com/foaf/0.1/"
> >
> <head>
> <title>Test 0048</title>
> </head>
> <body>
> <div about="http://www.example.org" rel="foaf:knows" instanceof="foaf:Person">
> <p property="foaf:name">John Doe</p>
> </div>
> </body>
> </html>
>
>
>
>
> Triples (N3 format)
>
>
> @prefix foaf: <http://xmlns.com/foaf/0.1/>.
> @prefix rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#>.
>
> <http://www.example.org> a foaf:Person;
> foaf:knows [ foaf:name "John Doe"].
>
>
>
> SPARQL (FAILED)
>
> ASK WHERE {
> <http://www.example.org> <http://xmlns.com/foaf/0.1/knows> _:a .
> _:a <http://xmlns.com/foaf/0.1/name> "John Doe" .
> <http://www.example.org> <http://www.w3.org/1999/02/22-rdf-syntax-ns#type> <http://xmlns.com/foaf/0.1/Person> .
> }
>
>--
Ivan Herman, W3C Semantic Web Activity Lead
Home: http://www.w3.org/People/Ivan/
PGP Key: http://www.ivan-herman.net/pgpkey.html
FOAF: http://www.ivan-herman.net/foaf.rdf
I was thinking of something similar, although my strategy was meant to
be a bit different
- I would do the RDFa->RDF conversion using an online service. I guess
it would be possible to send a URI to either Fabien's service or pyRdfa
(well, I know about the latter) or others. In other words, a proper URI
for a test result could be minted. For example,
http://www.w3.org/2007/08/pyRdfa/extract?uri=http%3A//www.w3.org/2006/07/SWD/RDFa/testsuite/xhtml1-testcases/0012.xhtml
is a URI for test #12 with pyRdfa.
- I would create a full SPARQL query using that URI in a 'FROM'
- submit the SPARQL query to an online SPARQL site, like sparqler
http://www.sparql.org/
(I have created a SPARQL library to make such submission easier:
http://ivanherman.wordpress.com/2007/07/06/sparql-endpoint-interface-to-python/
that could come handy here. But there are others.)
This would have, in my view, several advantages:
- the test would be done on the latest versions. At least for pyRdfa I
know for a fact that the version running on the web is ahead of the
tar.gz distribution
- the same test could be done on different implementations easily with a
minor change in the target URI
- the test would rely on an external and, possibly, more reliable sparql
service.
As an example, you can send to sparqler the following query:
ASK
FROM
<http://www.w3.org/2007/08/pyRdfa/extract?uri=http%3A//www.w3.org/2006/07/SWD/RDFa/testsuite/xhtml1-testcases/0012.xhtml>
WHERE {
<http://example.org/node> <http://example.org/property> "chat"@fr .
}
to test pyRdfa and test #12.
Actually... I picked on #12 because it is also an example where the
python SPARQL goes wrong. Indeed, the sparqler result above is 'True'.
However, you test lists it as failing. For reference, here is the result
in Turtle generated by PyRDFa:
[[[
@prefix ex: <http://example.org/>.
ex:node ex:property "chat"@fr.
]]]
Which _is_ correct with the test above:-)
We will have something nice here.
Ivan
Manu Sporny wrote:
> I have had a very hard time reading each RDFa test suite XHTML file,
> SPARQL file and attempting to determine what triples should be generated
> from the given XHTML. We might expect beginning RDFa implementers to
> have the same problem. The test suite is not only good for validation,
> but for understanding how RDFa works.
>
> With the previous problems/thoughts in mind, I hacked together a quick
> and dirty test suite reviewer. The reviewer is attached, along with two
> examples of the output of the tool. It builds upon Ivan Herman's work on
> pyRDFa and Ed Summer's modifications to the tool.
>
> The new tool, tsreview.py, will do the following:
> - Retrieve all approved or unreviewed RDFa test cases
> - Execute the SPARQL queries on the generated graph
> - Dump the graph, in N3 format, to a file.
> - Dump the raw XHTML and raw SPARQL to a file.
> - Generate a test report, containing all the information necessary
> (XHTML, N3, and SPARQL), to understand whether the test case is
> behaving or not.
>
> All of the current N3 and review files are included in the attached
> .tar.bz2, in a directory called "review".
>
> Two of the files that are generated, as a part of each test, are
> important to implementers.
>
> 1. The review file very clearly shows the input, the output in N3, and
> the SPARQL validation code. This will help developers understand
> what is supposed to happen in their parsers.
> 2. The graph N3 serialization is provided. This ensures that developers
> won't have to write a SPARQL engine to test whether or not their
> parser is compliant.
>
> Could we include the auto-generated N3 serialization as part of the RDFa
> test suite (for documentation purposes)?
>
> Could we include the review files (or something similar) as part of the
> RDFa test suite (again, for documentation purposes)?
>
> ------------------------------------------------------------------
> Command line output from the tsreview.py tool:
>
> test-01-approved (__main__.XhtmlTests) ... ok
> test-06-approved (__main__.XhtmlTests) ... ok
> test-07-approved (__main__.XhtmlTests) ... ok
> test-08-approved (__main__.XhtmlTests) ... ok
> test-09-approved (__main__.XhtmlTests) ... ok
> test-10-approved (__main__.XhtmlTests) ... ok
> test-11-approved (__main__.XhtmlTests) ... FAIL
> test-12-approved (__main__.XhtmlTests) ... FAIL
> test-13-approved (__main__.XhtmlTests) ... FAIL
> test-14-approved (__main__.XhtmlTests) ... FAIL
> test-15-approved (__main__.XhtmlTests) ... ok
> test-18-approved (__main__.XhtmlTests) ... ok
> test-19-approved (__main__.XhtmlTests) ... ok
> test-20-approved (__main__.XhtmlTests) ... ok
> test-21-approved (__main__.XhtmlTests) ... ok
> test-23-approved (__main__.XhtmlTests) ... ok
> test-25-approved (__main__.XhtmlTests) ... ok
> test-26-approved (__main__.XhtmlTests) ... ok
> test-27-approved (__main__.XhtmlTests) ... ok
> test-29-approved (__main__.XhtmlTests) ... FAIL
> test-30-approved (__main__.XhtmlTests) ... ok
> test-31-approved (__main__.XhtmlTests) ... ok
> test-32-approved (__main__.XhtmlTests) ... ok
> test-33-approved (__main__.XhtmlTests) ... FAIL
> test-34-approved (__main__.XhtmlTests) ... ok
> test-35-approved (__main__.XhtmlTests) ... ok
> test-36-approved (__main__.XhtmlTests) ... ok
> test-37-approved (__main__.XhtmlTests) ... ok
> test-38-approved (__main__.XhtmlTests) ... ok
> test-41-approved (__main__.XhtmlTests) ... ok
> test-46-unreviewed (__main__.XhtmlTests) ... FAIL
> test-47-unreviewed (__main__.XhtmlTests) ... ok
> test-48-unreviewed (__main__.XhtmlTests) ... FAIL
> test-49-unreviewed (__main__.XhtmlTests) ... ok
> test-50-unreviewed (__main__.XhtmlTests) ... FAIL
> test-51-unreviewed (__main__.XhtmlTests) ... FAIL
> test-52-unreviewed (__main__.XhtmlTests) ... ok
> test-53-unreviewed (__main__.XhtmlTests) ... ok
> test-54-unreviewed (__main__.XhtmlTests) ... ok
> test-55-unreviewed (__main__.XhtmlTests) ... ok
> test-56-unreviewed (__main__.XhtmlTests) ... ERROR
>
> -- manu
>
>
>
> ------------------------------------------------------------------------
>
>
> XHTML
>
> <?xml version="1.0" encoding="UTF-8"?>
> <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML+RDFa 1.0//EN" "http://www.w3.org/MarkUp/DTD/xhtml-rdfa-1.dtd">
> <html xmlns="http://www.w3.org/1999/xhtml"
> xmlns:cc="http://creativecommons.org/ns#">
> <head>
> <title>Test 0008</title>
> </head>
> <body>
> <p>This document is licensed under a
> <a about="" rel="cc:license"
> href="http://creativecommons.org/licenses/by-nc-nd/2.5/">
> Creative Commons
> </a>.
> </p>
> </body>
> </html>
>
>
>
>
> Triples (N3 format)
>
>
> @prefix _5: <http://www.w3.org/2006/07/SWD/RDFa/testsuite/xhtml1-testcases/0008.>.
> @prefix cc: <http://creativecommons.org/ns#>.
>
> _5:xhtml cc:license <http://creativecommons.org/licenses/by-nc-nd/2.5/>.
>
>
>
> SPARQL (PASSED)
>
> ASK WHERE {
> <http://www.w3.org/2006/07/SWD/RDFa/testsuite/xhtml1-testcases/0008.xhtml> <http://creativecommons.org/ns#license> <http://creativecommons.org/licenses/by-nc-nd/2.5/> .
> }
>
>
>
> ------------------------------------------------------------------------
>
>
> XHTML
>
> <?xml version="1.0" encoding="UTF-8"?>
> <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML+RDFa 1.0//EN" "http://www.w3.org/MarkUp/DTD/xhtml-rdfa-1.dtd">
> <html xmlns="http://www.w3.org/1999/xhtml"
> xmlns:foaf="http://xmlns.com/foaf/0.1/"
> >
> <head>
> <title>Test 0048</title>
> </head>
> <body>
> <div about="http://www.example.org" rel="foaf:knows" instanceof="foaf:Person">
> <p property="foaf:name">John Doe</p>
> </div>
> </body>
> </html>
>
>
>
>
> Triples (N3 format)
>
>
> @prefix foaf: <http://xmlns.com/foaf/0.1/>.
> @prefix rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#>.
>
> <http://www.example.org> a foaf:Person;
> foaf:knows [ foaf:name "John Doe"].
>
>
>
> SPARQL (FAILED)
>
> ASK WHERE {
> <http://www.example.org> <http://xmlns.com/foaf/0.1/knows> _:a .
> _:a <http://xmlns.com/foaf/0.1/name> "John Doe" .
> <http://www.example.org> <http://www.w3.org/1999/02/22-rdf-syntax-ns#type> <http://xmlns.com/foaf/0.1/Person> .
> }
>
>
Ivan Herman, W3C Semantic Web Activity Lead
Home: http://www.w3.org/People/Ivan/
PGP Key: http://www.ivan-herman.net/pgpkey.html
FOAF: http://www.ivan-herman.net/foaf.rdf
smime.p7s (4K) « Return to Thread: pyRDFa Test Suite Reviewer
| Free embeddable forum powered by Nabble | Forum Help |
