Escalation mechanism for different interpretation of W3C XML-Schema specification ?

[XMLSchema at XML4Pharma]

Some parts of this message have been removed. Learn more about Nabble's security policy.

We, the CDISC XML-Tech Governance Team (and other CDISC teams) have developed a number of extensible standards for exchange of clinical data and for submitting information to the regulatory authorities (FDA).

CDISC is a Standardization Organization active in the healthcare world.

Our extension mechanism is based on the “import” and “redefine” elements of XML-Schema.

We now have a serious dispute with one technology vendor (Altova) about the way “import” and “redefine” are used. Instance files of one of our extensions (so-called “define.xml”) validate well in all major validators and XML-editors, except for the products of this one vendor.

When confronted with this result, the reaction of Altova essentially is that “Altova is right, all others are wrong”. The dispute and discussion with Altova can be followed at:
http://www.altova.com/forum/default.aspx?g=posts&m=1000005665

The issue were not so serious if it were not that our standard “define.xml” is a standard for submission of information to the regulatory authorities, and these are (mostly) using the Altova product for validation.

We now want to escalate the issue to the W3C itself, and would like to know what the mechanism is to do so.

Jozef Aerts
CDISC XML-Tech Governance Team

[Andrew Welch]

2009/9/29 XMLSchema at XML4Pharma <XMLSchema@...>:

> We, the CDISC XML-Tech Governance Team (and other CDISC teams) have
> developed a number of extensible standards for exchange of clinical data and
> for submitting information to the regulatory authorities (FDA).
>
> CDISC is a Standardization Organization active in the healthcare world.
>
> Our extension mechanism is based on the “import” and “redefine” elements of
> XML-Schema.
>
> We now have a serious dispute with one technology vendor (Altova) about the
> way “import” and “redefine” are used. Instance files of one of our
> extensions (so-called “define.xml”) validate well in all major validators
> and XML-editors, except for the products of this one vendor.
>
> When confronted with this result, the reaction of Altova essentially is that
> “Altova is right, all others are wrong”. The dispute and discussion with
> Altova can be followed at:
> http://www.altova.com/forum/default.aspx?g=posts&m=1000005665
>
> The issue were not so serious if it were not that our standard “define.xml”
> is a standard for submission of information to the regulatory authorities,
> and these are (mostly) using the Altova product for validation.

This was talked about before:

http://markmail.org/message/ccxa6gbyvn3773kx

The advice you were given back then was "xs:redefine is pretty messy,
and the best advice is to avoid it" so it seems a little odd to now
take the stance you have... especially when you say most of your
customers use Altova.

I can understand your frustrations, but I'm not too sure what your
want to achieve here - even if Altova agree you're right and a make a
fix for their next release, it will still take a long time for that
version to filter down to your existing customers, if at all.  By the
way, what does "escalating it to the W3C" mean?

This reminds me of a project where the app worked fine on everything
except IE 5.5 on windows NT, and that was what the majority of the
users were on.... it would have been nice to escalate the issue to
make the problem go away :) but sadly we had to code around it.

Not very helpful I know, sorry, perhaps you could move straight to XSD 1.1?


--
Andrew Welch
http://andrewjwelch.com
Kernow: http://kernowforsaxon.sf.net/

[Henry S. Thompson]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

XMLSchema at XML4Pharma writes:

> Our extension mechanism is based on the "import" and "redefine"
> elements of XML-Schema.

> [see] http://www.altova.com/forum/default.aspx?g=posts&m=1000005665

> We now want to escalate the issue to the W3C itself, and would like
> to know what the mechanism is to do so.

We can only try to clarify and persuade -- we have no authority to
compel anyone.

I can only explain how/why XSV behaves as it does, and offer my
understand of the spec.

The spec. allows (note, _not_ requires) implementations to ignore
attempts to import schema documents for namespaces for which a schema
document is already known:

  "Given that the schemaLocation [attribute] is only a hint, it is
   open to applications to ignore all but the first <import> for a
   given namespace, regardless of the ·actual value· of
   schemaLocation, but such a strategy risks missing useful
   information when new schemaLocations are offered." [1]

XSV follows this strategy.  In the case of the CDISC schemas, this
means the xs:import of ODM1-2-1.xsd is skipped, because a schema
document (namely define-1.0.0.xsd) is already known for the odm/v1.2
namespace.

In the case of the Altova example, this means the xs:import of
XSD2.xsd is skipped, because a schema document (namely XSD1.xsd) is
already known for the no-target-namespace case.

In other words, in neither case is the behaviour of XSD to do with how
it resolves redefine/import conflicts.

But note that any schema processor which _does_ follow all
schemaLocation hints, and therefore does import multiple documents for
the same target namespace, is perfectly conformant as well.

So, what about redefine/import conflicts?

The spec. contains no explicit guidance.  Furthermore, it is impossible to
construct an example which would provoke such a conflict in XSV,
because it would require two imports of the same targetNamespace (the
reason for this is left as an exercise for the reader -- if I've
missed a way to bring this about, I owe the first person to point out
how a beer :-).

We can however construct an example of conflicting redefine/include in
XSV, by making small modifications to the XSD3.xsd document given in
the Altova example:

XSD3.xsd
<xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema"
  elementFormDefault="qualified" attributeFormDefault="unqualified"
  targetNamespace="http://XSD3" xmlns:x="http://XSD3">
<xs:include schemaLocation="XSD2.xsd"/>
<xs:include schemaLocation="XSD1.xsd"/>
<xs:element name="elt" type="x:CT"/>
</xs:schema>

We're now chameleon including/redefining, and the result works with
XSV 3.1, regardless of the order of the two include statements.  By
'works', I mean
 a) The schema is judged to be OK;
 b) The instance is judged to be bad.

I read the spec. as _requiring_ this behaviour, because it says

  "The modifications have a pervasive impact, that is, only the
   redefined components are used, even when referenced from other
   incorporated components, whether redefined themselves or not." [2]

What this means is that no conflict occurs between the redefined and
non-redefined components, because the later are not part of the schema
used for validation.  The same argument seems to me to apply to the
redefine/import case.

I would be interested to know how other products behave with the
redefine/include conflict example given above.

Summary: I believe the spec. intends the redefine semantics to 'trump'
the include and import semantics wrt which components are present in
cases of conflict, but I do not disagree that this is not so clear as
to be uncontestable.

I also think it is unfortunate that all implementors cannot agree on a
single interpretation.  If it is true, as alleged, that the situation
is that many implementations agree on the interpretation but one does
not, that's particularly unfortunate.

ht
 
[1] http://www.w3.org/TR/xmlschema-1/#src-import
[1] http://www.w3.org/TR/xmlschema-1/#modify-schema
- --
       Henry S. Thompson, School of Informatics, University of Edinburgh
                         Half-time member of W3C Team
      10 Crichton Street, Edinburgh EH8 9AB, SCOTLAND -- (44) 131 650-4440
                Fax: [phone], e-mail: ht@...
                       URL: http://www.ltg.ed.ac.uk/~ht/
[mail really from me _always_ has this .sig -- mail without it is forged spam]
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.6 (GNU/Linux)

iD8DBQFKwh4EkjnJixAXWBoRAsVuAJ9PFjtJMUNAycnVwR3WlFIrJVCI3wCfVeQX
c0UQGS14hYnXrTq2lW2ADVE=
=m9RM
-----END PGP SIGNATURE-----

[C. M. Sperberg-McQueen-2]

On 29 Sep 2009, at 05:24 , XMLSchema at XML4Pharma wrote:

> We, the CDISC XML-Tech Governance Team (and other CDISC teams) have  
> developed a number of extensible standards for exchange of clinical  
> data and for submitting information to the regulatory authorities  
> (FDA).
>
> CDISC is a Standardization Organization active in the healthcare  
> world.
>
> Our extension mechanism is based on the “import” and “redefine”  
> elements of XML-Schema.
>
> We now have a serious dispute with one technology vendor (Altova)  
> about the way “import” and “redefine” are used. Instance files of  
> one of our extensions (so-called “define.xml”) validate well in all  
> major validators and XML-editors, except for the products of this  
> one vendor.
>
> When confronted with this result, the reaction of Altova essentially  
> is that “Altova is right, all others are wrong”. The dispute and  
> discussion with Altova can be followed at:
> http://www.altova.com/forum/default.aspx?g=posts&m=1000005665
>
> The issue were not so serious if it were not that our standard  
> “define.xml” is a standard for submission of information to the  
> regulatory authorities, and these are (mostly) using the Altova  
> product for validation.
>
> We now want to escalate the issue to the W3C itself, and would like  
> to know what the mechanism is to do so.
>
> Jozef Aerts
> CDISC XML-Tech Governance Team
>

I believe that some standards bodies have mechanisms for requesting
authoritative interpretations of doubtful points in specifications
published by those bodies.  W3C does not have any formal mechanism
for such interpretations, but you can certainly request an opinion
from the W3C XML Schema Working Group by sending mail to the
XSD comments list at www-xml-schema-comments@... (or by raising
a Bugzilla issue against XSD 1.0).

The public record shows, however, that the XML Schema Working Group
has already addressed the question of what happens when the same
schema document is referred to both by a direct import or include
and indirectly through a reference to another schema document
which redefines the first (see Bugzilla bugs 2330, 2557, 2824,
and on related issues also 2826 and 5425, in the W3C public
instance of Bugzilla at http://www.w3.org/Bugs/Public/).  As the
record shows, the Working Group has been unable to resolve those
issues in part because the group has not been able to reach
consensus on the meaning of the 1.0 specification in the cases
in question.

Reviewing the discussion record on the Altova forum I am struck by
the repeated remark that the CDISC team has consulted "several
distinguished members of the W3C XML-Schema working group" who
confirmed CDISC's interpretation of the XSD 1.0 spec.  I can only
hope that these consultations took place some years ago, before
the working group realized just how divergent its interpretations
of the specification are; if you have consulted with any member
of the working group in the last four years about this case, and
they have failed to mention to you that the spec invites multiple
contradictory interpretations, and that the working group has
been unable to agree on what a single interpretation, then I
fear they have misled you sadly.

I'm also struck by the report that XML Spy is an outlier in its
behavior on this case.  The work I did in 2007 on an analogous
case suggested that depending on how they were invoked, the
eight processors tested behaved in nine or ten different ways
(fourteen if you count error messages reporting different
diagnoses of the problem as different behaviors).  The
work is described in a document available at
http://www.cmsmcq.com/2007/schema_composition/model.xml
(the test cases which elicit these different behaviors are
described in section 5.2.2 and its subsctions).

The XSD 1.0 specification does not explicitly address the case
when the same schema document in referenced both directly
(via import or include) and indirectly (via redefine).  The
rules for import specify that the components imported
via the import or include should be present in the resulting
schema; the discussion of redefine says that the effect of
redefine is pervasive.

Some readers interpret this to mean that the pervasive-redefinition
rule overrides, or modifies the effect of, the import and include
rules.  On this reading the schema is legal and the components
are present in their redefined form (and only that form).  This
reading effectively attaches to the rules for include and import
an unspoken exception to the effect that the components imported
or included are present in the resulting schema unless they are
redefined by some other schema document reference.

Other readers interpret the rules as requiring that the
components in question should be present both in their
original and in their redefined form, which means that the
schema violates the rule that there must be at most one
component of any kind with a given expanded name, and thus
that the schema should be rejected.

It is my personal belief that the most plausible interpretation
of the specification is the latter; as you have observed, at
least some implementors disagree and prefer the other
interpretation.

The inability of the XML Schema working group to agree on a
normative interpretation of XSD 1.0 has led to the agreement to
deprecate the redefine element in XSD 1.1.  My personal
recommendation is either to avoid redefine entirely or to
avoid the use of schemaLocation information in your primary
schema documents, restricting the identification of specific
schema documents to top-level 'driver' documents, or to
invocation-time parameters and options passed to the
validator.

I hope this helps.

Michael Sperberg-McQueen



--
****************************************************************
* C. M. Sperberg-McQueen, Black Mesa Technologies LLC
* http://www.blackmesatech.com
* http://cmsmcq.com/mib
* http://balisage.net
****************************************************************

[Cesar Ariza (gmd)]

Hi,

 

Some time ago we have a similar problem in a project with one schema per "concept" aproach project, we create almost 3500 schema-s. (GEL-XML ).

 

The XML-Spy validator is so different (so good)  to other parsers, so you have o create (readable) schema-s  for all parsers implementations, even worse to all code generators (e.g. jaxb, .net, etc), in order to be usable-s in the real world.

 

The solution was to ban redefines and to use hierarchical proxy-schema-s (libraries), so the parser read one time and only one time the imported library with an unique namepace and/per location. 

 

The other (administrative) solution was to define one unique parser-brand to validate the schema-s. 

 

Cheers,

 

César

 

On Tue, Sep 29, 2009 at 9:47 AM, Henry S. Thompson <ht@...> wrote:

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

XMLSchema at XML4Pharma writes:

> Our extension mechanism is based on the "import" and "redefine"
> elements of XML-Schema.

> [see] http://www.altova.com/forum/default.aspx?g=posts&m=1000005665

> We now want to escalate the issue to the W3C itself, and would like
> to know what the mechanism is to do so.

We can only try to clarify and persuade -- we have no authority to
compel anyone.

I can only explain how/why XSV behaves as it does, and offer my
understand of the spec.

The spec. allows (note, _not_ requires) implementations to ignore
attempts to import schema documents for namespaces for which a schema
document is already known:

 "Given that the schemaLocation [attribute] is only a hint, it is
  open to applications to ignore all but the first <import> for a
  given namespace, regardless of the ·actual value· of
  schemaLocation, but such a strategy risks missing useful
  information when new schemaLocations are offered." [1]

XSV follows this strategy.  In the case of the CDISC schemas, this
means the xs:import of ODM1-2-1.xsd is skipped, because a schema
document (namely define-1.0.0.xsd) is already known for the odm/v1.2
namespace.

In the case of the Altova example, this means the xs:import of
XSD2.xsd is skipped, because a schema document (namely XSD1.xsd) is
already known for the no-target-namespace case.

In other words, in neither case is the behaviour of XSD to do with how
it resolves redefine/import conflicts.

But note that any schema processor which _does_ follow all
schemaLocation hints, and therefore does import multiple documents for
the same target namespace, is perfectly conformant as well.

So, what about redefine/import conflicts?

The spec. contains no explicit guidance.  Furthermore, it is impossible to
construct an example which would provoke such a conflict in XSV,
because it would require two imports of the same targetNamespace (the
reason for this is left as an exercise for the reader -- if I've
missed a way to bring this about, I owe the first person to point out
how a beer :-).

We can however construct an example of conflicting redefine/include in
XSV, by making small modifications to the XSD3.xsd document given in
the Altova example:

XSD3.xsd
<xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema"
 elementFormDefault="qualified" attributeFormDefault="unqualified"
 targetNamespace="http://XSD3" xmlns:x="http://XSD3">
<xs:include schemaLocation="XSD2.xsd"/>
<xs:include schemaLocation="XSD1.xsd"/>
<xs:element name="elt" type="x:CT"/>
</xs:schema>

We're now chameleon including/redefining, and the result works with
XSV 3.1, regardless of the order of the two include statements.  By
'works', I mean
 a) The schema is judged to be OK;
 b) The instance is judged to be bad.

I read the spec. as _requiring_ this behaviour, because it says

 "The modifications have a pervasive impact, that is, only the
  redefined components are used, even when referenced from other
  incorporated components, whether redefined themselves or not." [2]

What this means is that no conflict occurs between the redefined and
non-redefined components, because the later are not part of the schema
used for validation.  The same argument seems to me to apply to the
redefine/import case.

I would be interested to know how other products behave with the
redefine/include conflict example given above.

Summary: I believe the spec. intends the redefine semantics to 'trump'
the include and import semantics wrt which components are present in
cases of conflict, but I do not disagree that this is not so clear as
to be uncontestable.

I also think it is unfortunate that all implementors cannot agree on a
single interpretation.  If it is true, as alleged, that the situation
is that many implementations agree on the interpretation but one does
not, that's particularly unfortunate.

ht

[1] http://www.w3.org/TR/xmlschema-1/#src-import
[1] http://www.w3.org/TR/xmlschema-1/#modify-schema
- --
      Henry S. Thompson, School of Informatics, University of Edinburgh
                        Half-time member of W3C Team
     10 Crichton Street, Edinburgh EH8 9AB, SCOTLAND -- (44) 131 650-4440
               Fax: [phone], e-mail: ht@...
                      URL: http://www.ltg.ed.ac.uk/~ht/
[mail really from me _always_ has this .sig -- mail without it is forged spam]
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.6 (GNU/Linux)

iD8DBQFKwh4EkjnJixAXWBoRAsVuAJ9PFjtJMUNAycnVwR3WlFIrJVCI3wCfVeQX
c0UQGS14hYnXrTq2lW2ADVE=
=m9RM
-----END PGP SIGNATURE-----

[noah_mendelsohn]

I strongly urge you to read Michael Sperberg-McQueen's note carefully.
While, as noted below, some of my preferences for resolving ambiguities
happen to be different from his, I believe that his email very accurately
and carefully summarizes the state of play on this issue.  Let me just
quote and comment on a few sections of Michael's note:

> The XSD 1.0 specification does not explicitly address the case
> when the same schema document in referenced both directly
> (via import or include) and indirectly (via redefine).  The
> rules for import specify that the components imported
> via the import or include should be present in the resulting
> schema; the discussion of redefine says that the effect of
> redefine is pervasive.

Yes, that's what it says, and it's contradictory.

> Some readers interpret this to mean that the pervasive-redefinition
> rule overrides, or modifies the effect of, the import and include
> rules.  On this reading the schema is legal and the components
> are present in their redefined form (and only that form).  This
> reading effectively attaches to the rules for include and import
> an unspoken exception to the effect that the components imported
> or included are present in the resulting schema unless they are
> redefined by some other schema document reference.
>
> Other readers interpret the rules as requiring that the
> components in question should be present both in their
> original and in their redefined form, which means that the
> schema violates the rule that there must be at most one
> component of any kind with a given expanded name, and thus
> that the schema should be rejected.

Yes, those are at least two of the widely held positions as to what was
"really" intended, and I don't think I'm aware of any others.

> It is my personal belief that the most plausible interpretation
> of the specification is the latter; as you have observed, at
> least some implementors disagree and prefer the other
> interpretation.

I happen to be one of the others who would prefer that the redefine be
pervasive, but the important point here is that either interpretation is
plausible, and we agree on that.

> The inability of the XML Schema working group to agree on a
> normative interpretation of XSD 1.0 has led to the agreement to
> deprecate the redefine element in XSD 1.1.

Here, I'd like to add one bit.  While Michael is correct that the working
group has "agreed" on this, some of us joined in that agreement with some
hesitancy, because we believe that <redefine> has seen widespread use,
including in cases that do not trigger (or at least do not as clearly
trigger) the ambiguities that are causing trouble here.  So, in light of
those concerns, the working group also agreed to make the deprecation of
<redefine> a so-called Priority Feedback issue.  Quoting from the working
draft [1]:

----
Note: The redefinition feature described in the remainder of this section
is ·deprecated· and may be removed from future versions of this
specification. Schema authors are encouraged to avoid its use in cases
where interoperability or compatibility with later versions of this
specification are important.

Editorial Note: Priority Feedback Request

The Working Group requests feedback from readers, schema authors,
implementors, and other users of this specification as to the desirability
of retaining, removing, deprecating, or not deprecating the use of
<redefine>. Since the <override> facility provides similar functionality
but does not require a restriction or extension relation between the new
and the old definitions of redefined components, the Working Group is
particularly interested in learning whether users of this specification
find that requirement useful or not.
----

So, if any readers of this thread have opinions on the plan to deprecate,
the Schema Working group would welcome hearing about them.  I think it's
worth noting that this thread has already made clear that: 1) this is a
known area of complexity and the working group has already tried and so
far failed to find an easy resolution acceptable to all; 2) there are
incompatibilities in widely deployed implementations, so any clear
resolution is quite likely to make someone with an investment in code very
unhappy.  That's not to say I wouldn't like it cleaned up;  indeed, I'm
among those who put many months into trying a few years ago (as did
Michael). I'm just pointing out that our choices may be to deprecate or
undeprecate, but going further to remove the ambiguity is likely to be a
significant effort that will introduce incompatibilities for at least
somebody.  Maybe or maybe not we could find a way to warn people off from
the most troublesome cases, but I know that Michael and perhaps others
will correctly point out that the whole conceptual framework for this
"composition" function is sufficiently shaky in the current drafts that
any fix short of a complete rewrite is likely to leave things in a messy
state.

Noah

[1] http://www.w3.org/TR/xmlschema11-1/#modify-schema

--------------------------------------
Noah Mendelsohn
IBM Corporation
One Rogers Street
Cambridge, MA 02142
1-617-693-4036
--------------------------------------

[noah_mendelsohn]

Just the sort of feedback we need, thank you!  It's quite possible that my
intuition on this is wrong.

Noah

--------------------------------------
Noah Mendelsohn
IBM Corporation
One Rogers Street
Cambridge, MA 02142
1-617-693-4036
--------------------------------------








Boris Kolpackov <boris@...>
10/01/2009 11:57 AM
 
        To:     noah_mendelsohn@...
        cc:     XMLSchema at XML4Pharma <XMLSchema@...>, "C. M.
Sperberg-McQueen" <cmsmcq@...>, xmlschema-dev@...
        Subject:        Re: Escalation mechanism for different
interpretation of W3C   XML-Schema  specification ?


Hi,

noah_mendelsohn@... <noah_mendelsohn@...> writes:

> because we believe that <redefine> has seen widespread use,

I have a completely opposite experience. That is, I have seen
people trying to use redefine, quickly finding that no two
processors handle it the same way, and giving up.

We also have a fairly large XML Schema repository which includes
hundreds of schemas for various public and proprietary real-world
vocabularies. I just did a quick check and the repository contains
over 2,000 schema files. Only one vocabulary uses redefine.


> So, if any readers of this thread have opinions on the plan to
deprecate,
> the Schema Working group would welcome hearing about them.

I am strongly for depreciating redefine (as well as inheritance by
restriction, while we are at it ;-)).

Boris

--
Boris Kolpackov, Code Synthesis Tools  
http://codesynthesis.com/~boris/blog
Open-source XML data binding for C++:  
http://codesynthesis.com/products/xsd
XML data binding for embedded systems:
http://codesynthesis.com/products/xsde

[Boris Kolpackov-2]

Hi,

noah_mendelsohn@... <noah_mendelsohn@...> writes:

> because we believe that <redefine> has seen widespread use,

I have a completely opposite experience. That is, I have seen
people trying to use redefine, quickly finding that no two
processors handle it the same way, and giving up.

We also have a fairly large XML Schema repository which includes
hundreds of schemas for various public and proprietary real-world
vocabularies. I just did a quick check and the repository contains
over 2,000 schema files. Only one vocabulary uses redefine.


> So, if any readers of this thread have opinions on the plan to deprecate,
> the Schema Working group would welcome hearing about them.

I am strongly for depreciating redefine (as well as inheritance by
restriction, while we are at it ;-)).

Boris

--
Boris Kolpackov, Code Synthesis Tools  http://codesynthesis.com/~boris/blog
Open-source XML data binding for C++:  http://codesynthesis.com/products/xsd
XML data binding for embedded systems: http://codesynthesis.com/products/xsde

[Michael Kay]

> I am strongly for depreciating redefine (as well as inheritance by
> restriction, while we are at it ;-)).

I wouldn't like to see that deprecated, but as a matter of good practice,
once people adopt XSD 1.1 I shall probably be advising them always to do
restriction of complex content models by means of assertions.

Michael Kay
Saxonica

[Michael Kay]

Some parts of this message have been removed. Learn more about Nabble's security policy.

Just for the record (though it only really confirms the point that there are differing interpretations), I implemented the XSD 1.0 specification without the benefit of knowing anything about the intent of the WG beyond what was written in the spec, and I came to the conclusion that there was only one way of interpreting the rule about xsd:redefine being "pervasive". But then I also came to the conclusion that other parts of the schema composition rules had to be treated as loose statements of intent which didn't cover all contingencies.

 

I would certainly advise against using xs:redefine in an industry schema specification. Its only possible justification in my view is to define a variant of a schema produced by a third party. And for that use case, I think it only works unambiguously if that schema uses a single target namespace throughout.

Michael Kay

Saxonica

---

From: xmlschema-dev-request@... [mailto:xmlschema-dev-request@...] On Behalf Of XMLSchema at XML4Pharma
Sent: 29 September 2009 12:24
To: xmlschema-dev@...
Subject: Escalation mechanism for different interpretation of W3C XML-Schema specification ?

We, the CDISC XML-Tech Governance Team (and other CDISC teams) have developed a number of extensible standards for exchange of clinical data and for submitting information to the regulatory authorities (FDA).

CDISC is a Standardization Organization active in the healthcare world.

Our extension mechanism is based on the “import” and “redefine” elements of XML-Schema.

We now have a serious dispute with one technology vendor (Altova) about the way “import” and “redefine” are used. Instance files of one of our extensions (so-called “define.xml”) validate well in all major validators and XML-editors, except for the products of this one vendor.

When confronted with this result, the reaction of Altova essentially is that “Altova is right, all others are wrong”. The dispute and discussion with Altova can be followed at:
http://www.altova.com/forum/default.aspx?g=posts&m=1000005665

The issue were not so serious if it were not that our standard “define.xml” is a standard for submission of information to the regulatory authorities, and these are (mostly) using the Altova product for validation.

We now want to escalate the issue to the W3C itself, and would like to know what the mechanism is to do so.

Jozef Aerts
CDISC XML-Tech Governance Team

[XMLSchema at XML4Pharma]

Some parts of this message have been removed. Learn more about Nabble's security policy.

Thanks Michael,

 

About the justification you are mentioning: that is exactly what our standard is about: allowing companies (but also the FDA) to add extra elements and attributes in a separate namespace.

 

The whole discussion is a trigger for us to move to XML-Schema 1.1 earlier than we envisaged. Problem here is that, except for Saxon, there do seem not to be very much tools (editors, libraries, etc.) that allow validation of XML files against XML-Schema 1.1.

Is there a list somewhere of tools that do so, or a list of vendors and when they intend to have Schema-1.1 validation ready ?

 

I will make a separate mailing with some comments/remarks/suggestions about escalation of issues, certification mechanisms, etc..

 

Jozef Aerts

XML4Pharma

----- Original Message -----

From: mike@...

To: XMLSchema@... ; xmlschema-dev@...

Sent: Friday, October 02, 2009 12:53 AM

Subject: RE: Escalation mechanism for different interpretation of W3C XML-Schema specification ?

Just for the record (though it only really confirms the point that there are differing interpretations), I implemented the XSD 1.0 specification without the benefit of knowing anything about the intent of the WG beyond what was written in the spec, and I came to the conclusion that there was only one way of interpreting the rule about xsd:redefine being "pervasive". But then I also came to the conclusion that other parts of the schema composition rules had to be treated as loose statements of intent which didn't cover all contingencies.

 

I would certainly advise against using xs:redefine in an industry schema specification. Its only possible justification in my view is to define a variant of a schema produced by a third party. And for that use case, I think it only works unambiguously if that schema uses a single target namespace throughout.

Michael Kay

Saxonica

---

From: xmlschema-dev-request@... [mailto:xmlschema-dev-request@...] On Behalf Of XMLSchema at XML4Pharma
Sent: 29 September 2009 12:24
To: xmlschema-dev@...
Subject: Escalation mechanism for different interpretation of W3C XML-Schema specification ?

We, the CDISC XML-Tech Governance Team (and other CDISC teams) have developed a number of extensible standards for exchange of clinical data and for submitting information to the regulatory authorities (FDA).

CDISC is a Standardization Organization active in the healthcare world.

Our extension mechanism is based on the “import” and “redefine” elements of XML-Schema.

We now have a serious dispute with one technology vendor (Altova) about the way “import” and “redefine” are used. Instance files of one of our extensions (so-called “define.xml”) validate well in all major validators and XML-editors, except for the products of this one vendor.

When confronted with this result, the reaction of Altova essentially is that “Altova is right, all others are wrong”. The dispute and discussion with Altova can be followed at:
http://www.altova.com/forum/default.aspx?g=posts&m=1000005665

The issue were not so serious if it were not that our standard “define.xml” is a standard for submission of information to the regulatory authorities, and these are (mostly) using the Altova product for validation.

We now want to escalate the issue to the W3C itself, and would like to know what the mechanism is to do so.

Jozef Aerts
CDISC XML-Tech Governance Team

[DSosnoski]

Hi Noah,

IMHO both xs:redefine and xs:complexType restriction should be
eliminated. With respect to the increasingly important area of XML data
binding for programming languages, redefine and complexType restriction
are generally ignored for the simple reason that they have no equivalent
in terms of data models. The object oriented programming equivalent of
complexType restriction would be to define a subclass by eliminating
information from the parent class, which is a bizarre concept in
programming terms - if you want a limited representation which is not
supported by the current data model you either refactor the data model
to support the particular subset you want or use a representation which
has more features than you need, with semantic constraints on the usage.
There are times when this limitation of object oriented programming is
inconvenient, but I suspect the vast majority of developers would agree
that the benefits in terms of clean data models more than outweigh the
inconvenience.

Beyond this, the very nature of complexType restriction, which requires
repeating all portions of the original model which are being retained,
creates brittle definitions.

redefine is an even stranger operation, effectively performing surgery
on an existing definition to deform it in ways never intended by the
original designers. One consequence is that anyone looking at an
instance of an XML structure defined by a schema needs to be aware of
other schemas which may have modified the original schema definition.
Requiring knowledge of the entire set of schemas involved in a document
in order to understand one particular component of the document is
contrary to good design principles.

It's hard for me to see how xs:override represents any kind of
improvement on redefine, so asking for feedback on deprecating redefine
while adding override into the mix seems deliberately futile. From my
point of view override suffers from the same design failures as
redefine, and should also be eliminated.

I realize that many uses of W3C schema are not concerned with data
binding, but I'd suggest that the difficulty of modeling these
constructs in terms of modern programming language structures shows that
they could be eliminated without significantly harming the usefulness of
schema.

  - Dennis

Dennis M. Sosnoski
XML and Web Services in Java
Training and Consulting
http://www.sosnoski.com - http://www.sosnoski.co.nz
Seattle, WA [phone] - Wellington, NZ [phone]



noah_mendelsohn@... wrote:

> I strongly urge you to read Michael Sperberg-McQueen's note carefully.
> While, as noted below, some of my preferences for resolving ambiguities
> happen to be different from his, I believe that his email very accurately
> and carefully summarizes the state of play on this issue.  Let me just
> quote and comment on a few sections of Michael's note:
>
>  
>> The XSD 1.0 specification does not explicitly address the case
>> when the same schema document in referenced both directly
>> (via import or include) and indirectly (via redefine).  The
>> rules for import specify that the components imported
>> via the import or include should be present in the resulting
>> schema; the discussion of redefine says that the effect of
>> redefine is pervasive.
>>    
>
> Yes, that's what it says, and it's contradictory.
>
>  
>> Some readers interpret this to mean that the pervasive-redefinition
>> rule overrides, or modifies the effect of, the import and include
>> rules.  On this reading the schema is legal and the components
>> are present in their redefined form (and only that form).  This
>> reading effectively attaches to the rules for include and import
>> an unspoken exception to the effect that the components imported
>> or included are present in the resulting schema unless they are
>> redefined by some other schema document reference.
>>
>> Other readers interpret the rules as requiring that the
>> components in question should be present both in their
>> original and in their redefined form, which means that the
>> schema violates the rule that there must be at most one
>> component of any kind with a given expanded name, and thus
>> that the schema should be rejected.
>>    
>
> Yes, those are at least two of the widely held positions as to what was
> "really" intended, and I don't think I'm aware of any others.
>
>  
>> It is my personal belief that the most plausible interpretation
>> of the specification is the latter; as you have observed, at
>> least some implementors disagree and prefer the other
>> interpretation.
>>    
>
> I happen to be one of the others who would prefer that the redefine be
> pervasive, but the important point here is that either interpretation is
> plausible, and we agree on that.
>
>  
>> The inability of the XML Schema working group to agree on a
>> normative interpretation of XSD 1.0 has led to the agreement to
>> deprecate the redefine element in XSD 1.1.
>>    
>
> Here, I'd like to add one bit.  While Michael is correct that the working
> group has "agreed" on this, some of us joined in that agreement with some
> hesitancy, because we believe that <redefine> has seen widespread use,
> including in cases that do not trigger (or at least do not as clearly
> trigger) the ambiguities that are causing trouble here.  So, in light of
> those concerns, the working group also agreed to make the deprecation of
> <redefine> a so-called Priority Feedback issue.  Quoting from the working
> draft [1]:
>
> ----
> Note: The redefinition feature described in the remainder of this section
> is ·deprecated· and may be removed from future versions of this
> specification. Schema authors are encouraged to avoid its use in cases
> where interoperability or compatibility with later versions of this
> specification are important.
>
> Editorial Note: Priority Feedback Request
>
> The Working Group requests feedback from readers, schema authors,
> implementors, and other users of this specification as to the desirability
> of retaining, removing, deprecating, or not deprecating the use of
> <redefine>. Since the <override> facility provides similar functionality
> but does not require a restriction or extension relation between the new
> and the old definitions of redefined components, the Working Group is
> particularly interested in learning whether users of this specification
> find that requirement useful or not.
> ----
>
> So, if any readers of this thread have opinions on the plan to deprecate,
> the Schema Working group would welcome hearing about them.  I think it's
> worth noting that this thread has already made clear that: 1) this is a
> known area of complexity and the working group has already tried and so
> far failed to find an easy resolution acceptable to all; 2) there are
> incompatibilities in widely deployed implementations, so any clear
> resolution is quite likely to make someone with an investment in code very
> unhappy.  That's not to say I wouldn't like it cleaned up;  indeed, I'm
> among those who put many months into trying a few years ago (as did
> Michael). I'm just pointing out that our choices may be to deprecate or
> undeprecate, but going further to remove the ambiguity is likely to be a
> significant effort that will introduce incompatibilities for at least
> somebody.  Maybe or maybe not we could find a way to warn people off from
> the most troublesome cases, but I know that Michael and perhaps others
> will correctly point out that the whole conceptual framework for this
> "composition" function is sufficiently shaky in the current drafts that
> any fix short of a complete rewrite is likely to leave things in a messy
> state.
>
> Noah
>
> [1] http://www.w3.org/TR/xmlschema11-1/#modify-schema
>
> --------------------------------------
> Noah Mendelsohn
> IBM Corporation
> One Rogers Street
> Cambridge, MA 02142
> 1-617-693-4036
> --------------------------------------
>
>
>
>
>
>
>  

[Michael Kay]

While I think many of your criticisms of xs:redefine and complex type
restriction and indeed xs:override are valid, I personally don't accept the
premise that XSD should be constrained or influenced by the data typing
paradigms of conventional programming languages. XML and XSD are first and
formost intended for designing hierarchic document structures (a discipline
with a far longer tradition than programming), and the fact that
conventional programming languages don't support such structures very well
means in my view that it's best to switch to languages that do.

Regards,

Michael Kay
http://www.saxonica.com/
http://twitter.com/michaelhkay 
 

> -----Original Message-----
> From: xmlschema-dev-request@...
> [mailto:xmlschema-dev-request@...] On Behalf Of Dennis Sosnoski
> Sent: 02 October 2009 23:10
> To: noah_mendelsohn@...
> Cc: XMLSchema at XML4Pharma; C. M. Sperberg-McQueen;
> xmlschema-dev@...
> Subject: Re: Escalation mechanism for different
> interpretation of W3C XML-Schema specification ?
>
> Hi Noah,
>
> IMHO both xs:redefine and xs:complexType restriction should
> be eliminated. With respect to the increasingly important
> area of XML data binding for programming languages, redefine
> and complexType restriction are generally ignored for the
> simple reason that they have no equivalent in terms of data
> models. The object oriented programming equivalent of
> complexType restriction would be to define a subclass by
> eliminating information from the parent class, which is a
> bizarre concept in programming terms - if you want a limited
> representation which is not supported by the current data
> model you either refactor the data model to support the
> particular subset you want or use a representation which has
> more features than you need, with semantic constraints on the usage.
> There are times when this limitation of object oriented
> programming is inconvenient, but I suspect the vast majority
> of developers would agree that the benefits in terms of clean
> data models more than outweigh the inconvenience.
>
> Beyond this, the very nature of complexType restriction,
> which requires repeating all portions of the original model
> which are being retained, creates brittle definitions.
>
> redefine is an even stranger operation, effectively
> performing surgery on an existing definition to deform it in
> ways never intended by the original designers. One
> consequence is that anyone looking at an instance of an XML
> structure defined by a schema needs to be aware of other
> schemas which may have modified the original schema definition.
> Requiring knowledge of the entire set of schemas involved in
> a document in order to understand one particular component of
> the document is contrary to good design principles.
>
> It's hard for me to see how xs:override represents any kind
> of improvement on redefine, so asking for feedback on
> deprecating redefine while adding override into the mix seems
> deliberately futile. From my point of view override suffers
> from the same design failures as redefine, and should also be
> eliminated.
>
> I realize that many uses of W3C schema are not concerned with
> data binding, but I'd suggest that the difficulty of modeling
> these constructs in terms of modern programming language
> structures shows that they could be eliminated without
> significantly harming the usefulness of schema.
>
>   - Dennis
>
> Dennis M. Sosnoski
> XML and Web Services in Java
> Training and Consulting
> http://www.sosnoski.com - http://www.sosnoski.co.nz Seattle,
> WA [phone] - Wellington, NZ [phone]
>
>
>
> noah_mendelsohn@... wrote:
> > I strongly urge you to read Michael Sperberg-McQueen's note
> carefully.
> > While, as noted below, some of my preferences for resolving
> > ambiguities happen to be different from his, I believe that
> his email
> > very accurately and carefully summarizes the state of play on this
> > issue.  Let me just quote and comment on a few sections of
> Michael's note:
> >
> >  
> >> The XSD 1.0 specification does not explicitly address the
> case when
> >> the same schema document in referenced both directly (via
> import or
> >> include) and indirectly (via redefine).  The rules for
> import specify
> >> that the components imported via the import or include should be
> >> present in the resulting schema; the discussion of
> redefine says that
> >> the effect of redefine is pervasive.
> >>    
> >
> > Yes, that's what it says, and it's contradictory.
> >
> >  
> >> Some readers interpret this to mean that the
> pervasive-redefinition
> >> rule overrides, or modifies the effect of, the import and include
> >> rules.  On this reading the schema is legal and the components are
> >> present in their redefined form (and only that form).  
> This reading
> >> effectively attaches to the rules for include and import
> an unspoken
> >> exception to the effect that the components imported or
> included are
> >> present in the resulting schema unless they are redefined by some
> >> other schema document reference.
> >>
> >> Other readers interpret the rules as requiring that the
> components in
> >> question should be present both in their original and in their
> >> redefined form, which means that the schema violates the rule that
> >> there must be at most one component of any kind with a
> given expanded
> >> name, and thus that the schema should be rejected.
> >>    
> >
> > Yes, those are at least two of the widely held positions as to what
> > was "really" intended, and I don't think I'm aware of any others.
> >
> >  
> >> It is my personal belief that the most plausible interpretation of
> >> the specification is the latter; as you have observed, at
> least some
> >> implementors disagree and prefer the other interpretation.
> >>    
> >
> > I happen to be one of the others who would prefer that the
> redefine be
> > pervasive, but the important point here is that either
> interpretation
> > is plausible, and we agree on that.
> >
> >  
> >> The inability of the XML Schema working group to agree on
> a normative
> >> interpretation of XSD 1.0 has led to the agreement to
> deprecate the
> >> redefine element in XSD 1.1.
> >>    
> >
> > Here, I'd like to add one bit.  While Michael is correct that the
> > working group has "agreed" on this, some of us joined in that
> > agreement with some hesitancy, because we believe that
> <redefine> has
> > seen widespread use, including in cases that do not trigger (or at
> > least do not as clearly
> > trigger) the ambiguities that are causing trouble here.  
> So, in light
> > of those concerns, the working group also agreed to make the
> > deprecation of <redefine> a so-called Priority Feedback issue.  
> > Quoting from the working draft [1]:
> >
> > ----
> > Note: The redefinition feature described in the remainder of this
> > section is .deprecated. and may be removed from future versions of
> > this specification. Schema authors are encouraged to avoid
> its use in
> > cases where interoperability or compatibility with later
> versions of
> > this specification are important.
> >
> > Editorial Note: Priority Feedback Request
> >
> > The Working Group requests feedback from readers, schema authors,
> > implementors, and other users of this specification as to the
> > desirability of retaining, removing, deprecating, or not
> deprecating
> > the use of <redefine>. Since the <override> facility
> provides similar
> > functionality but does not require a restriction or
> extension relation
> > between the new and the old definitions of redefined
> components, the
> > Working Group is particularly interested in learning
> whether users of
> > this specification find that requirement useful or not.
> > ----
> >
> > So, if any readers of this thread have opinions on the plan to
> > deprecate, the Schema Working group would welcome hearing
> about them.  
> > I think it's worth noting that this thread has already made clear
> > that: 1) this is a known area of complexity and the working
> group has
> > already tried and so far failed to find an easy resolution
> acceptable
> > to all; 2) there are incompatibilities in widely deployed
> > implementations, so any clear resolution is quite likely to make
> > someone with an investment in code very unhappy.  That's
> not to say I
> > wouldn't like it cleaned up;  indeed, I'm among those who put many
> > months into trying a few years ago (as did Michael). I'm
> just pointing
> > out that our choices may be to deprecate or undeprecate, but going
> > further to remove the ambiguity is likely to be a
> significant effort
> > that will introduce incompatibilities for at least
> somebody.  Maybe or
> > maybe not we could find a way to warn people off from the most
> > troublesome cases, but I know that Michael and perhaps others will
> > correctly point out that the whole conceptual framework for this
> > "composition" function is sufficiently shaky in the current drafts
> > that any fix short of a complete rewrite is likely to leave
> things in a messy state.
> >
> > Noah
> >
> > [1] http://www.w3.org/TR/xmlschema11-1/#modify-schema
> >
> > --------------------------------------
> > Noah Mendelsohn
> > IBM Corporation
> > One Rogers Street
> > Cambridge, MA 02142
> > 1-617-693-4036
> > --------------------------------------
> >
> >
> >
> >
> >
> >
> >  
>

[XML4Pharma]

In our extension schemas, we only use "redefine" to ADD extra attributes and
elements (in a separate namespace), not to change or remove any existing
ones.

One of my positive surprises with the way we implemented it, is that a XML
data binding like XMLBeans had no problem at all working with it: it was
very easy to derive all classes from our schemas including the "redefine".

Dennis suggested to have a look at JSON, which I did.
This might be a very good idea for a format for submitting information/data
to the FDA.
The problem at the FDA is that there is almost noone there with a basic
understanding of XML, and those who have left the FDA to become a well-paid
consultant. Some departments at the FDA have the greatest difficulty to
validate even very simple XML documents.
The FDA is currently planning to switch to HL7-v3-XML messages for
electronic submissions of data to the agency. As HL7-v3-XML is highly
complex (and even not good XML i.m.o.) I think this will lead to disaster.
So JSON may be a good alternative.
I had a quick look, but JSON does not have some data types like date, time
and dateTime. Would be nice if they had ...

Best,

Jozef Aerts
XML4Pharma


----- Original Message -----
From: "Michael Kay" <mike@...>
To: "'Dennis Sosnoski'" <dms@...>; <noah_mendelsohn@...>
Cc: "'XMLSchema at XML4Pharma'" <XMLSchema@...>; "'C. M.
Sperberg-McQueen'" <cmsmcq@...>; <xmlschema-dev@...>
Sent: Saturday, October 03, 2009 12:43 AM
Subject: RE: Escalation mechanism for different interpretation of W3C
XML-Schema specification ?

>
> While I think many of your criticisms of xs:redefine and complex type
> restriction and indeed xs:override are valid, I personally don't accept
> the
> premise that XSD should be constrained or influenced by the data typing
> paradigms of conventional programming languages. XML and XSD are first and
> formost intended for designing hierarchic document structures (a
> discipline
> with a far longer tradition than programming), and the fact that
> conventional programming languages don't support such structures very well
> means in my view that it's best to switch to languages that do.
>
> Regards,
>
> Michael Kay
> http://www.saxonica.com/
> http://twitter.com/michaelhkay
>
>
>> -----Original Message-----
>> From: xmlschema-dev-request@...
>> [mailto:xmlschema-dev-request@...] On Behalf Of Dennis Sosnoski
>> Sent: 02 October 2009 23:10
>> To: noah_mendelsohn@...
>> Cc: XMLSchema at XML4Pharma; C. M. Sperberg-McQueen;
>> xmlschema-dev@...
>> Subject: Re: Escalation mechanism for different
>> interpretation of W3C XML-Schema specification ?
>>
>> Hi Noah,
>>
>> IMHO both xs:redefine and xs:complexType restriction should
>> be eliminated. With respect to the increasingly important
>> area of XML data binding for programming languages, redefine
>> and complexType restriction are generally ignored for the
>> simple reason that they have no equivalent in terms of data
>> models. The object oriented programming equivalent of
>> complexType restriction would be to define a subclass by
>> eliminating information from the parent class, which is a
>> bizarre concept in programming terms - if you want a limited
>> representation which is not supported by the current data
>> model you either refactor the data model to support the
>> particular subset you want or use a representation which has
>> more features than you need, with semantic constraints on the usage.
>> There are times when this limitation of object oriented
>> programming is inconvenient, but I suspect the vast majority
>> of developers would agree that the benefits in terms of clean
>> data models more than outweigh the inconvenience.
>>
>> Beyond this, the very nature of complexType restriction,
>> which requires repeating all portions of the original model
>> which are being retained, creates brittle definitions.
>>
>> redefine is an even stranger operation, effectively
>> performing surgery on an existing definition to deform it in
>> ways never intended by the original designers. One
>> consequence is that anyone looking at an instance of an XML
>> structure defined by a schema needs to be aware of other
>> schemas which may have modified the original schema definition.
>> Requiring knowledge of the entire set of schemas involved in
>> a document in order to understand one particular component of
>> the document is contrary to good design principles.
>>
>> It's hard for me to see how xs:override represents any kind
>> of improvement on redefine, so asking for feedback on
>> deprecating redefine while adding override into the mix seems
>> deliberately futile. From my point of view override suffers
>> from the same design failures as redefine, and should also be
>> eliminated.
>>
>> I realize that many uses of W3C schema are not concerned with
>> data binding, but I'd suggest that the difficulty of modeling
>> these constructs in terms of modern programming language
>> structures shows that they could be eliminated without
>> significantly harming the usefulness of schema.
>>
>>   - Dennis
>>
>> Dennis M. Sosnoski
>> XML and Web Services in Java
>> Training and Consulting
>> http://www.sosnoski.com - http://www.sosnoski.co.nz Seattle,
>> WA [phone] - Wellington, NZ [phone]
>>
>>
>>
>> noah_mendelsohn@... wrote:
>> > I strongly urge you to read Michael Sperberg-McQueen's note
>> carefully.
>> > While, as noted below, some of my preferences for resolving
>> > ambiguities happen to be different from his, I believe that
>> his email
>> > very accurately and carefully summarizes the state of play on this
>> > issue.  Let me just quote and comment on a few sections of
>> Michael's note:
>> >
>> >
>> >> The XSD 1.0 specification does not explicitly address the
>> case when
>> >> the same schema document in referenced both directly (via
>> import or
>> >> include) and indirectly (via redefine).  The rules for
>> import specify
>> >> that the components imported via the import or include should be
>> >> present in the resulting schema; the discussion of
>> redefine says that
>> >> the effect of redefine is pervasive.
>> >>
>> >
>> > Yes, that's what it says, and it's contradictory.
>> >
>> >
>> >> Some readers interpret this to mean that the
>> pervasive-redefinition
>> >> rule overrides, or modifies the effect of, the import and include
>> >> rules.  On this reading the schema is legal and the components are
>> >> present in their redefined form (and only that form).
>> This reading
>> >> effectively attaches to the rules for include and import
>> an unspoken
>> >> exception to the effect that the components imported or
>> included are
>> >> present in the resulting schema unless they are redefined by some
>> >> other schema document reference.
>> >>
>> >> Other readers interpret the rules as requiring that the
>> components in
>> >> question should be present both in their original and in their
>> >> redefined form, which means that the schema violates the rule that
>> >> there must be at most one component of any kind with a
>> given expanded
>> >> name, and thus that the schema should be rejected.
>> >>
>> >
>> > Yes, those are at least two of the widely held positions as to what
>> > was "really" intended, and I don't think I'm aware of any others.
>> >
>> >
>> >> It is my personal belief that the most plausible interpretation of
>> >> the specification is the latter; as you have observed, at
>> least some
>> >> implementors disagree and prefer the other interpretation.
>> >>
>> >
>> > I happen to be one of the others who would prefer that the
>> redefine be
>> > pervasive, but the important point here is that either
>> interpretation
>> > is plausible, and we agree on that.
>> >
>> >
>> >> The inability of the XML Schema working group to agree on
>> a normative
>> >> interpretation of XSD 1.0 has led to the agreement to
>> deprecate the
>> >> redefine element in XSD 1.1.
>> >>
>> >
>> > Here, I'd like to add one bit.  While Michael is correct that the
>> > working group has "agreed" on this, some of us joined in that
>> > agreement with some hesitancy, because we believe that
>> <redefine> has
>> > seen widespread use, including in cases that do not trigger (or at
>> > least do not as clearly
>> > trigger) the ambiguities that are causing trouble here.
>> So, in light
>> > of those concerns, the working group also agreed to make the
>> > deprecation of <redefine> a so-called Priority Feedback issue.
>> > Quoting from the working draft [1]:
>> >
>> > ----
>> > Note: The redefinition feature described in the remainder of this
>> > section is .deprecated. and may be removed from future versions of
>> > this specification. Schema authors are encouraged to avoid
>> its use in
>> > cases where interoperability or compatibility with later
>> versions of
>> > this specification are important.
>> >
>> > Editorial Note: Priority Feedback Request
>> >
>> > The Working Group requests feedback from readers, schema authors,
>> > implementors, and other users of this specification as to the
>> > desirability of retaining, removing, deprecating, or not
>> deprecating
>> > the use of <redefine>. Since the <override> facility
>> provides similar
>> > functionality but does not require a restriction or
>> extension relation
>> > between the new and the old definitions of redefined
>> components, the
>> > Working Group is particularly interested in learning
>> whether users of
>> > this specification find that requirement useful or not.
>> > ----
>> >
>> > So, if any readers of this thread have opinions on the plan to
>> > deprecate, the Schema Working group would welcome hearing
>> about them.
>> > I think it's worth noting that this thread has already made clear
>> > that: 1) this is a known area of complexity and the working
>> group has
>> > already tried and so far failed to find an easy resolution
>> acceptable
>> > to all; 2) there are incompatibilities in widely deployed
>> > implementations, so any clear resolution is quite likely to make
>> > someone with an investment in code very unhappy.  That's
>> not to say I
>> > wouldn't like it cleaned up;  indeed, I'm among those who put many
>> > months into trying a few years ago (as did Michael). I'm
>> just pointing
>> > out that our choices may be to deprecate or undeprecate, but going
>> > further to remove the ambiguity is likely to be a
>> significant effort
>> > that will introduce incompatibilities for at least
>> somebody.  Maybe or
>> > maybe not we could find a way to warn people off from the most
>> > troublesome cases, but I know that Michael and perhaps others will
>> > correctly point out that the whole conceptual framework for this
>> > "composition" function is sufficiently shaky in the current drafts
>> > that any fix short of a complete rewrite is likely to leave
>> things in a messy state.
>> >
>> > Noah
>> >
>> > [1] http://www.w3.org/TR/xmlschema11-1/#modify-schema
>> >
>> > --------------------------------------
>> > Noah Mendelsohn
>> > IBM Corporation
>> > One Rogers Street
>> > Cambridge, MA 02142
>> > 1-617-693-4036
>> > --------------------------------------
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>>
>
>

[noah_mendelsohn]

Jozef Aerts writes:

> So JSON may be a good alternative.

I would never discourage you from looking for the best language to meet
your needs, and JSON is in many respects simpler than XML.  That said,
keep in mind that JSON is fundamentally aimed at what we informally call
"data", I.e. roughly the sorts of things that fit well into a Java, C, or
(of course) Javascript structure.  JSON is typically not good at dealing
with structured documents, in which multiple paragraphs must be kept in
order, and mixed with other structures such as address fields, approval
markings, signatures, or text formatting information.  It's not that you
can do that with JSON, but in complex cases you wind up doing a lot of
work in nonstandard ways.

So my guideline would be:  if you're really dealing with fairly pure
"data", take your pick according to the skills of your programmers, the
tooling you want to use, and especially the expectations of those with
whom you will be sharing the data.  If you have lots of document-style
information to deal with (and there are sort of vague estimates that the
vast majority of information that computers deal with is more like that
these day), then I'd think twice before using something like JSON.
Instead, I'd go with a custom XML vocabulary, or if you prefer, an Open
Document Format file.

Noah

--------------------------------------
Noah Mendelsohn
IBM Corporation
One Rogers Street
Cambridge, MA 02142
1-617-693-4036
--------------------------------------








"XML4Pharma" <info@...>
10/03/2009 06:13 AM
Please respond to "XML4Pharma"
 
        To:     "Michael Kay" <mike@...>, "'Dennis Sosnoski'"
<dms@...>, <noah_mendelsohn@...>
        cc:     "'C. M. Sperberg-McQueen'" <cmsmcq@...>,
<xmlschema-dev@...>
        Subject:        Re: Escalation mechanism for different
interpretation of W3C XML-Schema   specification ?


In our extension schemas, we only use "redefine" to ADD extra attributes
and
elements (in a separate namespace), not to change or remove any existing
ones.

One of my positive surprises with the way we implemented it, is that a XML

data binding like XMLBeans had no problem at all working with it: it was
very easy to derive all classes from our schemas including the "redefine".

Dennis suggested to have a look at JSON, which I did.
This might be a very good idea for a format for submitting
information/data
to the FDA.
The problem at the FDA is that there is almost noone there with a basic
understanding of XML, and those who have left the FDA to become a
well-paid
consultant. Some departments at the FDA have the greatest difficulty to
validate even very simple XML documents.
The FDA is currently planning to switch to HL7-v3-XML messages for
electronic submissions of data to the agency. As HL7-v3-XML is highly
complex (and even not good XML i.m.o.) I think this will lead to disaster.

So JSON may be a good alternative.
I had a quick look, but JSON does not have some data types like date, time

and dateTime. Would be nice if they had ...

Best,

Jozef Aerts
XML4Pharma


----- Original Message -----
From: "Michael Kay" <mike@...>
To: "'Dennis Sosnoski'" <dms@...>; <noah_mendelsohn@...>
Cc: "'XMLSchema at XML4Pharma'" <XMLSchema@...>; "'C. M.
Sperberg-McQueen'" <cmsmcq@...>; <xmlschema-dev@...>
Sent: Saturday, October 03, 2009 12:43 AM
Subject: RE: Escalation mechanism for different interpretation of W3C
XML-Schema specification ?


>
> While I think many of your criticisms of xs:redefine and complex type
> restriction and indeed xs:override are valid, I personally don't accept
> the
> premise that XSD should be constrained or influenced by the data typing
> paradigms of conventional programming languages. XML and XSD are first
and
> formost intended for designing hierarchic document structures (a
> discipline
> with a far longer tradition than programming), and the fact that
> conventional programming languages don't support such structures very
well

> means in my view that it's best to switch to languages that do.
>
> Regards,
>
> Michael Kay
> http://www.saxonica.com/
> http://twitter.com/michaelhkay
>
>
>> -----Original Message-----
>> From: xmlschema-dev-request@...
>> [mailto:xmlschema-dev-request@...] On Behalf Of Dennis Sosnoski
>> Sent: 02 October 2009 23:10
>> To: noah_mendelsohn@...
>> Cc: XMLSchema at XML4Pharma; C. M. Sperberg-McQueen;
>> xmlschema-dev@...
>> Subject: Re: Escalation mechanism for different
>> interpretation of W3C XML-Schema specification ?
>>
>> Hi Noah,
>>
>> IMHO both xs:redefine and xs:complexType restriction should
>> be eliminated. With respect to the increasingly important
>> area of XML data binding for programming languages, redefine
>> and complexType restriction are generally ignored for the
>> simple reason that they have no equivalent in terms of data
>> models. The object oriented programming equivalent of
>> complexType restriction would be to define a subclass by
>> eliminating information from the parent class, which is a
>> bizarre concept in programming terms - if you want a limited
>> representation which is not supported by the current data
>> model you either refactor the data model to support the
>> particular subset you want or use a representation which has
>> more features than you need, with semantic constraints on the usage.
>> There are times when this limitation of object oriented
>> programming is inconvenient, but I suspect the vast majority
>> of developers would agree that the benefits in terms of clean
>> data models more than outweigh the inconvenience.
>>
>> Beyond this, the very nature of complexType restriction,
>> which requires repeating all portions of the original model
>> which are being retained, creates brittle definitions.
>>
>> redefine is an even stranger operation, effectively
>> performing surgery on an existing definition to deform it in
>> ways never intended by the original designers. One
>> consequence is that anyone looking at an instance of an XML
>> structure defined by a schema needs to be aware of other
>> schemas which may have modified the original schema definition.
>> Requiring knowledge of the entire set of schemas involved in
>> a document in order to understand one particular component of
>> the document is contrary to good design principles.
>>
>> It's hard for me to see how xs:override represents any kind
>> of improvement on redefine, so asking for feedback on
>> deprecating redefine while adding override into the mix seems
>> deliberately futile. From my point of view override suffers
>> from the same design failures as redefine, and should also be
>> eliminated.
>>
>> I realize that many uses of W3C schema are not concerned with
>> data binding, but I'd suggest that the difficulty of modeling
>> these constructs in terms of modern programming language
>> structures shows that they could be eliminated without
>> significantly harming the usefulness of schema.
>>
>>   - Dennis
>>
>> Dennis M. Sosnoski
>> XML and Web Services in Java
>> Training and Consulting
>> http://www.sosnoski.com - http://www.sosnoski.co.nz Seattle,
>> WA [phone] - Wellington, NZ [phone]
>>
>>
>>
>> noah_mendelsohn@... wrote:
>> > I strongly urge you to read Michael Sperberg-McQueen's note
>> carefully.
>> > While, as noted below, some of my preferences for resolving
>> > ambiguities happen to be different from his, I believe that
>> his email
>> > very accurately and carefully summarizes the state of play on this
>> > issue.  Let me just quote and comment on a few sections of
>> Michael's note:
>> >
>> >
>> >> The XSD 1.0 specification does not explicitly address the
>> case when
>> >> the same schema document in referenced both directly (via
>> import or
>> >> include) and indirectly (via redefine).  The rules for
>> import specify
>> >> that the components imported via the import or include should be
>> >> present in the resulting schema; the discussion of
>> redefine says that
>> >> the effect of redefine is pervasive.
>> >>
>> >
>> > Yes, that's what it says, and it's contradictory.
>> >
>> >
>> >> Some readers interpret this to mean that the
>> pervasive-redefinition
>> >> rule overrides, or modifies the effect of, the import and include
>> >> rules.  On this reading the schema is legal and the components are
>> >> present in their redefined form (and only that form).
>> This reading
>> >> effectively attaches to the rules for include and import
>> an unspoken
>> >> exception to the effect that the components imported or
>> included are
>> >> present in the resulting schema unless they are redefined by some
>> >> other schema document reference.
>> >>
>> >> Other readers interpret the rules as requiring that the
>> components in
>> >> question should be present both in their original and in their
>> >> redefined form, which means that the schema violates the rule that
>> >> there must be at most one component of any kind with a
>> given expanded
>> >> name, and thus that the schema should be rejected.
>> >>
>> >
>> > Yes, those are at least two of the widely held positions as to what
>> > was "really" intended, and I don't think I'm aware of any others.
>> >
>> >
>> >> It is my personal belief that the most plausible interpretation of
>> >> the specification is the latter; as you have observed, at
>> least some
>> >> implementors disagree and prefer the other interpretation.
>> >>
>> >
>> > I happen to be one of the others who would prefer that the
>> redefine be
>> > pervasive, but the important point here is that either
>> interpretation
>> > is plausible, and we agree on that.
>> >
>> >
>> >> The inability of the XML Schema working group to agree on
>> a normative
>> >> interpretation of XSD 1.0 has led to the agreement to
>> deprecate the
>> >> redefine element in XSD 1.1.
>> >>
>> >
>> > Here, I'd like to add one bit.  While Michael is correct that the
>> > working group has "agreed" on this, some of us joined in that
>> > agreement with some hesitancy, because we believe that
>> <redefine> has
>> > seen widespread use, including in cases that do not trigger (or at
>> > least do not as clearly
>> > trigger) the ambiguities that are causing trouble here.
>> So, in light
>> > of those concerns, the working group also agreed to make the
>> > deprecation of <redefine> a so-called Priority Feedback issue.
>> > Quoting from the working draft [1]:
>> >
>> > ----
>> > Note: The redefinition feature described in the remainder of this
>> > section is .deprecated. and may be removed from future versions of
>> > this specification. Schema authors are encouraged to avoid
>> its use in
>> > cases where interoperability or compatibility with later
>> versions of
>> > this specification are important.
>> >
>> > Editorial Note: Priority Feedback Request
>> >
>> > The Working Group requests feedback from readers, schema authors,
>> > implementors, and other users of this specification as to the
>> > desirability of retaining, removing, deprecating, or not
>> deprecating
>> > the use of <redefine>. Since the <override> facility
>> provides similar
>> > functionality but does not require a restriction or
>> extension relation
>> > between the new and the old definitions of redefined
>> components, the
>> > Working Group is particularly interested in learning
>> whether users of
>> > this specification find that requirement useful or not.
>> > ----
>> >
>> > So, if any readers of this thread have opinions on the plan to
>> > deprecate, the Schema Working group would welcome hearing
>> about them.
>> > I think it's worth noting that this thread has already made clear
>> > that: 1) this is a known area of complexity and the working
>> group has
>> > already tried and so far failed to find an easy resolution
>> acceptable
>> > to all; 2) there are incompatibilities in widely deployed
>> > implementations, so any clear resolution is quite likely to make
>> > someone with an investment in code very unhappy.  That's
>> not to say I
>> > wouldn't like it cleaned up;  indeed, I'm among those who put many
>> > months into trying a few years ago (as did Michael). I'm
>> just pointing
>> > out that our choices may be to deprecate or undeprecate, but going
>> > further to remove the ambiguity is likely to be a
>> significant effort
>> > that will introduce incompatibilities for at least
>> somebody.  Maybe or
>> > maybe not we could find a way to warn people off from the most
>> > troublesome cases, but I know that Michael and perhaps others will
>> > correctly point out that the whole conceptual framework for this
>> > "composition" function is sufficiently shaky in the current drafts
>> > that any fix short of a complete rewrite is likely to leave
>> things in a messy state.
>> >
>> > Noah
>> >
>> > [1] http://www.w3.org/TR/xmlschema11-1/#modify-schema
>> >
>> > --------------------------------------
>> > Noah Mendelsohn
>> > IBM Corporation
>> > One Rogers Street
>> > Cambridge, MA 02142
>> > 1-617-693-4036
>> > --------------------------------------
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>>
>
>

[Michael Kay]

> > So JSON may be a good alternative.
>
> I would never discourage you from looking for the best
> language to meet your needs, and JSON is in many respects
> simpler than XML.  That said, keep in mind that JSON is
> fundamentally aimed at what we informally call "data", I.e.
> roughly the sorts of things that fit well into a Java, C, or
> (of course) Javascript structure.  

When XML was first conceived, I think most people expected it to be used
primarily for documents (I had heard people talking about using SGML for
data in the 1980s, and I thought they were nuts). It's interesting to review
why XML proved successful as a data interchange format even though it wasn't
primarily designed for that role.

* XML was very cheap to implement (cost of writing tools, cost of buying
tools, cost of putting them to good use)

* For some reason I have never understood, XML had no serious competition,
and had universal endorsement from all influential players

* XML filled a gap. Many people with data interchange problems had devised
custom solutions at layer 6 of the stack, and they all involved a lot of
effort to maintain, and no generic tooling was available. These solutions
often failed to solve the character sets problem. Other people had adopted
ASN.1, which was unaffordable by the masses ($100K for a parser).

* XML was genuinely open and platform-independent: people trusted the
independence of the authority responsible for the specification, and there
was neither an explicit nor a covert bias to particular operating systems,
vendors, or programming languages. And there were no doubts about what was
or was not valid XML (as there are with JSON).

* For the first time in the history of computing, people were finding that
documents and data could no longer be kept separate. People were building
web sites in which information, entertainment, and transactions needed to be
seamlessly mixed.

* The high level of redundancy in XML, which we love to complain about,
proved a winner in making it easy to formats to evolve gracefully.

If JSON had been around before XML, the first three arguments would have
been far less compelling. But the others would still have been strong.

Regards,

Michael Kay
http://www.saxonica.com/
http://twitter.com/michaelhkay 

[noah_mendelsohn]

Michael Kay writes:

> When XML was first conceived, I think most people expected it to be used
> primarily for documents (I had heard people talking about using SGML for
> data in the 1980s, and I thought they were nuts). It's
> interesting to review
> why XML proved successful as a data interchange format even
> though it wasn't
> primarily designed for that role.

Well, as I've said many times, I believe it's in large part because the
use cases aren't always separate.  Many of the most interesting documents
mix in real data.  An insurance policy template, for example, probably has
some sort of smart hook to pull in the name of the user and the exact
amount of the policy, very possibly sourced from a relational database.
So, I think that's what gets you into the business of having robust data
typing, etc.  Then the question is, do you for uniformity keep using XML
in the degenerate case where it really is pure data, or drop back to
something simpler.

BTW: one of the reasons I'm attuned to this is that Lotus Notes (which I
used early when I worked at Lotus, but did not help design) built a
business on creating and managing just such mixed document/data
constructions, well before XML (but not SGML) became popular.

Noah

--------------------------------------
Noah Mendelsohn
IBM Corporation
One Rogers Street
Cambridge, MA 02142
1-617-693-4036
--------------------------------------








"Michael Kay" <mike@...>
10/03/2009 12:11 PM
 
        To:     <noah_mendelsohn@...>, "'XML4Pharma'"
<info@...>
        cc:     "'C. M. Sperberg-McQueen'" <cmsmcq@...>,
"'Dennis Sosnoski'" <dms@...>, <xmlschema-dev@...>
        Subject:        RE: Escalation mechanism for different
interpretation of W3C XML-Schema   specification ?


> > So JSON may be a good alternative.
>
> I would never discourage you from looking for the best
> language to meet your needs, and JSON is in many respects
> simpler than XML.  That said, keep in mind that JSON is
> fundamentally aimed at what we informally call "data", I.e.
> roughly the sorts of things that fit well into a Java, C, or
> (of course) Javascript structure.

When XML was first conceived, I think most people expected it to be used
primarily for documents (I had heard people talking about using SGML for
data in the 1980s, and I thought they were nuts). It's interesting to
review
why XML proved successful as a data interchange format even though it
wasn't
primarily designed for that role.

* XML was very cheap to implement (cost of writing tools, cost of buying
tools, cost of putting them to good use)

* For some reason I have never understood, XML had no serious competition,
and had universal endorsement from all influential players

* XML filled a gap. Many people with data interchange problems had devised
custom solutions at layer 6 of the stack, and they all involved a lot of
effort to maintain, and no generic tooling was available. These solutions
often failed to solve the character sets problem. Other people had adopted
ASN.1, which was unaffordable by the masses ($100K for a parser).

* XML was genuinely open and platform-independent: people trusted the
independence of the authority responsible for the specification, and there
was neither an explicit nor a covert bias to particular operating systems,
vendors, or programming languages. And there were no doubts about what was
or was not valid XML (as there are with JSON).

* For the first time in the history of computing, people were finding that
documents and data could no longer be kept separate. People were building
web sites in which information, entertainment, and transactions needed to
be
seamlessly mixed.

* The high level of redundancy in XML, which we love to complain about,
proved a winner in making it easy to formats to evolve gracefully.

If JSON had been around before XML, the first three arguments would have
been far less compelling. But the others would still have been strong.

Regards,

Michael Kay
http://www.saxonica.com/
http://twitter.com/michaelhkay 

[Peter Geraghty]

Some parts of this message have been removed. Learn more about Nabble's security policy.

Michael – you’ve asked for feedback about whether redefine should be deprecated.  I believe it should.

 

I believe that the primary purpose for which schemas are written is an attempt to facilitate automated data exchange between computer systems.   Features which in practice provide an “impedance” to the smooth automation of data exchange are bad.  This email thread provides a classic illustration of a feature which has resulted in troubled and problematic implementation.

 

With regard to this discussion creeping onto other features such as complex type derivation by restriction, I don’t think it is particularly helpful of you to suggest that people stick to programming languages which support the same data structure concepts as XML Schema!  Picking up your own point from another email on this thread, schema is widely used as a data interchange format even though it originated in document markup.  Data interchange involves sending and receiving systems, and to my mind an interchange definition which aims at facilitating smooth automation should dovetail reasonably easily with technologies commonly used – the words “cart”, “horse”, “tail” and “dog” spring to mind! I think schema has often been criticised as over-complicated and over-sophisticated.  Maybe definition of some subset of schema facilities for common data interchange would help in this whole area, although obviously attempting such an exercise would not be an easy one.

 

From: xmlschema-dev-request@... [mailto:xmlschema-dev-request@...] On Behalf Of Michael Kay
Sent: 01 October 2009 23:53
To: 'XMLSchema at XML4Pharma'; xmlschema-dev@...
Subject: RE: Escalation mechanism for different interpretation of W3C XML-Schema specification ?

 

Just for the record (though it only really confirms the point that there are differing interpretations), I implemented the XSD 1.0 specification without the benefit of knowing anything about the intent of the WG beyond what was written in the spec, and I came to the conclusion that there was only one way of interpreting the rule about xsd:redefine being "pervasive". But then I also came to the conclusion that other parts of the schema composition rules had to be treated as loose statements of intent which didn't cover all contingencies.

 

I would certainly advise against using xs:redefine in an industry schema specification. Its only possible justification in my view is to define a variant of a schema produced by a third party. And for that use case, I think it only works unambiguously if that schema uses a single target namespace throughout.

Michael Kay

Saxonica

---

From: xmlschema-dev-request@... [mailto:xmlschema-dev-request@...] On Behalf Of XMLSchema at XML4Pharma
Sent: 29 September 2009 12:24
To: xmlschema-dev@...
Subject: Escalation mechanism for different interpretation of W3C XML-Schema specification ?

We, the CDISC XML-Tech Governance Team (and other CDISC teams) have developed a number of extensible standards for exchange of clinical data and for submitting information to the regulatory authorities (FDA).

CDISC is a Standardization Organization active in the healthcare world.

Our extension mechanism is based on the “import” and “redefine” elements of XML-Schema.

We now have a serious dispute with one technology vendor (Altova) about the way “import” and “redefine” are used. Instance files of one of our extensions (so-called “define.xml”) validate well in all major validators and XML-editors, except for the products of this one vendor.

When confronted with this result, the reaction of Altova essentially is that “Altova is right, all others are wrong”. The dispute and discussion with Altova can be followed at:
http://www.altova.com/forum/default.aspx?g=posts&m=1000005665

The issue were not so serious if it were not that our standard “define.xml” is a standard for submission of information to the regulatory authorities, and these are (mostly) using the Altova product for validation.

We now want to escalate the issue to the W3C itself, and would like to know what the mechanism is to do so.

Jozef Aerts
CDISC XML-Tech Governance Team

---

Disclaimer: The contents of this E-mail plus any attachment is intended for the use of the addressee only and is confidential, proprietary and may be privileged. It will not be binding upon Trace Group or any group company (Trace). Opinions, conclusions, contractual obligations and other information in this message in so far as they relate to the official business of Trace must be specifically confirmed in writing by Trace. If you are not the intended recipient you must not copy this message or attachment, use or disclose the contents to any other person, but are requested to telephone or E-mail the sender and delete the message and any attachment from your system. Trace takes all reasonable precautions to ensure that no virus or defect is transmitted via this e mail, however Trace accepts no responsibility for any virus or defect that might arise from opening this E-mail or attachments.

---

[Henry S. Thompson]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Stimulated in equal measure by this thread, and a request from the XML
Security WG [0], I came up with my personal 'best practice' guideline
for schema document design for multi-namespace schemas.  I include the
core bit below -- for the full context and a worked example, see my
reply to the DSIG WG [1].

ht

[0] http://lists.w3.org/Archives/Public/public-xmlsec/2009Oct/0019.html
[1] http://lists.w3.org/Archives/Public/public-xmlsec/2009Oct/0023.html
- -------------------------
HST's best practice guide for multiple-namespace schema composition

It's helpful in discussions of this sort to be careful with
vocabulary:

  Schema document: An XML document in the W3C XML Schema namespace
  Schema: An abstract data object, typically composed from one or more
          schema documents according to the rules given in the W3C XML
          Schema specs.

A schema document specifies either one or no target namespace.

A schema may well contain element & attribute declarations and/or type
definitions for many target namespaces.

The W3C XML Schema specs provide a number of mechanisms for assembling
the schema documents needed to construct a schema for a particular
validation episode:

 1) the schemaLocation attribute on xs:include elements.
 2) xsi:schemaLocation (in the instance to be validated);
 3) the schemaLocation attribute on xs:import elements;
 4) the namespace URIs themselves (in the instance to be validated, or
    from the namespace attribute on xs:import elements);

Without going into the details of what's optional and what's required,
or of the range of choices implementations exhibit, here's my
recommendation for multi-namespace schemas:

1) For each namespace in your language, there should be one primary
   schema document.  It may be the only schema document for that
   namespace, or it may use xs:include (which must have a
   schemaLocation attribute) to allow for modular development by
   including multiple as-it-were secondary schema documents;

2) Whenever reference is made to names in another namespace from a
   primary or secondary schema document, use an xs:import _without_ a
   schemaLocation attribute;

3) Create a driver schema document, whose target namespace is the
   namespace of the (most common) document element of instance
   documents in the language.  It should contain an xs:include of the
   primary schema document for that namespace, and xs:imports for all
   the other namespaces, _with_ schemaLocation attributes pointing to
   the primary schema documents for those other namespaces.

Then, by passing the location of the driver schema document to schema
validation tools and/or by including an xsi:schemaLocation attribute
on the document element of instances to be validated which associates
the target namespace of the driver schema document with its location,
you will get the behaviour you need from all the tools/implementations
I am aware of.

Schematically, this looks like

               driver.xsd -- TNS = http://www.example.org/rootNS
                              /|\
                             / | \
                            /  |  \
                           /   |   \
                   xs:include  |    \
                   schemaLoc=  |     \
                        /      |      \
                       /   xs:import   \
                      /    NS=...NS2    \
                     /     schemaLoc=    \
                    /          |       xs:import
                   /           |       NS=...NS3
                  /            |       schemaLoc=
                 /             |             \
            primary1.xsd   primary2.xsd  primary3.xsd
            TNS=...rootNS   TNS=...NS2    TNS=...NS3    
                 / \                         /|\
                /   \                       / | \
               /     \                     /  |  \
              /       \                   /   |   \
        xs:import  xs:import             /    |    \
        NS=...NS2  NS=...NS3      xs:include  |     \
                                  schemaLoc=  |      \
                                       /      |       \
                                  3.1.xsd     |        \
                                          xs:include    \
                                          schemaLoc=     \
                                              |           \
                                          3.2.xsd          \
                                                     xs:include
                                                     schemaLoc=
                                                          |
                                                     3.3.xsd

- --
       Henry S. Thompson, School of Informatics, University of Edinburgh
                         Half-time member of W3C Team
      10 Crichton Street, Edinburgh EH8 9AB, SCOTLAND -- (44) 131 650-4440
                Fax: [phone], e-mail: ht@...
                       URL: http://www.ltg.ed.ac.uk/~ht/
[mail really from me _always_ has this .sig -- mail without it is forged spam]
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.6 (GNU/Linux)

iD8DBQFK0021kjnJixAXWBoRAmPMAJ9dLLKZsj+r2y/tUtIsJ/wAjuL0kQCfWwzT
LqnqG7FgzH5zwP3XMIrNsb4=
=vK80
-----END PGP SIGNATURE-----

[Michael Kay]

I agree with all of this, but it leaves the question of xs:redefine.

I suspect, but without testing(!) that the strategy can be extended to
xs:redefine as follows:

(a) if redefinition for a namespace is required, create a new schema
document with that targetNamespace that invokes the (previous) primary
schema document for that namespace using xs:redefine, and contains all the
redefinitions needed.

(b) change the driver document to refer to the redefinition documents
introduced in step (1) in place of the (previous) primary shcema documents.

A similar approach might also work reliably for "chameleon include".

But better advice is not to use redefine or chameleon include.

Regards,

Michael Kay
http://www.saxonica.com/
http://twitter.com/michaelhkay 

 

> -----Original Message-----
> From: xmlschema-dev-request@...
> [mailto:xmlschema-dev-request@...] On Behalf Of Henry S. Thompson
> Sent: 12 October 2009 16:40
> To: XMLSchema at XML4Pharma
> Cc: xmlschema-dev@...
> Subject: Re: Escalation mechanism for different
> interpretation of W3C XML-Schema specification ?
>
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> Stimulated in equal measure by this thread, and a request
> from the XML Security WG [0], I came up with my personal
> 'best practice' guideline for schema document design for
> multi-namespace schemas.  I include the core bit below -- for
> the full context and a worked example, see my reply to the
> DSIG WG [1].
>
> ht
>
> [0]
> http://lists.w3.org/Archives/Public/public-xmlsec/2009Oct/0019.html
> [1]
> http://lists.w3.org/Archives/Public/public-xmlsec/2009Oct/0023.html
> - -------------------------
> HST's best practice guide for multiple-namespace schema composition
>
> It's helpful in discussions of this sort to be careful with
> vocabulary:
>
>   Schema document: An XML document in the W3C XML Schema namespace
>   Schema: An abstract data object, typically composed from one or more
>           schema documents according to the rules given in the W3C XML
>           Schema specs.
>
> A schema document specifies either one or no target namespace.
>
> A schema may well contain element & attribute declarations
> and/or type definitions for many target namespaces.
>
> The W3C XML Schema specs provide a number of mechanisms for
> assembling the schema documents needed to construct a schema
> for a particular validation episode:
>
>  1) the schemaLocation attribute on xs:include elements.
>  2) xsi:schemaLocation (in the instance to be validated);
>  3) the schemaLocation attribute on xs:import elements;
>  4) the namespace URIs themselves (in the instance to be validated, or
>     from the namespace attribute on xs:import elements);
>
> Without going into the details of what's optional and what's
> required, or of the range of choices implementations exhibit,
> here's my recommendation for multi-namespace schemas:
>
> 1) For each namespace in your language, there should be one primary
>    schema document.  It may be the only schema document for that
>    namespace, or it may use xs:include (which must have a
>    schemaLocation attribute) to allow for modular development by
>    including multiple as-it-were secondary schema documents;
>
> 2) Whenever reference is made to names in another namespace from a
>    primary or secondary schema document, use an xs:import _without_ a
>    schemaLocation attribute;
>
> 3) Create a driver schema document, whose target namespace is the
>    namespace of the (most common) document element of instance
>    documents in the language.  It should contain an xs:include of the
>    primary schema document for that namespace, and xs:imports for all
>    the other namespaces, _with_ schemaLocation attributes pointing to
>    the primary schema documents for those other namespaces.
>
> Then, by passing the location of the driver schema document
> to schema validation tools and/or by including an
> xsi:schemaLocation attribute on the document element of
> instances to be validated which associates the target
> namespace of the driver schema document with its location,
> you will get the behaviour you need from all the
> tools/implementations I am aware of.
>
> Schematically, this looks like
>
>                driver.xsd -- TNS = http://www.example.org/rootNS
>                               /|\
>                              / | \
>                             /  |  \
>                            /   |   \
>                    xs:include  |    \
>                    schemaLoc=  |     \
>                         /      |      \
>                        /   xs:import   \
>                       /    NS=...NS2    \
>                      /     schemaLoc=    \
>                     /          |       xs:import
>                    /           |       NS=...NS3
>                   /            |       schemaLoc=
>                  /             |             \
>             primary1.xsd   primary2.xsd  primary3.xsd
>             TNS=...rootNS   TNS=...NS2    TNS=...NS3    
>                  / \                         /|\
>                 /   \                       / | \
>                /     \                     /  |  \
>               /       \                   /   |   \
>         xs:import  xs:import             /    |    \
>         NS=...NS2  NS=...NS3      xs:include  |     \
>                                   schemaLoc=  |      \
>                                        /      |       \
>                                   3.1.xsd     |        \
>                                           xs:include    \
>                                           schemaLoc=     \
>                                               |           \
>                                           3.2.xsd          \
>                                                      xs:include
>                                                      schemaLoc=
>                                                           |
>                                                      3.3.xsd
>
> - --
>        Henry S. Thompson, School of Informatics, University
> of Edinburgh
>                          Half-time member of W3C Team
>       10 Crichton Street, Edinburgh EH8 9AB, SCOTLAND -- (44)
> 131 650-4440
>                 Fax: [phone], e-mail: ht@...
>                        URL: http://www.ltg.ed.ac.uk/~ht/ 
> [mail really from me _always_ has this .sig -- mail without
> it is forged spam] -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.2.6 (GNU/Linux)
>
> iD8DBQFK0021kjnJixAXWBoRAmPMAJ9dLLKZsj+r2y/tUtIsJ/wAjuL0kQCfWwzT
> LqnqG7FgzH5zwP3XMIrNsb4=
> =vK80
> -----END PGP SIGNATURE-----
>