Re: getting WebDAV SEARCH ready for the IESG

[Julian Reschke]

Hi John,

thanks for the comments.

The server I worked on a long time ago never truncated results, so I
don't have any preference.

However, it seems to me that the text in 2.3.1 was phrased this way on
purpose -- there may be cases where it's not possible to first sort,
then truncate. For instance, when searching can be delegated to an
underlying SQL store, but ordering can not, how would you implement
that? Thus, I'm hesitant doing any change over here.

If you feel strongly about that, we *could* add a statement into the
"future extensions" appendix.

And yes, the inconsistency with 5.17.1 is a bit awkward, but I'm really
not sure we can change this at this point of time.

BR, Julian


John Barone wrote:

> Some comments:
>
> In section 2.3.1 Result Set Truncation
>
> - Would be nice to indicate what the search limit is (after what number
> of results was the query truncated)
>
> - Partial results: I read this to mean whatever partial results you send
> back, they must be ordered (within themselves)
> as the client requested.  In many cases the client wants the full list
> ordered, and then send back the partial results.
> Any way to indicate this in the request; i.e. if you have to send back
> partial results (a 507 condition) I want them fully ordered, not just
> within themselves?  Perhaps the server can send back a 507 response for
> the arbiter URI and no results, if it can't comply with ordering the
> full result set, and sending back partial results.
>
>
> In section 5.17.1 Relationship to Result Ordering
>
> - I read this to mean that the full results should first be ordered by
> the server, and then send back the requested limit.  This seems to
> contradict what's specified in section 2.3.1, where the results are
> limited and then ordered (if I'm reading it correctly).  I think these 2
> sections should be consistent with each other.
>
>
> Regards,
>
> -John
>
>
> -----Original Message-----
> From: w3c-dist-auth-request@... [mailto:w3c-dist-auth-request@...]
> On Behalf Of Julian Reschke
> Sent: Wednesday, July 02, 2008 9:12 AM
> To: w3c-dist-auth@...
> Cc: WebDAV
> Subject: getting WebDAV SEARCH ready for the IESG
>
>
> Hi,
>
> we recently made some progress on getting WebDAV SEARCH ready for
> publication.
>
> We received some feedback from Chris Newman, and the latest edits on the
> draft
> (<http://greenbytes.de/tech/webdav/draft-reschke-webdav-search-latest.ht
> ml>,
> <http://greenbytes.de/tech/webdav/draft-reschke-webdav-search-latest-fro
> m-previous.diff.html>)
> take those into account.
>
> Unless there's new feedback, I'm planning to submit this as draft 16,
> which, if all goes well, will then by last called.
>
> Feedback appreciated,
>
> Julian
>
>
>
>
> This email and any attachments may contain confidential and proprietary
> information of Blackboard that is for the sole use of the intended
> recipient.  If you are not the intended recipient, disclosure, copying,
> re-distribution or other use of any of this information is strictly
> prohibited.  Please immediately notify the sender and delete this
> transmission if you received this email in error.
>

[John Barone]

> However, it seems to me that the text in 2.3.1 was phrased this way on
purpose
> -- there may be cases where it's not possible to first sort,
> then truncate. For instance, when searching can be delegated to an
underlying
> SQL store, but ordering can not, how would you implement that?
> Thus, I'm hesitant doing any change over here.

Completely understood.  I'm just saying a client may not want results
that aren't ordered over the entire result set.  It might be preferred
to get no results (and have to further refine the search) than to get
truncated results that aren't "globaly" ordered.


> If you feel strongly about that, we *could* add a statement into the
"future extensions" appendix.

I don't feel that strongly about this, just a nice-to-have for some
clients.


> And yes, the inconsistency with 5.17.1 is a bit awkward, but I'm
really not
> sure we can change this at this point of time.

This I think is a bigger deal.  If you acknowledge that some servers
cannot (at least easily) order a global result set and then limit the
results returned, then how can this be a MUST?  Seems like the same
issue to me.

-John


John Barone wrote:

> Some comments:
>
> In section 2.3.1 Result Set Truncation
>
> - Would be nice to indicate what the search limit is (after what
> number of results was the query truncated)
>
> - Partial results: I read this to mean whatever partial results you
> send back, they must be ordered (within themselves) as the client
> requested.  In many cases the client wants the full list ordered, and
> then send back the partial results.
> Any way to indicate this in the request; i.e. if you have to send back

> partial results (a 507 condition) I want them fully ordered, not just
> within themselves?  Perhaps the server can send back a 507 response
> for the arbiter URI and no results, if it can't comply with ordering
> the full result set, and sending back partial results.
>
>
> In section 5.17.1 Relationship to Result Ordering
>
> - I read this to mean that the full results should first be ordered by

> the server, and then send back the requested limit.  This seems to
> contradict what's specified in section 2.3.1, where the results are
> limited and then ordered (if I'm reading it correctly).  I think these

> 2 sections should be consistent with each other.
>
>
> Regards,
>
> -John
>
>
> -----Original Message-----
> From: w3c-dist-auth-request@...
> [mailto:w3c-dist-auth-request@...]
> On Behalf Of Julian Reschke
> Sent: Wednesday, July 02, 2008 9:12 AM
> To: w3c-dist-auth@...
> Cc: WebDAV
> Subject: getting WebDAV SEARCH ready for the IESG
>
>
> Hi,
>
> we recently made some progress on getting WebDAV SEARCH ready for
> publication.
>
> We received some feedback from Chris Newman, and the latest edits on
> the draft
> (<http://greenbytes.de/tech/webdav/draft-reschke-webdav-search-latest.
> ht
> ml>,
> <http://greenbytes.de/tech/webdav/draft-reschke-webdav-search-latest-f
> ro
> m-previous.diff.html>)
> take those into account.
>
> Unless there's new feedback, I'm planning to submit this as draft 16,
> which, if all goes well, will then by last called.
>
> Feedback appreciated,
>
> Julian
>
>
>
>
> This email and any attachments may contain confidential and
> proprietary information of Blackboard that is for the sole use of the
> intended recipient.  If you are not the intended recipient,
> disclosure, copying, re-distribution or other use of any of this
> information is strictly prohibited.  Please immediately notify the
> sender and delete this transmission if you received this email in

error.
>




This email and any attachments may contain confidential and proprietary
information of Blackboard that is for the sole use of the intended
recipient.  If you are not the intended recipient, disclosure, copying,
re-distribution or other use of any of this information is strictly
prohibited.  Please immediately notify the sender and delete this
transmission if you received this email in error.

[Julian Reschke]

John Barone wrote:

>> However, it seems to me that the text in 2.3.1 was phrased this way on
> purpose
>> -- there may be cases where it's not possible to first sort,
>> then truncate. For instance, when searching can be delegated to an
> underlying
>> SQL store, but ordering can not, how would you implement that?
>> Thus, I'm hesitant doing any change over here.
>
> Completely understood.  I'm just saying a client may not want results
> that aren't ordered over the entire result set.  It might be preferred
> to get no results (and have to further refine the search) than to get
> truncated results that aren't "globaly" ordered.

I do agree that this may be more useful. I'm just skeptic about making
this change many years after people have written implementations.

>> If you feel strongly about that, we *could* add a statement into the
> "future extensions" appendix.
>
> I don't feel that strongly about this, just a nice-to-have for some
> clients.
>
>
>> And yes, the inconsistency with 5.17.1 is a bit awkward, but I'm
> really not
>> sure we can change this at this point of time.
>
> This I think is a bigger deal.  If you acknowledge that some servers
> cannot (at least easily) order a global result set and then limit the
> results returned, then how can this be a MUST?  Seems like the same
> issue to me.

I just checked the document's history, and that particular requirement
was added in 2003, see the thread around
<http://lists.w3.org/Archives/Public/www-webdav-dasl/2002OctDec/0033.html>.
Back then we probably did not realize that we're introducing an
inconsistency between truncation (server enforced) and limiting (on
behalf of the client).

If this is a minor problem, we should just state it somewhere. If it's a
major problem, we could try to fix it. The server I worked on didn't
truncate, so I don't have a strong preference. That being said, it would
be interesting to know how the other servers (Xythos, Catacomb,
Slide...?) behave...

BR, Julian

[John Barone]

The Xythos server currently doesn't implement the limit feature.  The
server does truncate results based on a server setting.  Making sure the
truncated results are globally ordered is difficult, for the reasons you
outlined and particularly when the search spans multiple data stores.
Implementing the limit feature would pose the same ordering challenges.
I think making 5.17.1 a MUST places a heavy burden on the server
implementation.

-John

-----Original Message-----
From: Julian Reschke [mailto:julian.reschke@...]
Sent: Monday, August 04, 2008 9:24 AM
To: John Barone
Cc: w3c-dist-auth@...; www-webdav-dasl@...
Subject: Re: getting WebDAV SEARCH ready for the IESG

John Barone wrote:

>> However, it seems to me that the text in 2.3.1 was phrased this way
>> on
> purpose
>> -- there may be cases where it's not possible to first sort, then
>> truncate. For instance, when searching can be delegated to an
> underlying
>> SQL store, but ordering can not, how would you implement that?
>> Thus, I'm hesitant doing any change over here.
>
> Completely understood.  I'm just saying a client may not want results
> that aren't ordered over the entire result set.  It might be preferred

> to get no results (and have to further refine the search) than to get
> truncated results that aren't "globaly" ordered.

I do agree that this may be more useful. I'm just skeptic about making
this change many years after people have written implementations.

>> If you feel strongly about that, we *could* add a statement into the
> "future extensions" appendix.
>
> I don't feel that strongly about this, just a nice-to-have for some
> clients.
>
>
>> And yes, the inconsistency with 5.17.1 is a bit awkward, but I'm
> really not
>> sure we can change this at this point of time.
>
> This I think is a bigger deal.  If you acknowledge that some servers
> cannot (at least easily) order a global result set and then limit the
> results returned, then how can this be a MUST?  Seems like the same
> issue to me.

I just checked the document's history, and that particular requirement
was added in 2003, see the thread around
<http://lists.w3.org/Archives/Public/www-webdav-dasl/2002OctDec/0033.htm
l>.
Back then we probably did not realize that we're introducing an
inconsistency between truncation (server enforced) and limiting (on
behalf of the client).

If this is a minor problem, we should just state it somewhere. If it's a
major problem, we could try to fix it. The server I worked on didn't
truncate, so I don't have a strong preference. That being said, it would
be interesting to know how the other servers (Xythos, Catacomb,
Slide...?) behave...

BR, Julian






This email and any attachments may contain confidential and proprietary
information of Xythos that is for the sole use of the intended
recipient.  If you are not the intended recipient, disclosure, copying,
re-distribution or other use of any of this information is strictly
prohibited.  Please immediately notify the sender and delete this
transmission if you received this email in error.

[Julian Reschke]

John Barone wrote:
> The Xythos server currently doesn't implement the limit feature.  The
> server does truncate results based on a server setting.  Making sure the
> truncated results are globally ordered is difficult, for the reasons you
> outlined and particularly when the search spans multiple data stores.

...another good point I forgot.

> Implementing the limit feature would pose the same ordering challenges.
> I think making 5.17.1 a MUST places a heavy burden on the server
> implementation.

One alternative would be "SHOULD", another one would be just stating
that DAV:limit is optional, and servers that can't do the MUST level
requirement should reject the query.

Any preference?

BR, Julian

[John Barone]

First and foremost I would be in favor of wording that is consistent
with what's outlined in section 2.3.1, for truncation.  From a client
perspective, I would think that the MUST wording in section 5.17.1 is
most desirable.  However, from a practical (and admittedly self-serving)
point of view, simply stating that the results MUST ordered as the
client directed, would be preferred.  Section 2.3.1 goes on to say:

"... the partial results returned MAY be any subset of the result set
that would have satisfied the original query".

Perhaps in section 5.17.1 the additional sentence could be phrased:

"... the results that are included in the response document SHOULD be
those that order highest"


Regards,

-John

-----Original Message-----
From: Julian Reschke [mailto:julian.reschke@...]
Sent: Monday, August 04, 2008 10:39 AM
To: John Barone
Cc: w3c-dist-auth@...; www-webdav-dasl@...
Subject: Re: getting WebDAV SEARCH ready for the IESG

John Barone wrote:
> The Xythos server currently doesn't implement the limit feature.  The
> server does truncate results based on a server setting.  Making sure
> the truncated results are globally ordered is difficult, for the
> reasons you outlined and particularly when the search spans multiple
data stores.

...another good point I forgot.

> Implementing the limit feature would pose the same ordering
challenges.
> I think making 5.17.1 a MUST places a heavy burden on the server
> implementation.

One alternative would be "SHOULD", another one would be just stating
that DAV:limit is optional, and servers that can't do the MUST level
requirement should reject the query.

Any preference?

BR, Julian



This email and any attachments may contain confidential and proprietary
information of Blackboard that is for the sole use of the intended
recipient.  If you are not the intended recipient, disclosure, copying,
re-distribution or other use of any of this information is strictly
prohibited.  Please immediately notify the sender and delete this
transmission if you received this email in error.

[Julian Reschke]

John Barone wrote:

> First and foremost I would be in favor of wording that is consistent
> with what's outlined in section 2.3.1, for truncation.  From a client
> perspective, I would think that the MUST wording in section 5.17.1 is
> most desirable.  However, from a practical (and admittedly self-serving)
> point of view, simply stating that the results MUST ordered as the
> client directed, would be preferred.  Section 2.3.1 goes on to say:
>
> "... the partial results returned MAY be any subset of the result set
> that would have satisfied the original query".
>
> Perhaps in section 5.17.1 the additional sentence could be phrased:
>
> "... the results that are included in the response document SHOULD be
> those that order highest"

So, to be precise, the single change you're proposing is to relax the
"must" to a "should"?

I'd be ok with that.

BR, Julian

[John Barone]

Yes, that would be fine.

I wasn't privy to the original discussion that resulted in the MUST.
So, I don't know if there are some client requirements/concerns that
would be stepped on by changing it to a SHOULD.

Regards,

-John

-----Original Message-----
From: Julian Reschke [mailto:julian.reschke@...]
Sent: Monday, August 04, 2008 11:35 AM
To: John Barone
Cc: w3c-dist-auth@...; www-webdav-dasl@...
Subject: Re: getting WebDAV SEARCH ready for the IESG

John Barone wrote:
> First and foremost I would be in favor of wording that is consistent
> with what's outlined in section 2.3.1, for truncation.  From a client
> perspective, I would think that the MUST wording in section 5.17.1 is
> most desirable.  However, from a practical (and admittedly
> self-serving) point of view, simply stating that the results MUST
> ordered as the client directed, would be preferred.  Section 2.3.1
goes on to say:
>
> "... the partial results returned MAY be any subset of the result set
> that would have satisfied the original query".
>
> Perhaps in section 5.17.1 the additional sentence could be phrased:
>
> "... the results that are included in the response document SHOULD be
> those that order highest"

So, to be precise, the single change you're proposing is to relax the
"must" to a "should"?

I'd be ok with that.

BR, Julian





This email and any attachments may contain confidential and proprietary
information of Blackboard that is for the sole use of the intended
recipient.  If you are not the intended recipient, disclosure, copying,
re-distribution or other use of any of this information is strictly
prohibited.  Please immediately notify the sender and delete this
transmission if you received this email in error.

[Julian Reschke]

John Barone wrote:
> Yes, that would be fine.
>
> I wasn't privy to the original discussion that resulted in the MUST.
> So, I don't know if there are some client requirements/concerns that
> would be stepped on by changing it to a SHOULD.
> ...

I would assume that any query that DAV:limits itself, or results in
server-enforced truncation, is unlikely to be useful in a programmatic
client. So I wouldn't expect this to be a problem.

BR, Julian