WebDAV/CalDAV integration - first questions

[Evert Pot-2]

Hey guys,

=Base url=

I was told the best place to integrate CalDAV, was through rpc.php.

Looking at that file, it appears it is almost completely based on  
autodetection.
I think it's preferable to have a dedicated url for CalDAV support.

I can either make the base url something like:

/horde/rpc.php/caldav/

or:

/horde/caldav.php/

Any preferences?

=Relation to WebDAV=

I also heard there was a desire to also use SabreDAV as the standard  
library for the existing WebDAV implementation.
I'm happy to provide this, and perhaps I should even start off with  
this; so I can get more familiar with the horde codebase.

Any opinions about this?

I will be able to either provide a different endpoint for WebDAV, e.g.:

/horde/rpc.php/webdav
/horde/webdav.php

Or I can combine both the WebDAV and CalDAV directory structure. The  
latter might be a bit slower, due to the extra hook-ins the CalDAV-
related plugins have.

=Authentication=

Most clients seem to prefer HTTP Digest over HTTP Basic. Although HTTP  
Basic auth is support by virtually any client, many will give warnings  
if it's not used in combination with SSL.
I've noticed Horde uses Basic everywhere.

If you guys want to use Digest, some changes will have to be made in  
the authentication system to store an extra hash.

Opinions?

=CalDAV virtual directory structure=

I just wanted to bring this topic up once more. Some ideas have been  
brought forward by Jan on how to build up the CalDAV directory  
structure.
One of these ideas was to separate the virtual directories for Nag and  
Kronolith.

I'm against this.. Users will generally never be aware of these  
directory structures as they are obfuscated with long uuids. In order  
to keep things reasonably simple, I will need  to start off with  
something like :

caldavroot/users/Administrator/calendars/UUID/UUID

Perhaps down the road there will be the possibility to juggle a little  
bit with these pathnames, but in the short term I kinda need some  
flexibility.
I'm just mentioning this so there is no confusion around how this is  
going to work.

Thanks!
Evert

--
Horde developers mailing list - Join the hunt: http://horde.org/bounties/
Frequently Asked Questions: http://horde.org/faq/
To unsubscribe, mail: dev-unsubscribe@...

[Jan Schneider]

Zitat von Evert Pot <evertpot@...>:

> Hey guys,
>
> =Base url=
>
> I was told the best place to integrate CalDAV, was through rpc.php.
>
> Looking at that file, it appears it is almost completely based on  
> autodetection.
> I think it's preferable to have a dedicated url for CalDAV support.
>
> I can either make the base url something like:
>
> /horde/rpc.php/caldav/
>
> or:
>
> /horde/caldav.php/
>
> Any preferences?

Why? WebDAV detection works fine, and if CalDAV is going to replace  
WebDAV, I see no reason why this shouldn't keep working.

> =Relation to WebDAV=
>
> I also heard there was a desire to also use SabreDAV as the standard  
> library for the existing WebDAV implementation.
> I'm happy to provide this, and perhaps I should even start off with  
> this; so I can get more familiar with the horde codebase.
>
> Any opinions about this?

Since this was my idea, I'm obviously biased, but I still like the idea.

> I will be able to either provide a different endpoint for WebDAV, e.g.:
>
> /horde/rpc.php/webdav
> /horde/webdav.php
>
> Or I can combine both the WebDAV and CalDAV directory structure. The  
> latter might be a bit slower, due to the extra hook-ins the  
> CalDAV-related plugins have.

I prefer the latter. There is no use in keeping two different  
endpoints or libraries IMO.

> =Authentication=
>
> Most clients seem to prefer HTTP Digest over HTTP Basic. Although  
> HTTP Basic auth is support by virtually any client, many will give  
> warnings if it's not used in combination with SSL.
> I've noticed Horde uses Basic everywhere.
>
> If you guys want to use Digest, some changes will have to be made in  
> the authentication system to store an extra hash.
>
> Opinions?

Horde supports a huge variety of authentication backend, so this could  
only be an optional addition for backends that are completely under  
our control. This actually reduces the capable backends to SQL, and  
maybe LDAP if we introduce an optional custom attribute.
Most people use backends that don't provide clear text passwords or  
allow to store arbitrary data for users.

So it boils down to: it's nice to have but won't work with most  
installations anyway. It makes sense for the SQL authentication  
backend, but priority should be low.

> =CalDAV virtual directory structure=
>
> I just wanted to bring this topic up once more. Some ideas have been  
> brought forward by Jan on how to build up the CalDAV directory  
> structure.
> One of these ideas was to separate the virtual directories for Nag  
> and Kronolith.
>
> I'm against this.. Users will generally never be aware of these  
> directory structures as they are obfuscated with long uuids. In  
> order to keep things reasonably simple, I will need  to start off  
> with something like :
>
> caldavroot/users/Administrator/calendars/UUID/UUID
>
> Perhaps down the road there will be the possibility to juggle a  
> little bit with these pathnames, but in the short term I kinda need  
> some flexibility.
> I'm just mentioning this so there is no confusion around how this is  
> going to work.

I'm not sure if this will work, because tasks and calendars are indeed  
provided by completely different applications. We might allow  
Kronolith to proxy CalDAV task access to Nag though.

For the directory structure, I still prefer to keep BC with the  
current structure we have that would be:
kronolith/username/calendarid/eventid

If you don't want this to be the default structure for SabreDAV, then  
we need some hook/plugin to provide a different structure with our  
backend.

Jan.

--
Do you need professional PHP or Horde consulting?
http://horde.org/consulting/


--
Horde developers mailing list - Join the hunt: http://horde.org/bounties/
Frequently Asked Questions: http://horde.org/faq/
To unsubscribe, mail: dev-unsubscribe@...

[Evert Pot-2]

On 2009-09-23, at 12:18 PM, Jan Schneider wrote:

> Zitat von Evert Pot <evertpot@...>:
>
>> Hey guys,
>>
>> =Base url=
>>
>> I was told the best place to integrate CalDAV, was through rpc.php.
>>
>> Looking at that file, it appears it is almost completely based on  
>> autodetection.
>> I think it's preferable to have a dedicated url for CalDAV support.
>>
>> I can either make the base url something like:
>>
>> /horde/rpc.php/caldav/
>>
>> or:
>>
>> /horde/caldav.php/
>>
>> Any preferences?
>
> Why? WebDAV detection works fine, and if CalDAV is going to replace  
> WebDAV, I see no reason why this shouldn't keep working.

Largly its a downright aversion against magic behaviour. In this case  
specifically, I'm worried about certain requests triggering one of the  
other RPC's.
It's also not very clear to me how GET requests are intercepted. How  
is this currently done?

Lastly, the entire response appears to be buffered and sent as a  
string from the default rpc.php. I'd like to avoid this and use php  
streams instead. This can be a big memory saver. I've been able to up/
download up to 2GB files without triggering a memory_limit setting of  
64MB.

> I'm not sure if this will work, because tasks and calendars are  
> indeed provided by completely different applications. We might allow  
> Kronolith to proxy CalDAV task access to Nag though.
>
> For the directory structure, I still prefer to keep BC with the  
> current structure we have that would be:
> kronolith/username/calendarid/eventid
>
> If you don't want this to be the default structure for SabreDAV,  
> then we need some hook/plugin to provide a different structure with  
> our backend.

 From the SabreDAV library there is some flexibility to setup  
different structures. Just for my own sanity I will start off with a  
structure that is the easiest to understand. Based on what I learned  
at that point, we can look at this again.

My point of view is that there is no need for applications (kronolith,  
nag) to inject new paths in CalDAV structure, instead they could just  
register themselves as a new CalDAV-enabled backend. I plan to hide as  
much of the protocol as possible from the backends, if we need support  
for multiple collections of calendars per user (which is what this is  
really about) it could imply that the various backends need to be much  
more aware of the structure as needed; which is something I'd like to  
avoid.

Regardless I will start off simple, and just with kronolith. Once  
we're there it will be easier for me to figure out how multiple  
directory structures work, or articulate why it will be inconvenient.  
Just keep an open mind, and so will I =)

Evert



--
Horde developers mailing list - Join the hunt: http://horde.org/bounties/
Frequently Asked Questions: http://horde.org/faq/
To unsubscribe, mail: dev-unsubscribe@...

[Jan Schneider]

Zitat von Evert Pot <evertpot@...>:

>
> On 2009-09-23, at 12:18 PM, Jan Schneider wrote:
>
>> Zitat von Evert Pot <evertpot@...>:
>>
>>> Hey guys,
>>>
>>> =Base url=
>>>
>>> I was told the best place to integrate CalDAV, was through rpc.php.
>>>
>>> Looking at that file, it appears it is almost completely based on  
>>> autodetection.
>>> I think it's preferable to have a dedicated url for CalDAV support.
>>>
>>> I can either make the base url something like:
>>>
>>> /horde/rpc.php/caldav/
>>>
>>> or:
>>>
>>> /horde/caldav.php/
>>>
>>> Any preferences?
>>
>> Why? WebDAV detection works fine, and if CalDAV is going to replace  
>> WebDAV, I see no reason why this shouldn't keep working.
>
> Largly its a downright aversion against magic behaviour. In this  
> case specifically, I'm worried about certain requests triggering one  
> of the other RPC's.

It worked fine for the last few years, and I don't see why it should  
stop working. If it breaks at some point, we can always change it later.

> It's also not very clear to me how GET requests are intercepted. How  
> is this currently done?

By detecting PATH_INFO requests.

> Lastly, the entire response appears to be buffered and sent as a  
> string from the default rpc.php. I'd like to avoid this and use php  
> streams instead. This can be a big memory saver. I've been able to  
> up/download up to 2GB files without triggering a memory_limit  
> setting of 64MB.

That's nice to have, but not a show stopper. Beside that, everything  
can be changed for H4 as needed. So if using streams is really an  
important concern for you, you can change Horde_Rpc to support them.  
We already did this for some other libraries where it made sense.

>> I'm not sure if this will work, because tasks and calendars are  
>> indeed provided by completely different applications. We might  
>> allow Kronolith to proxy CalDAV task access to Nag though.
>>
>> For the directory structure, I still prefer to keep BC with the  
>> current structure we have that would be:
>> kronolith/username/calendarid/eventid
>>
>> If you don't want this to be the default structure for SabreDAV,  
>> then we need some hook/plugin to provide a different structure with  
>> our backend.
>
> From the SabreDAV library there is some flexibility to setup  
> different structures. Just for my own sanity I will start off with a  
> structure that is the easiest to understand. Based on what I learned  
> at that point, we can look at this again.
>
> My point of view is that there is no need for applications  
> (kronolith, nag) to inject new paths in CalDAV structure, instead  
> they could just register themselves as a new CalDAV-enabled backend.  
> I plan to hide as much of the protocol as possible from the  
> backends, if we need support for multiple collections of calendars  
> per user (which is what this is really about) it could imply that  
> the various backends need to be much more aware of the structure as  
> needed; which is something I'd like to avoid.
>
> Regardless I will start off simple, and just with kronolith. Once  
> we're there it will be easier for me to figure out how multiple  
> directory structures work, or articulate why it will be  
> inconvenient. Just keep an open mind, and so will I =)
>
> Evert
>
>

Jan.

--
Do you need professional PHP or Horde consulting?
http://horde.org/consulting/


--
Horde developers mailing list - Join the hunt: http://horde.org/bounties/
Frequently Asked Questions: http://horde.org/faq/
To unsubscribe, mail: dev-unsubscribe@...

[Chuck Hagenbuch]

Quoting Evert Pot <evertpot@...>:

> Hey guys,
>
> =Base url=
>
> I was told the best place to integrate CalDAV, was through rpc.php.
>
> Looking at that file, it appears it is almost completely based on  
> autodetection.
> I think it's preferable to have a dedicated url for CalDAV support.
>
> I can either make the base url something like:
>
> /horde/rpc.php/caldav/
>
> or:
>
> /horde/caldav.php/
>
> Any preferences?
>
> =Relation to WebDAV=
>
> I also heard there was a desire to also use SabreDAV as the standard  
> library for the existing WebDAV implementation.
> I'm happy to provide this, and perhaps I should even start off with  
> this; so I can get more familiar with the horde codebase.
>
> Any opinions about this?
>
> I will be able to either provide a different endpoint for WebDAV, e.g.:
>
> /horde/rpc.php/webdav
> /horde/webdav.php
>
> Or I can combine both the WebDAV and CalDAV directory structure. The  
> latter might be a bit slower, due to the extra hook-ins the  
> CalDAV-related plugins have.

I know Jan likes the auto-detection, but I wonder if different  
endpoints make some sense here. Having a combined endpoint for webdav  
and caldav of [horde-prefix]/dav/ sounds good to me. How does that work?

> =Authentication=
>
> Most clients seem to prefer HTTP Digest over HTTP Basic. Although  
> HTTP Basic auth is support by virtually any client, many will give  
> warnings if it's not used in combination with SSL.
> I've noticed Horde uses Basic everywhere.
>
> If you guys want to use Digest, some changes will have to be made in  
> the authentication system to store an extra hash.

I'd be fine with supporting it, but it does require plaintext  
passwords, which is its own risk, and not supported by every backend.

-chuck

--
Horde developers mailing list - Join the hunt: http://horde.org/bounties/
Frequently Asked Questions: http://horde.org/faq/
To unsubscribe, mail: dev-unsubscribe@...

[Chuck Hagenbuch]

Quoting Evert Pot <evertpot@...>:

> Lastly, the entire response appears to be buffered and sent as a  
> string from the default rpc.php. I'd like to avoid this and use php  
> streams instead. This can be a big memory saver. I've been able to  
> up/download up to 2GB files without triggering a memory_limit  
> setting of 64MB.

This would be fantastic. You might even look into using  
Horde_Controller and putting stream support into the response objects  
if we're rewriting things. Like Jan said probably not a requirement  
though.

-chuck

--
Horde developers mailing list - Join the hunt: http://horde.org/bounties/
Frequently Asked Questions: http://horde.org/faq/
To unsubscribe, mail: dev-unsubscribe@...

[Evert Pot-2]

On 2009-09-25, at 5:33 PM, Chuck Hagenbuch wrote:

>>
>>
>> /horde/rpc.php/webdav
>> /horde/webdav.php
>>
>> Or I can combine both the WebDAV and CalDAV directory structure.  
>> The latter might be a bit slower, due to the extra hook-ins the  
>> CalDAV-related plugins have.
>
> I know Jan likes the auto-detection, but I wonder if different  
> endpoints make some sense here. Having a combined endpoint for  
> webdav and caldav of [horde-prefix]/dav/ sounds good to me. How does  
> that work?

I'd prefer to follow the consensus, but I tend to agree. RPC !=  
WebDAV, digging into the code further I've also noticed both the RPC  
and WebDAV elements share the same setup. Although there are  
similarities, it's very much a different beast, and it leads me to  
believe it's a bit of a case of over-abstraction.

>
>> =Authentication=
>>
>> Most clients seem to prefer HTTP Digest over HTTP Basic. Although  
>> HTTP Basic auth is support by virtually any client, many will give  
>> warnings if it's not used in combination with SSL.
>> I've noticed Horde uses Basic everywhere.
>>
>> If you guys want to use Digest, some changes will have to be made  
>> in the authentication system to store an extra hash.
>
> I'd be fine with supporting it, but it does require plaintext  
> passwords, which is its own risk, and not supported by every backend.

There's also the possibility of storing the 'A1' part of the hash,  
which is basically md5(username + ':' + realm + ':' + password); This  
is something that could be done down the road though, but I would  
advice anyone using webdav, to use SSL as long as HTTP Basic is used.

Evert

--
Horde developers mailing list - Join the hunt: http://horde.org/bounties/
Frequently Asked Questions: http://horde.org/faq/
To unsubscribe, mail: dev-unsubscribe@...

[Evert Pot-2]

On 2009-09-25, at 5:41 PM, Chuck Hagenbuch wrote:

> Quoting Evert Pot <evertpot@...>:
>
>> Lastly, the entire response appears to be buffered and sent as a  
>> string from the default rpc.php. I'd like to avoid this and use php  
>> streams instead. This can be a big memory saver. I've been able to  
>> up/download up to 2GB files without triggering a memory_limit  
>> setting of 64MB.
>
> This would be fantastic. You might even look into using  
> Horde_Controller and putting stream support into the response  
> objects if we're rewriting things. Like Jan said probably not a  
> requirement though.
>

The biggest benefit for using streams, is that it's possible to reduce  
memory consumption. This helps in 2 areas:
   * files coming from the filesystem won't have to stay in memory
   * it's very easy to copy strings, which leads to a lot of duplicate  
strings in memory

I was very surprised how well PHP is suited for it though.. php://temp  
is great, it will keep a stream in memory if it's under 2 megs, and  
will switch to using temporary files if it exceeds that. I pretty much  
only use it in SabreDAV for GET and PUT requests, just to deal with  
large files.

If Horde_Controller only really deals with HTML, you probably trade in  
the minimal memory savings for I/O and CPU. I think the benefit of  
streams only starts at more than a few megs.

Evert

--
Horde developers mailing list - Join the hunt: http://horde.org/bounties/
Frequently Asked Questions: http://horde.org/faq/
To unsubscribe, mail: dev-unsubscribe@...

[Evert Pot-2]

>
> That's nice to have, but not a show stopper. Beside that, everything  
> can be changed for H4 as needed. So if using streams is really an  
> important concern for you, you can change Horde_Rpc to support them.  
> We already did this for some other libraries where it made sense.

If you're set on using the rpc system for this, I would suggest  
removing the following lines from rpc.php:

/* Return the response to the client. */
header('Content-Type: ' . $server->getResponseContentType());
header('Content-length: ' . strlen($out));
header('Accept-Charset: UTF-8');
echo $out;

..and let the various RPC classes be responsible for both HTTP  
headers, and any output. It would be easy to just copy these lines to  
the abstract RPC class, so transition is seamless.
SabreDAV also needs to be responsible for the Content-Length header,  
because the size of $out does not match the Content-Length header when  
a GET request with a Range: header is made, or when an HTTP HEAD  
request is made.

I could add an output(); method for example..



--
Horde developers mailing list - Join the hunt: http://horde.org/bounties/
Frequently Asked Questions: http://horde.org/faq/
To unsubscribe, mail: dev-unsubscribe@...

[Chuck Hagenbuch]

Quoting Evert Pot <evertpot@...>:

> If you're set on using the rpc system for this, I would suggest  
> removing the following lines from rpc.php:
>
> /* Return the response to the client. */
> header('Content-Type: ' . $server->getResponseContentType());
> header('Content-length: ' . strlen($out));
> header('Accept-Charset: UTF-8');
> echo $out;
>
> ..and let the various RPC classes be responsible for both HTTP  
> headers, and any output. It would be easy to just copy these lines  
> to the abstract RPC class, so transition is seamless.
> SabreDAV also needs to be responsible for the Content-Length header,  
> because the size of $out does not match the Content-Length header  
> when a GET request with a Range: header is made, or when an HTTP  
> HEAD request is made.
>
> I could add an output(); method for example..

I'd be in favor of this. It also seems like a good step towards having  
Rpc/Dav controllers, instead of the current single-entry point system.

In other words, I'd help you with this if you started a patch.

Thanks!

-chuck

--
Horde developers mailing list - Join the hunt: http://horde.org/bounties/
Frequently Asked Questions: http://horde.org/faq/
To unsubscribe, mail: dev-unsubscribe@...

[Jan Schneider]

Zitat von Evert Pot <evertpot@...>:

>>
>> That's nice to have, but not a show stopper. Beside that,  
>> everything can be changed for H4 as needed. So if using streams is  
>> really an important concern for you, you can change Horde_Rpc to  
>> support them. We already did this for some other libraries where it  
>> made sense.
>
> If you're set on using the rpc system for this, I would suggest  
> removing the following lines from rpc.php:
>
> /* Return the response to the client. */
> header('Content-Type: ' . $server->getResponseContentType());
> header('Content-length: ' . strlen($out));
> header('Accept-Charset: UTF-8');
> echo $out;
>
> ..and let the various RPC classes be responsible for both HTTP  
> headers, and any output. It would be easy to just copy these lines  
> to the abstract RPC class, so transition is seamless.
> SabreDAV also needs to be responsible for the Content-Length header,  
> because the size of $out does not match the Content-Length header  
> when a GET request with a Range: header is made, or when an HTTP  
> HEAD request is made.
>
> I could add an output(); method for example..
>
>

Sounds fine.

Jan.

--
Do you need professional PHP or Horde consulting?
http://horde.org/consulting/


--
Horde developers mailing list - Join the hunt: http://horde.org/bounties/
Frequently Asked Questions: http://horde.org/faq/
To unsubscribe, mail: dev-unsubscribe@...