CatalystX::CRUD

[Peter Karman]

I've been thinking the last couple days about ways to expand
Catalyst::Controller::Rose to play more nicely with other models besides
C::M::RDBO. This is per mst's request to open up the RHTMLO goodness to
non-RDBO users, and because I now find myself wanting the same thing. I have a
model that isn't RDBO that I'd like to use in a project.

So I'm proposing the following -- comments/criticism welcome.

* CatalystX::CRUD::Model (CXCM)

A base class for CRUD-like models. CXCM isa Catalyst::Model. Any CXCM subclass
could be used with the C::C::Rose classes (or any other controller that decided
to adhere to the CXCM API). C::M::RDBO would become a CXCM subclass.

CXCM subclasses would need to implement at least the following methods:

 * new_object  - returns CatalystX::CRUD::Object->new()
 * fetch       - returns CatalystX::CRUD::Object->new()->read()
 * search      - returns zero or more CXCO instances as an arrayref
 * interator   - like search() but returns an iterator
 * count       - like search() but returns an integer

(For those following along at home, you'll notice that's basically the
C::M::RDBO API.)

* CatalystX::CRUD::Object (CXCO)

A base class for objects returned by CatalystX::CRUD::Model subclasses. In the
case of RDBO, this would just be a thin wrapper class that 'hasa' RDBO object.
So e.g. calling create() or update() on a CatalystX::CRUD::Object::RDBO object
would just look something like:

 sub create {
    my $self = shift;  # CXCO object
    $self->rdbo->save(@_);
 }

 # same thing for update()

CXCO subclasses would need to implement at least the following methods:

 * create    - write a new object to store
 * read      - load a new object from store
 * update    - save an existing object to store
 * delete    - remove an existing object from store

(How original! CRUD!)

You'll notice that the required CXCO methods are intentionally few. I assume
that subclasses would want to also provide accessors to any underlying objects
so that controllers could act directly on them (e.g., the rdbo() method in the
above create example).

I imagine that there could then be classes like:

 CatalystX::CRUD::Model::DBIC
 CatalystX::CRUD::Object::DBIC
 CatalystX::CRUD::Model::CDBI
 CatalystX::CRUD::Object::CDBI

etc., that would all play nicely with C::C::Rose.

Thoughts?


--
Peter Karman  .  peter@...  .  http://peknet.com/


_______________________________________________
List: Catalyst@...
Listinfo: http://lists.rawmode.org/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/catalyst@.../
Dev site: http://dev.catalyst.perl.org/

[Matt S Trout-2]

On Fri, Sep 21, 2007 at 08:57:35AM -0500, Peter Karman wrote:

> I've been thinking the last couple days about ways to expand
> Catalyst::Controller::Rose to play more nicely with other models besides
> C::M::RDBO. This is per mst's request to open up the RHTMLO goodness to
> non-RDBO users, and because I now find myself wanting the same thing. I have a
> model that isn't RDBO that I'd like to use in a project.
>
> So I'm proposing the following -- comments/criticism welcome.
>
> * CatalystX::CRUD::Model (CXCM)
>
> A base class for CRUD-like models. CXCM isa Catalyst::Model. Any CXCM subclass
> could be used with the C::C::Rose classes (or any other controller that decided
> to adhere to the CXCM API). C::M::RDBO would become a CXCM subclass.
>
> CXCM subclasses would need to implement at least the following methods:
>
>  * new_object  - returns CatalystX::CRUD::Object->new()
>  * fetch       - returns CatalystX::CRUD::Object->new()->read()
>  * search      - returns zero or more CXCO instances as an arrayref
>  * interator   - like search() but returns an iterator
>  * count       - like search() but returns an integer

Have you looked at the way the Handel storage stuff works? That already
supports both DBIC and RDBO so might be an interesting start.

Do you have any thoughts on how to paper over DBIC's ability to chain searches
vs. the lack of that feature in RDBO? I know most RDBO users don't feel
they're deprived not having it but DBIC users are generally fairly addicted :)

--
      Matt S Trout       Need help with your Catalyst or DBIx::Class project?
   Technical Director                    http://www.shadowcat.co.uk/catalyst/
 Shadowcat Systems Ltd.  Want a managed development or deployment platform?
http://chainsawblues.vox.com/            http://www.shadowcat.co.uk/servers/

_______________________________________________
List: Catalyst@...
Listinfo: http://lists.rawmode.org/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/catalyst@.../
Dev site: http://dev.catalyst.perl.org/

[Perrin Harkins]

On 9/23/07, Matt S Trout <dbix-class@...> wrote:
> Do you have any thoughts on how to paper over DBIC's ability to chain searches
> vs. the lack of that feature in RDBO?

Isn't it primarily a way to build up your search criteria with
multiple small method calls instead of one big one?  It doesn't seem
very relevant to a CRUD app where all the criteria come in at once
from a web form.

- Perrin

_______________________________________________
List: Catalyst@...
Listinfo: http://lists.rawmode.org/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/catalyst@.../
Dev site: http://dev.catalyst.perl.org/

[Matt S Trout-2]

On Sun, Sep 23, 2007 at 01:42:00PM -0400, Perrin Harkins wrote:
> On 9/23/07, Matt S Trout <dbix-class@...> wrote:
> > Do you have any thoughts on how to paper over DBIC's ability to chain searches
> > vs. the lack of that feature in RDBO?
>
> Isn't it primarily a way to build up your search criteria with
> multiple small method calls instead of one big one?  It doesn't seem
> very relevant to a CRUD app where all the criteria come in at once
> from a web form.

You've never implemented a saved search function that allows you to
'search within' a saved search?

You've never done a "view all posts owned by the selected set of users" link?

This sort of thing in DBIC just becomes

$posts_rs = $users_rs->search_related('posts');

whereas (so far as I'm aware, please do correct me if I'm wrong) it's a bit
more work in RDBO.

--
      Matt S Trout       Need help with your Catalyst or DBIx::Class project?
   Technical Director                    http://www.shadowcat.co.uk/catalyst/
 Shadowcat Systems Ltd.  Want a managed development or deployment platform?
http://chainsawblues.vox.com/            http://www.shadowcat.co.uk/servers/

_______________________________________________
List: Catalyst@...
Listinfo: http://lists.rawmode.org/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/catalyst@.../
Dev site: http://dev.catalyst.perl.org/

[Perrin Harkins]

On 9/23/07, Matt S Trout <dbix-class@...> wrote:
> You've never implemented a saved search function that allows you to
> 'search within' a saved search?

That's usually not a difficult problem, since you already have a way
to turn criteria from a web form into a search, and you're just adding
one more criterion to the pile.

> You've never done a "view all posts owned by the selected set of users" link?
>
> This sort of thing in DBIC just becomes
>
> $posts_rs = $users_rs->search_related('posts');
>
> whereas (so far as I'm aware, please do correct me if I'm wrong) it's a bit
> more work in RDBO.

I'm not an expert on RDBO, but I believe you would need to call a
different class (the posts class, not the users) and adjust the
criteria you pass, e.g. change "last_name => 'Smith'" to
"user.last_name => 'Smith'".

Any set of ORM tools is going to have different unique features, and a
full abstraction would mean giving up all of the unique features.
However, it looked like Peter was trying to do something at a pretty
high level where hopefully the differences don't come into play much.

- Perrin

_______________________________________________
List: Catalyst@...
Listinfo: http://lists.rawmode.org/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/catalyst@.../
Dev site: http://dev.catalyst.perl.org/

[Matt S Trout-2]

On Sun, Sep 23, 2007 at 09:31:35PM -0400, Perrin Harkins wrote:

> On 9/23/07, Matt S Trout <dbix-class@...> wrote:
> > You've never done a "view all posts owned by the selected set of users" link?
> >
> > This sort of thing in DBIC just becomes
> >
> > $posts_rs = $users_rs->search_related('posts');
> >
> > whereas (so far as I'm aware, please do correct me if I'm wrong) it's a bit
> > more work in RDBO.
>
> I'm not an expert on RDBO, but I believe you would need to call a
> different class (the posts class, not the users) and adjust the
> criteria you pass, e.g. change "last_name => 'Smith'" to
> "user.last_name => 'Smith'".
>
> Any set of ORM tools is going to have different unique features, and a
> full abstraction would mean giving up all of the unique features.
> However, it looked like Peter was trying to do something at a pretty
> high level where hopefully the differences don't come into play much.

So the answer is "no, I don't have any ideas about this". Sure, fair enough.
You could have said that in one line though, I was only wondering.

I'll wait and see whether Peter does.

--
      Matt S Trout       Need help with your Catalyst or DBIx::Class project?
   Technical Director                    http://www.shadowcat.co.uk/catalyst/
 Shadowcat Systems Ltd.  Want a managed development or deployment platform?
http://chainsawblues.vox.com/            http://www.shadowcat.co.uk/servers/

_______________________________________________
List: Catalyst@...
Listinfo: http://lists.rawmode.org/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/catalyst@.../
Dev site: http://dev.catalyst.perl.org/

[Christopher H. Laco]

Matt S Trout wrote:

> On Fri, Sep 21, 2007 at 08:57:35AM -0500, Peter Karman wrote:
>> I've been thinking the last couple days about ways to expand
>> Catalyst::Controller::Rose to play more nicely with other models besides
>> C::M::RDBO. This is per mst's request to open up the RHTMLO goodness to
>> non-RDBO users, and because I now find myself wanting the same thing. I have a
>> model that isn't RDBO that I'd like to use in a project.
>>
>> So I'm proposing the following -- comments/criticism welcome.
>>
>> * CatalystX::CRUD::Model (CXCM)
>>
>> A base class for CRUD-like models. CXCM isa Catalyst::Model. Any CXCM subclass
>> could be used with the C::C::Rose classes (or any other controller that decided
>> to adhere to the CXCM API). C::M::RDBO would become a CXCM subclass.
>>
>> CXCM subclasses would need to implement at least the following methods:
>>
>>  * new_object  - returns CatalystX::CRUD::Object->new()
>>  * fetch       - returns CatalystX::CRUD::Object->new()->read()
>>  * search      - returns zero or more CXCO instances as an arrayref
>>  * interator   - like search() but returns an iterator
>>  * count       - like search() but returns an integer
>
> Have you looked at the way the Handel storage stuff works? That already
> supports both DBIC and RDBO so might be an interesting start.

/me runs and hides.



_______________________________________________
List: Catalyst@...
Listinfo: http://lists.rawmode.org/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/catalyst@.../
Dev site: http://dev.catalyst.perl.org/

[Peter Karman]

On 09/23/2007 11:24 AM, Matt S Trout wrote:
> On Fri, Sep 21, 2007 at 08:57:35AM -0500, Peter Karman wrote:

>> CXCM subclasses would need to implement at least the following methods:
>>
>>  * new_object  - returns CatalystX::CRUD::Object->new()
>>  * fetch       - returns CatalystX::CRUD::Object->new()->read()
>>  * search      - returns zero or more CXCO instances as an arrayref
>>  * interator   - like search() but returns an iterator
>>  * count       - like search() but returns an integer
>
> Have you looked at the way the Handel storage stuff works? That already
> supports both DBIC and RDBO so might be an interesting start.
>

I have looked through it (I had to figure out that Handel-Storage-RDBO is
packaged separately...) and it looks like the base Storage API is much more
complex and full-featured than what I am suggesting. But all the methods I
mention are there in one or another. I'll probably dig in to the DBIC
implementation when/if I need help with that part of CXCM (/me hoping not write
that all by myself...).

> Do you have any thoughts on how to paper over DBIC's ability to chain searches
> vs. the lack of that feature in RDBO? I know most RDBO users don't feel
> they're deprived not having it but DBIC users are generally fairly addicted :)
>

I've read about that feature difference in the mail archives, but since I
haven't used DBIC myself, other than to just play for about an hour, I'm not
sure what the equivalent RDBO code would look like. I do all kinds of multiple
relationships in my RDBO code, where I chain together method calls to get at
related data. Is the difference that RDBO might be making multiple db calls,
whereas DBIC doesn't?

For example, in your reply to Perrin you used this example:

> This sort of thing in DBIC just becomes

> $posts_rs = $users_rs->search_related('posts');

In RDBO, if I wanted all the posts for a given user, I might just say:

 $posts = $user->posts;

So how is that different? Because $users_rs might represent multiple users? If
so, yes RDBO would require something a little longer, like:

 map { push @posts, $_->posts } @$users;

Which, depending on if I pre-fetched the posts when creating $users, might end
up being lots more trips to the db. And it doesn't give me $posts_iterator,
which might be preferable to @posts.

But I think for the purposes of this CatalystX::CRUD proposal, the harder
issues are going to be abstracting the conversion of $c->params() to what gets
passed to search(), iterator(), count(), etc. Right now that is all very
RDBO-specific in CCR::Search.

Couldn't the whole DBIC search_related issue just be resolved by pushing the
issue out into the controller and/or view space? In your CCR::Search subclass:

 my $results = $c->model('My::CatX::CRUD::Model::DBIC')
                      ->search(@params)->search_related('foo');


--
Peter Karman  .  peter@...  .  http://peknet.com/


_______________________________________________
List: Catalyst@...
Listinfo: http://lists.rawmode.org/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/catalyst@.../
Dev site: http://dev.catalyst.perl.org/

[Christopher H. Laco]

Peter Karman wrote:

>
> On 09/23/2007 11:24 AM, Matt S Trout wrote:
>> On Fri, Sep 21, 2007 at 08:57:35AM -0500, Peter Karman wrote:
>
>>> CXCM subclasses would need to implement at least the following methods:
>>>
>>>  * new_object  - returns CatalystX::CRUD::Object->new()
>>>  * fetch       - returns CatalystX::CRUD::Object->new()->read()
>>>  * search      - returns zero or more CXCO instances as an arrayref
>>>  * interator   - like search() but returns an iterator
>>>  * count       - like search() but returns an integer
>> Have you looked at the way the Handel storage stuff works? That already
>> supports both DBIC and RDBO so might be an interesting start.
>>
>
> I have looked through it (I had to figure out that Handel-Storage-RDBO is
> packaged separately...) and it looks like the base Storage API is much more
> complex and full-featured than what I am suggesting. But all the methods I
> mention are there in one or another. I'll probably dig in to the DBIC
> implementation when/if I need help with that part of CXCM (/me hoping not write
> that all by myself...).

Give me a shout out if you need help deciphering the storage layer
madness in Handel. Aside from the add/remove column/constraint/default
values/inflate/deflate magic, it's fairly specific to cart-ing. Then
again, CRUD is CRUD, even if it's named add_item, delete_item, update().

even when it's doing CRUD on something that isn't a db, like XML:
http://search.cpan.org/dist/Handel/lib/Handel/Manual/Cookbook/WritingCustomStorage.pod

-=Chris



_______________________________________________
List: Catalyst@...
Listinfo: http://lists.rawmode.org/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/catalyst@.../
Dev site: http://dev.catalyst.perl.org/

[Matt S Trout-2]

On Mon, Sep 24, 2007 at 10:23:11AM -0500, Peter Karman wrote:

>
>
> On 09/23/2007 11:24 AM, Matt S Trout wrote:
> > On Fri, Sep 21, 2007 at 08:57:35AM -0500, Peter Karman wrote:
>
> >> CXCM subclasses would need to implement at least the following methods:
> >>
> >>  * new_object  - returns CatalystX::CRUD::Object->new()
> >>  * fetch       - returns CatalystX::CRUD::Object->new()->read()
> >>  * search      - returns zero or more CXCO instances as an arrayref
> >>  * interator   - like search() but returns an iterator
> >>  * count       - like search() but returns an integer
> >
> > Have you looked at the way the Handel storage stuff works? That already
> > supports both DBIC and RDBO so might be an interesting start.
> >
>
> I have looked through it (I had to figure out that Handel-Storage-RDBO is
> packaged separately...) and it looks like the base Storage API is much more
> complex and full-featured than what I am suggesting. But all the methods I
> mention are there in one or another. I'll probably dig in to the DBIC
> implementation when/if I need help with that part of CXCM (/me hoping not write
> that all by myself...).

Yeah, don't blame you :)
 

> > Do you have any thoughts on how to paper over DBIC's ability to chain searches
> > vs. the lack of that feature in RDBO? I know most RDBO users don't feel
> > they're deprived not having it but DBIC users are generally fairly addicted :)
> >
>
> I've read about that feature difference in the mail archives, but since I
> haven't used DBIC myself, other than to just play for about an hour, I'm not
> sure what the equivalent RDBO code would look like. I do all kinds of multiple
> relationships in my RDBO code, where I chain together method calls to get at
> related data. Is the difference that RDBO might be making multiple db calls,
> whereas DBIC doesn't?
>
> For example, in your reply to Perrin you used this example:
>
> > This sort of thing in DBIC just becomes
>
> > $posts_rs = $users_rs->search_related('posts');
>
> In RDBO, if I wanted all the posts for a given user, I might just say:
>
>  $posts = $user->posts;
>
> So how is that different? Because $users_rs might represent multiple users? If
> so, yes RDBO would require something a little longer, like:
>
>  map { push @posts, $_->posts } @$users;
>
> Which, depending on if I pre-fetched the posts when creating $users, might end
> up being lots more trips to the db. And it doesn't give me $posts_iterator,
> which might be preferable to @posts.

The main diff here is the DBIC users_rs could represent many users without
ever actually fetching them.

Looking at your example though, I think I'm worrying too much.

> But I think for the purposes of this CatalystX::CRUD proposal, the harder
> issues are going to be abstracting the conversion of $c->params() to what gets
> passed to search(), iterator(), count(), etc. Right now that is all very
> RDBO-specific in CCR::Search.
>
> Couldn't the whole DBIC search_related issue just be resolved by pushing the
> issue out into the controller and/or view space? In your CCR::Search subclass:
>
>  my $results = $c->model('My::CatX::CRUD::Model::DBIC')
>                       ->search(@params)->search_related('foo');

I think we'd just end up providing traverse_related methods that did stuff.

The RDBO implementation might be a little less efficient than the DBIC one
but for the sort of workload we're looking at here it's probably -fairly-
academic performance-wise.

And if it's not, you can go bug siracusa until that changes :)

--
      Matt S Trout       Need help with your Catalyst or DBIx::Class project?
   Technical Director                    http://www.shadowcat.co.uk/catalyst/
 Shadowcat Systems Ltd.  Want a managed development or deployment platform?
http://chainsawblues.vox.com/            http://www.shadowcat.co.uk/servers/

_______________________________________________
List: Catalyst@...
Listinfo: http://lists.rawmode.org/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/catalyst@.../
Dev site: http://dev.catalyst.perl.org/