Query closures for JDO 2.2

[Craig L Russell]

As you know, I'm not fond of the non-object style of query update,  
and there are cases where you want to do a bulk update efficiently in  
the datastore, but SQL just doesn't capture the object mapping that  
is used in the rest of the application.

So what about defining a query closure, that is, declare a set of  
statements that is executed for each qualifying instance that  
satisfies the filter.

This could execute in memory for memory collections, or in the  
datastore for non-instantiated collections. For portability we need  
to define the closure in terms that can be mapped directly to SQL.

For example,

PERFORM salary *= 1. + :percent; lastSalaryAdjustment = :date FROM  
Employee WHERE rating == :rating

Query q = pm.newQuery(Employee, "rating > :rating");
q.setPerform("salary *= 1.06; lastSalaryAdjustment = :date");
q.execute(.06, new Date(), 7);


Craig Russell
Architect, Sun Java Enterprise System http://java.sun.com/products/jdo
408 276-5638 mailto:Craig.Russell@...
P.S. A good JDO? O, Gasp!

[Wesley Biggs]

+1.  I like the "perform" term; it reinforces the abstraction of
"select these into Java, then perform this action" -- i.e. you get
side effects.

Wes

P.S. Your code example needs 1 + :percent instead of 1.06.

On 01/12/2007, Craig L Russell <Craig.Russell@...> wrote:

> As you know, I'm not fond of the non-object style of query update,
> and there are cases where you want to do a bulk update efficiently in
> the datastore, but SQL just doesn't capture the object mapping that
> is used in the rest of the application.
>
> So what about defining a query closure, that is, declare a set of
> statements that is executed for each qualifying instance that
> satisfies the filter.
>
> This could execute in memory for memory collections, or in the
> datastore for non-instantiated collections. For portability we need
> to define the closure in terms that can be mapped directly to SQL.
>
> For example,
>
> PERFORM salary *= 1. + :percent; lastSalaryAdjustment = :date FROM
> Employee WHERE rating == :rating
>
> Query q = pm.newQuery(Employee, "rating > :rating");
> q.setPerform("salary *= 1.06; lastSalaryAdjustment = :date");
> q.execute(.06, new Date(), 7);
>
>
> Craig Russell
> Architect, Sun Java Enterprise System http://java.sun.com/products/jdo
> 408 276-5638 mailto:Craig.Russell@...
> P.S. A good JDO? O, Gasp!
>
>
>

[Michael Bouschen-2]

Hi,

during the JDO TCK conference call last Friday a question came up about
possible restrictions on such a perform block. SQL does not allow
multiple tables in the FROM clause of an UPDATE statement. So should we
put restrictions on JDOQL to allow a 1-1 mapping to SQL? Issues to be
considered: update of fields mapped to a secondary table, relationship
updates and use of variables to update related instances. However,
multiple SQL statements and/or subqueries might be needed anyway for
inheritance and existential queries (meaning relationship navigation in
the WHERE clause).

Another question is about the term "perform". It reinforces the idea
that you select a number of Java instances and perform some block action
on them. However, people might expect to use the word "update" for such
an operation.

What do you think?

Regards Michael

> +1: bulk update is efficient feature.
>
> 2007/12/1, Wesley Biggs <wesley@...>:
>  
>> +1.  I like the "perform" term; it reinforces the abstraction of
>> "select these into Java, then perform this action" -- i.e. you get
>> side effects.
>>
>> Wes
>>
>> P.S. Your code example needs 1 + :percent instead of 1.06.
>>
>> On 01/12/2007, Craig L Russell <Craig.Russell@...> wrote:
>>    
>>> As you know, I'm not fond of the non-object style of query update,
>>> and there are cases where you want to do a bulk update efficiently in
>>> the datastore, but SQL just doesn't capture the object mapping that
>>> is used in the rest of the application.
>>>
>>> So what about defining a query closure, that is, declare a set of
>>> statements that is executed for each qualifying instance that
>>> satisfies the filter.
>>>
>>> This could execute in memory for memory collections, or in the
>>> datastore for non-instantiated collections. For portability we need
>>> to define the closure in terms that can be mapped directly to SQL.
>>>
>>> For example,
>>>
>>> PERFORM salary *= 1. + :percent; lastSalaryAdjustment = :date FROM
>>> Employee WHERE rating == :rating
>>>
>>> Query q = pm.newQuery(Employee, "rating > :rating");
>>> q.setPerform("salary *= 1.06; lastSalaryAdjustment = :date");
>>> q.execute(.06, new Date(), 7);
>>>
>>>
>>> Craig Russell
>>> Architect, Sun Java Enterprise System http://java.sun.com/products/jdo
>>> 408 276-5638 mailto:Craig.Russell@...
>>> P.S. A good JDO? O, Gasp!
>>>
>>>
>>>
>>>      
>>    
>
>
>  

--
Tech@Spree Engineering GmbH  Tel.: [phone]
Buelowstr. 66                Fax.: [phone]
10783 Berlin                 mailto:mbo.tech@...
 
Geschaeftsfuehrung: Anna-Kristin Proefrock
Sitz Berlin, Amtsgericht Charlottenburg, HRB 564 52

[Erik Bengtson-2]

The implementation can do the necessary in 1 or more statements as you said.

--   BlackBerry® from Mobistar    ---

-----Original Message-----
From: Michael Bouschen <mbo.tech@...>

Date: Sun, 09 Dec 2007 22:22:27
To:Apache JDO project <jdo-dev@...>
Cc:JDO Expert Group <jdo-experts-ext@...>
Subject: Re: Query closures for JDO 2.2


Hi,

during the JDO TCK conference call last Friday a question came up about
possible restrictions on such a perform block. SQL does not allow
multiple tables in the FROM clause of an UPDATE statement. So should we
put restrictions on JDOQL to allow a 1-1 mapping to SQL? Issues to be
considered: update of fields mapped to a secondary table, relationship
updates and use of variables to update related instances. However,
multiple SQL statements and/or subqueries might be needed anyway for
inheritance and existential queries (meaning relationship navigation in
the WHERE clause).

Another question is about the term "perform". It reinforces the idea
that you select a number of Java instances and perform some block action
on them. However, people might expect to use the word "update" for such
an operation.

What do you think?

Regards Michael

> +1: bulk update is efficient feature.
>
> 2007/12/1, Wesley Biggs <wesley@...>:
>  
>> +1.  I like the "perform" term; it reinforces the abstraction of
>> "select these into Java, then perform this action" -- i.e. you get
>> side effects.
>>
>> Wes
>>
>> P.S. Your code example needs 1 + :percent instead of 1.06.
>>
>> On 01/12/2007, Craig L Russell <Craig.Russell@...> wrote:
>>    
>>> As you know, I'm not fond of the non-object style of query update,
>>> and there are cases where you want to do a bulk update efficiently in
>>> the datastore, but SQL just doesn't capture the object mapping that
>>> is used in the rest of the application.
>>>
>>> So what about defining a query closure, that is, declare a set of
>>> statements that is executed for each qualifying instance that
>>> satisfies the filter.
>>>
>>> This could execute in memory for memory collections, or in the
>>> datastore for non-instantiated collections. For portability we need
>>> to define the closure in terms that can be mapped directly to SQL.
>>>
>>> For example,
>>>
>>> PERFORM salary *= 1. + :percent; lastSalaryAdjustment = :date FROM
>>> Employee WHERE rating == :rating
>>>
>>> Query q = pm.newQuery(Employee, "rating > :rating");
>>> q.setPerform("salary *= 1.06; lastSalaryAdjustment = :date");
>>> q.execute(.06, new Date(), 7);
>>>
>>>
>>> Craig Russell
>>> Architect, Sun Java Enterprise System http://java.sun.com/products/jdo
>>> 408 276-5638 mailto:Craig.Russell@...
>>> P.S. A good JDO? O, Gasp!
>>>
>>>
>>>
>>>      
>>    
>
>
>  

--
Tech@Spree Engineering GmbH  Tel.: [phone]
Buelowstr. 66                Fax.: [phone]
10783 Berlin                 mailto:mbo.tech@...
 
Geschaeftsfuehrung: Anna-Kristin Proefrock
Sitz Berlin, Amtsgericht Charlottenburg, HRB 564 52

[Christiaan]

Hi,
since you are calling this "perform" would there also be a possibility to somehow incorporate my copy by query feature request?
http://www.nabble.com/Feature-request-3A-copy-by-query-to13855888.html

As a side and practical note, I am working in a domain where many objects are involved, so for performance and memory issues it would really benefit from these "bulk" operations. The delete-by-query was a real must have in that regard. However, in the field I am working, I doubt whether update-by-query would be of the same added value for two reasons:
1) updating in our domain often involves some complex logic, eg. values should be updated based on values in other (children) objects and other factors. Could this be expressed in the closure? (and if it is, wouldn't that be too much going against the OO paradigm)
2) our experience is that these update operations can be done really quick within reasonable memory usage due to fetchgroups (you only load what you need) and fetchsize (having the right value really speeds up iterating).

On the other hand, as mentioned in the copy-by-query request, copying can be a very time and memory consuming task, since both original and copy need to be completely in memory, while most of the time there is no need to load them into memory.

kind regards,
Christiaan