Query objects and hashCode() + equals()

[Joerg von Frantzius-2]

Hi,

it would be nice if the spec would mandate that implementations of
javax.jdo.Query do correctly implement hashCode() and equals().
Alternatively, Query.toString() could be required to return the
single-string representation of the query, or some dedicated method was
added to provide it.

This would make it easier to e.g. implement some custom cache for query
results, or, more generally, it would make it easier to put Query
objects as keys into Maps.

Regards,
Jörg

--
____________________________________________________________________
artnology GmbH - Milastraße 4 - 10437 Berlin - Germany
Geschäftsführer: Ekkehard Blome (CEO), Felix Kuschnick (CCO)
Registergericht: Amtsgericht Berlin Charlottenburg HRB 76376
UST-Id. DE 217652550

[Craig L Russell]

Hi Jörg,

On Oct 16, 2009, at 9:50 AM, Joerg von Frantzius wrote:

> Hi,
>
> it would be nice if the spec would mandate that implementations of
> javax.jdo.Query do correctly implement hashCode() and equals().

As users execute mutator methods on queries, the string representation  
would change. And for proper use in sets or map keys, hashCode and  
equals should be  immutable after construction. So I don't see how we  
can mandate that hashCode and equals rely in any internal state.

> Alternatively, Query.toString() could be required to return the
> single-string representation of the query, or some dedicated method  
> was
> added to provide it.
>
Having the single-string version of a query is useful but adds  
implementation complexity. While there is a requirement for an  
implementation to parse a single string query into executable form,  
there's no current requirement to create the single string form from  
the internal form.

> This would make it easier to e.g. implement some custom cache for  
> query
> results, or, more generally, it would make it easier to put Query
> objects as keys into Maps.

It seems to me that if you want this functionality at the application  
framework layer, then the framework can mandate that the single string  
form be used and at the time the query is created, the single string  
form be used as the key into a framework map.

Regards,

Craig
> ____________________________________________________________________
> artnology GmbH - Milastraße 4 - 10437 Berlin - Germany
> Geschäftsführer: Ekkehard Blome (CEO), Felix Kuschnick (CCO)
> Registergericht: Amtsgericht Berlin Charlottenburg HRB 76376
> UST-Id. DE 217652550
>

Craig L Russell
Architect, Sun Java Enterprise System http://db.apache.org/jdo
408 276-5638 mailto:Craig.Russell@...
P.S. A good JDO? O, Gasp!

[Joerg von Frantzius-2]

Hi Craig,

you are absolutely right about the Query object being mutable, and this
conflicting with its use as a key in a HashMap. But telling from the
general contracts of Map, equals() and hashCode(), I don't think that
there exists a requirement for Map keys to be immutable. There may also
be other purposes for comparing Query objects using equals(), or Map
implementations that don't require immutability of keys.

I wouldn't ask if datanucleus wasn't already there halfway: datanucleus'
JDO query object delegates to org.datanucleus.store.query.Query, which
already correctly implements equals(), hashCode() and toString(). The
only thing missing is the wrapper query object to also delegate these
methods (I guess that was simply forgotten when that additional layer
was introduced with JPOX implementing JPA).

Just a small thing, I hope I'll find the time to fix it in datanucleus
myself. When that has happened, it could still be worth putting in the spec?

Regards,
Jörg

On 10/23/2009 06:57 PM, Craig L Russell wrote:

> Hi Jörg,
>
> On Oct 16, 2009, at 9:50 AM, Joerg von Frantzius wrote:
>
>> Hi,
>>
>> it would be nice if the spec would mandate that implementations of
>> javax.jdo.Query do correctly implement hashCode() and equals().
>
> As users execute mutator methods on queries, the string representation
> would change. And for proper use in sets or map keys, hashCode and
> equals should be  immutable after construction. So I don't see how we
> can mandate that hashCode and equals rely in any internal state.
>
>> Alternatively, Query.toString() could be required to return the
>> single-string representation of the query, or some dedicated method was
>> added to provide it.
>>
> Having the single-string version of a query is useful but adds
> implementation complexity. While there is a requirement for an
> implementation to parse a single string query into executable form,
> there's no current requirement to create the single string form from
> the internal form.
>
>> This would make it easier to e.g. implement some custom cache for query
>> results, or, more generally, it would make it easier to put Query
>> objects as keys into Maps.
>
> It seems to me that if you want this functionality at the application
> framework layer, then the framework can mandate that the single string
> form be used and at the time the query is created, the single string
> form be used as the key into a framework map.
>
> Regards,
>
> Craig
>> ____________________________________________________________________
>> artnology GmbH - Milastraße 4 - 10437 Berlin - Germany
>> Geschäftsführer: Ekkehard Blome (CEO), Felix Kuschnick (CCO)
>> Registergericht: Amtsgericht Berlin Charlottenburg HRB 76376
>> UST-Id. DE 217652550
>>
>
> Craig L Russell
> Architect, Sun Java Enterprise System http://db.apache.org/jdo
> 408 276-5638 mailto:Craig.Russell@...
> P.S. A good JDO? O, Gasp!
>

--
____________________________________________________________________
artnology GmbH - Milastraße 4 - 10437 Berlin - Germany
Geschäftsführer: Ekkehard Blome (CEO), Felix Kuschnick (CCO)
Registergericht: Amtsgericht Berlin Charlottenburg HRB 76376
UST-Id. DE 217652550

[Craig L Russell]

Hi Jörg,

On Nov 16, 2009, at 8:32 AM, Joerg von Frantzius wrote:

> Hi Craig,
>
> you are absolutely right about the Query object being mutable, and  
> this
> conflicting with its use as a key in a HashMap. But telling from the
> general contracts of Map, equals() and hashCode(), I don't think that
> there exists a requirement for Map keys to be immutable.

Right. What I said is that for use as an element in a Set or as a key  
in a Map, the instance should return the identical hashCode and behave  
identically with respect to equals. So while the instance itself does  
not have to be immutable, hashCode and equals should act on immutable  
properties of the instance.

As I understand how these collections work, if you do change how  
hashCode and equals behave after construction, the instance should be  
removed from the collection and added again after the change is done.  
This is pretty tricky in terms of synchronization and order of  
operation.

> There may also
> be other purposes for comparing Query objects using equals(), or Map
> implementations that don't require immutability of keys.
>
> I wouldn't ask if datanucleus wasn't already there halfway:  
> datanucleus'
> JDO query object delegates to org.datanucleus.store.query.Query, which
> already correctly implements equals(), hashCode() and toString().

So does the query object change hashCode when the filter changes?

> The
> only thing missing is the wrapper query object to also delegate these
> methods (I guess that was simply forgotten when that additional layer
> was introduced with JPOX implementing JPA).
>
> Just a small thing, I hope I'll find the time to fix it in datanucleus
> myself. When that has happened, it could still be worth putting in  
> the spec?

I don't yet understand the use case. Can you be specific as to what  
behavior you expect from a "proper implementation" of hashCode,  
equals, and toString, and how the implementation would be exploited by  
an application?

Regards,

Craig

>
> Regards,
> Jörg
>
> On 10/23/2009 06:57 PM, Craig L Russell wrote:
>> Hi Jörg,
>>
>> On Oct 16, 2009, at 9:50 AM, Joerg von Frantzius wrote:
>>
>>> Hi,
>>>
>>> it would be nice if the spec would mandate that implementations of
>>> javax.jdo.Query do correctly implement hashCode() and equals().
>>
>> As users execute mutator methods on queries, the string  
>> representation
>> would change. And for proper use in sets or map keys, hashCode and
>> equals should be  immutable after construction. So I don't see how we
>> can mandate that hashCode and equals rely in any internal state.
>>
>>> Alternatively, Query.toString() could be required to return the
>>> single-string representation of the query, or some dedicated  
>>> method was
>>> added to provide it.
>>>
>> Having the single-string version of a query is useful but adds
>> implementation complexity. While there is a requirement for an
>> implementation to parse a single string query into executable form,
>> there's no current requirement to create the single string form from
>> the internal form.
>>
>>> This would make it easier to e.g. implement some custom cache for  
>>> query
>>> results, or, more generally, it would make it easier to put Query
>>> objects as keys into Maps.
>>
>> It seems to me that if you want this functionality at the application
>> framework layer, then the framework can mandate that the single  
>> string
>> form be used and at the time the query is created, the single string
>> form be used as the key into a framework map.
>>
>> Regards,
>>
>> Craig
>>> ____________________________________________________________________
>>> artnology GmbH - Milastraße 4 - 10437 Berlin - Germany
>>> Geschäftsführer: Ekkehard Blome (CEO), Felix Kuschnick (CCO)
>>> Registergericht: Amtsgericht Berlin Charlottenburg HRB 76376
>>> UST-Id. DE 217652550
>>>
>>
>> Craig L Russell
>> Architect, Sun Java Enterprise System http://db.apache.org/jdo
>> 408 276-5638 mailto:Craig.Russell@...
>> P.S. A good JDO? O, Gasp!
>>
>
>
> --
> ____________________________________________________________________
> artnology GmbH - Milastraße 4 - 10437 Berlin - Germany
> Geschäftsführer: Ekkehard Blome (CEO), Felix Kuschnick (CCO)
> Registergericht: Amtsgericht Berlin Charlottenburg HRB 76376
> UST-Id. DE 217652550
>

Craig L Russell
Architect, Sun Java Enterprise System http://db.apache.org/jdo
408 276-5638 mailto:Craig.Russell@...
P.S. A good JDO? O, Gasp!

[Joerg von Frantzius-2]

Hi Craig,

please see below...

On 11/17/2009 12:52 AM, Craig L Russell wrote:

[..]
> Right. What I said is that for use as an element in a Set or as a key
> in a Map, the instance should return the identical hashCode and behave
> identically with respect to equals. So while the instance itself does
> not have to be immutable, hashCode and equals should act on immutable
> properties of the instance.
I didn't find a hint in the javadocs that hashCode and equals should
only act on immutable properties of the instance. But admittedly that's
just nit-picking on my side, as I do agree that the Query objects must
remain unchanged once they are put in a Set or used as a key in a HashMap.
>
> As I understand how these collections work, if you do change how
> hashCode and equals behave after construction, the instance should be
> removed from the collection and added again after the change is done.
> This is pretty tricky in terms of synchronization and order of operation.
That's true. So an application framework which uses Query objects as
keys in a Map must not hand out Query objects to application code (and
of course, once a Query object is put in a map, the framework must not
invoke mutators on it).

[..]

>> I wouldn't ask if datanucleus wasn't already there halfway: datanucleus'
>> JDO query object delegates to org.datanucleus.store.query.Query, which
>> already correctly implements equals(), hashCode() and toString().
>
>
> So does the query object change hashCode when the filter changes?
Yes, it does.

>
>> [..]
>> Just a small thing, I hope I'll find the time to fix it in datanucleus
>> myself. When that has happened, it could still be worth putting in
>> the spec?
>
> I don't yet understand the use case. Can you be specific as to what
> behavior you expect from a "proper implementation" of hashCode,
> equals, and toString, and how the implementation would be exploited by
> an application?

Generally speaking, two Query objects should be equal if they represent

the same query, i.e. they are equal if their execution at an ideally
same point in time would yield the same result. The implementation of
hashCode() should follow and fulfill the usual contract between equals()
and hashCode().

If that was given, I wouldn't have any requirements on the
implementation of toString(). Currently datanucleus returns the
single-string representation of the query, which is just great.

Our application framework caches the results of certain queries. It does
not hand out the respective Query objects to application code, and it
does not itself invoke any mutators on these Query objects. A query
result caching like that currently is hard to do when it is solely based
on the spec, because neither equals(), hashCode() or toString() can
reliably be used. It also is impossible for application (framework) code
itself to build a cache key from a given Query object, because the Query
interface does not provide any getters, e.g. for its filter etc. The
only solution would be to separately remember the properties of the
Query object when it is built.

Regards,
Jörg

>
> Regards,
>
> Craig
>>
>> Regards,
>> Jörg
>>
>> On 10/23/2009 06:57 PM, Craig L Russell wrote:
>>> Hi Jörg,
>>>
>>> On Oct 16, 2009, at 9:50 AM, Joerg von Frantzius wrote:
>>>
>>>> Hi,
>>>>
>>>> it would be nice if the spec would mandate that implementations of
>>>> javax.jdo.Query do correctly implement hashCode() and equals().
>>>
>>> As users execute mutator methods on queries, the string representation
>>> would change. And for proper use in sets or map keys, hashCode and
>>> equals should be  immutable after construction. So I don't see how we
>>> can mandate that hashCode and equals rely in any internal state.
>>>
>>>> Alternatively, Query.toString() could be required to return the
>>>> single-string representation of the query, or some dedicated method
>>>> was
>>>> added to provide it.
>>>>
>>> Having the single-string version of a query is useful but adds
>>> implementation complexity. While there is a requirement for an
>>> implementation to parse a single string query into executable form,
>>> there's no current requirement to create the single string form from
>>> the internal form.
>>>
>>>> This would make it easier to e.g. implement some custom cache for
>>>> query
>>>> results, or, more generally, it would make it easier to put Query
>>>> objects as keys into Maps.
>>>
>>> It seems to me that if you want this functionality at the application
>>> framework layer, then the framework can mandate that the single string
>>> form be used and at the time the query is created, the single string
>>> form be used as the key into a framework map.
>>>
>>> Regards,
>>>
>>> Craig
>>>> ____________________________________________________________________
>>>> artnology GmbH - Milastraße 4 - 10437 Berlin - Germany
>>>> Geschäftsführer: Ekkehard Blome (CEO), Felix Kuschnick (CCO)
>>>> Registergericht: Amtsgericht Berlin Charlottenburg HRB 76376
>>>> UST-Id. DE 217652550
>>>>
>>>
>>> Craig L Russell
>>> Architect, Sun Java Enterprise System http://db.apache.org/jdo
>>> 408 276-5638 mailto:Craig.Russell@...
>>> P.S. A good JDO? O, Gasp!
>>>
>>
>>
>> --
>> ____________________________________________________________________
>> artnology GmbH - Milastraße 4 - 10437 Berlin - Germany
>> Geschäftsführer: Ekkehard Blome (CEO), Felix Kuschnick (CCO)
>> Registergericht: Amtsgericht Berlin Charlottenburg HRB 76376
>> UST-Id. DE 217652550
>>
>
> Craig L Russell
> Architect, Sun Java Enterprise System http://db.apache.org/jdo
> 408 276-5638 mailto:Craig.Russell@...
> P.S. A good JDO? O, Gasp!
>

--
____________________________________________________________________
artnology GmbH - Milastraße 4 - 10437 Berlin - Germany
Geschäftsführer: Ekkehard Blome (CEO), Felix Kuschnick (CCO)
Registergericht: Amtsgericht Berlin Charlottenburg HRB 76376
UST-Id. DE 217652550

[CRomberg]

Hi Jörg,

that the value of hashCode must not change, once an object is stored in a
(hashCode aware) Map, might
not be explicitly stated in the javadoc, but this is a natural
requirement.

The behavior of equals() then can actually change, provided the general
equals/hashCode contract is respected,
although a change of the behavior of equals() is in most cases not
desirable.

Changing the hashCode once an object is in a Map, breaks the Map contract.

Changing equals behavior once an object is in a Map, does not break the
Map contract, but the key
that the value is to mapped to is changed.

For tree structured, not hashCode aware maps, both can change within the
limits of the equals/hashCode contract,
but the total ordering imposed by the Comparator or the Comparable
implementation must remain unchanged.

The easiest implementation is basing hashCode, equals and compareTo on a
(sub)set of the immutable fields.

Cheers,

Christian





From:
Joerg von Frantzius <joerg.von.frantzius@...>
To:
Craig L Russell <Craig.Russell@...>
Cc:
jdo-dev@..., JDO Expert Group <jdo-experts-ext@...>
Date:
17.11.2009 12:55
Subject:
Re: Query objects and hashCode()  + equals()



Hi Craig,

please see below...

On 11/17/2009 12:52 AM, Craig L Russell wrote:

[..]
> Right. What I said is that for use as an element in a Set or as a key
> in a Map, the instance should return the identical hashCode and behave
> identically with respect to equals. So while the instance itself does
> not have to be immutable, hashCode and equals should act on immutable
> properties of the instance.
I didn't find a hint in the javadocs that hashCode and equals should
only act on immutable properties of the instance. But admittedly that's
just nit-picking on my side, as I do agree that the Query objects must
remain unchanged once they are put in a Set or used as a key in a HashMap.
>
> As I understand how these collections work, if you do change how
> hashCode and equals behave after construction, the instance should be
> removed from the collection and added again after the change is done.
> This is pretty tricky in terms of synchronization and order of
operation.
That's true. So an application framework which uses Query objects as
keys in a Map must not hand out Query objects to application code (and
of course, once a Query object is put in a map, the framework must not
invoke mutators on it).

[..]

>> I wouldn't ask if datanucleus wasn't already there halfway:
datanucleus'
>> JDO query object delegates to org.datanucleus.store.query.Query, which
>> already correctly implements equals(), hashCode() and toString().
>
>
> So does the query object change hashCode when the filter changes?
Yes, it does.

>
>> [..]
>> Just a small thing, I hope I'll find the time to fix it in datanucleus
>> myself. When that has happened, it could still be worth putting in
>> the spec?
>
> I don't yet understand the use case. Can you be specific as to what
> behavior you expect from a "proper implementation" of hashCode,
> equals, and toString, and how the implementation would be exploited by
> an application?

Generally speaking, two Query objects should be equal if they represent
the same query, i.e. they are equal if their execution at an ideally
same point in time would yield the same result. The implementation of
hashCode() should follow and fulfill the usual contract between equals()
and hashCode().

If that was given, I wouldn't have any requirements on the
implementation of toString(). Currently datanucleus returns the
single-string representation of the query, which is just great.

Our application framework caches the results of certain queries. It does
not hand out the respective Query objects to application code, and it
does not itself invoke any mutators on these Query objects. A query
result caching like that currently is hard to do when it is solely based
on the spec, because neither equals(), hashCode() or toString() can
reliably be used. It also is impossible for application (framework) code
itself to build a cache key from a given Query object, because the Query
interface does not provide any getters, e.g. for its filter etc. The
only solution would be to separately remember the properties of the
Query object when it is built.

Regards,
Jörg

>
> Regards,
>
> Craig
>>
>> Regards,
>> Jörg
>>
>> On 10/23/2009 06:57 PM, Craig L Russell wrote:
>>> Hi Jörg,
>>>
>>> On Oct 16, 2009, at 9:50 AM, Joerg von Frantzius wrote:
>>>
>>>> Hi,
>>>>
>>>> it would be nice if the spec would mandate that implementations of
>>>> javax.jdo.Query do correctly implement hashCode() and equals().
>>>
>>> As users execute mutator methods on queries, the string representation
>>> would change. And for proper use in sets or map keys, hashCode and
>>> equals should be  immutable after construction. So I don't see how we
>>> can mandate that hashCode and equals rely in any internal state.
>>>
>>>> Alternatively, Query.toString() could be required to return the
>>>> single-string representation of the query, or some dedicated method
>>>> was
>>>> added to provide it.
>>>>
>>> Having the single-string version of a query is useful but adds
>>> implementation complexity. While there is a requirement for an
>>> implementation to parse a single string query into executable form,
>>> there's no current requirement to create the single string form from
>>> the internal form.
>>>
>>>> This would make it easier to e.g. implement some custom cache for
>>>> query
>>>> results, or, more generally, it would make it easier to put Query
>>>> objects as keys into Maps.
>>>
>>> It seems to me that if you want this functionality at the application
>>> framework layer, then the framework can mandate that the single string
>>> form be used and at the time the query is created, the single string
>>> form be used as the key into a framework map.
>>>
>>> Regards,
>>>
>>> Craig
>>>> ____________________________________________________________________
>>>> artnology GmbH - Milastraße 4 - 10437 Berlin - Germany
>>>> Geschäftsführer: Ekkehard Blome (CEO), Felix Kuschnick (CCO)
>>>> Registergericht: Amtsgericht Berlin Charlottenburg HRB 76376
>>>> UST-Id. DE 217652550
>>>>
>>>
>>> Craig L Russell
>>> Architect, Sun Java Enterprise System http://db.apache.org/jdo
>>> 408 276-5638 mailto:Craig.Russell@...
>>> P.S. A good JDO? O, Gasp!
>>>
>>
>>
>> --
>> ____________________________________________________________________
>> artnology GmbH - Milastraße 4 - 10437 Berlin - Germany
>> Geschäftsführer: Ekkehard Blome (CEO), Felix Kuschnick (CCO)
>> Registergericht: Amtsgericht Berlin Charlottenburg HRB 76376
>> UST-Id. DE 217652550
>>
>
> Craig L Russell
> Architect, Sun Java Enterprise System http://db.apache.org/jdo
> 408 276-5638 mailto:Craig.Russell@...
> P.S. A good JDO? O, Gasp!
>

--
____________________________________________________________________
artnology GmbH - Milastraße 4 - 10437 Berlin - Germany
Geschäftsführer: Ekkehard Blome (CEO), Felix Kuschnick (CCO)
Registergericht: Amtsgericht Berlin Charlottenburg HRB 76376
UST-Id. DE 217652550

[attachment "joerg_von_frantzius.vcf" deleted by Christian
Romberg/VERSANT]

[Andy Jefferson-4]

> Our application framework caches the results of certain queries. It does
> not hand out the respective Query objects to application code, and it
> does not itself invoke any mutators on these Query objects.

DataNucleus v2.x also allows caching of query results (for non-legacy JDOQL
implementations). They are not cached by the "Query" object, but by a key that
represents the query string and the input parameters. Consequently no such map
lookup issues happen.


--
Andy
DataNucleus (http://www.datanucleus.org)

[Joerg von Frantzius-2]

Hi Andy,

all I could find in the docs was
http://www.datanucleus.org/extensions/query_cache.html which is about
caching of "query compilations". Or do you mean that it could be used to
store query results along with a cached QueryCompilation object?

On 11/17/2009 06:19 PM, Andy Jefferson wrote:

>> Our application framework caches the results of certain queries. It does
>> not hand out the respective Query objects to application code, and it
>> does not itself invoke any mutators on these Query objects.
>>    
> DataNucleus v2.x also allows caching of query results (for non-legacy JDOQL
> implementations). They are not cached by the "Query" object, but by a key that
> represents the query string and the input parameters. Consequently no such map
> lookup issues happen.
>
>
>  

--
____________________________________________________________________
artnology GmbH - Milastraße 4 - 10437 Berlin - Germany
Geschäftsführer: Ekkehard Blome (CEO), Felix Kuschnick (CCO)
Registergericht: Amtsgericht Berlin Charlottenburg HRB 76376
UST-Id. DE 217652550

[Andy Jefferson-4]

> all I could find in the docs was
> http://www.datanucleus.org/extensions/query_cache.html which is about
> caching of "query compilations". Or do you mean that it could be used to
> store query results along with a cached QueryCompilation object?

PMF prop(s)
http://www.datanucleus.org/products/accessplatform_2_0/persistence_properties.html#query

Query Cache(s)
http://www.datanucleus.org/products/accessplatform_2_0/jdo/query_cache.html

or look at the code ...

--
Andy
DataNucleus (http://www.datanucleus.org)