Question/Suggestion: obtaining older versions of values

[Doug Meil-2]

Hi there-

Question:  how does one obtain older versions of values from Result?

As I understand it, the following Scan instance should return the last 5 versions of values for a table, provided that the table is versioned.

    Scan scan = new Scan();
    scan.setMaxVersions(5);

From the Javadoc, it looks like what I want is in here (from Result)....
getMap

public NavigableMap<http://java.sun.com/javase/6/docs/api/java/util/NavigableMap.html?is-external=true><byte[],NavigableMap<http://java.sun.com/javase/6/docs/api/java/util/NavigableMap.html?is-external=true><byte[],NavigableMap<http://java.sun.com/javase/6/docs/api/java/util/NavigableMap.html?is-external=true><Long<http://java.sun.com/javase/6/docs/api/java/lang/Long.html?is-external=true>,byte[]>>> getMap()
Map of families to all versions of its qualifiers and values.

...  but there is a fair amount of gymnastics involved to get the data.

Suggestion:   can an example of iterating over this map (and contained maps) be added to the Javadoc?  Versioning is a powerful feature of HBase but using it isn't obvious in result processing.

Thanks!


Doug Meil
Director of Engineering
doug.meil@...

[Jonathan Gray-2]

Personally, when I need to dig into a complex result with multiple
columns and versions, I iterate over the KeyValues directly rather than
messing with the Map-based return formats from Result.

In your example, are you just returning versions/values for a single
column?  Maybe we could add some methods that returned subsets of the
total set of KeyValues (for example, Result.getKeyValues(family,
qualifier) would return all versions of that column in the result in
KeyValue format).

Going to drop a note in HBASE-1937.

JG

Doug Meil wrote:

> Hi there-
>
> Question:  how does one obtain older versions of values from Result?
>
> As I understand it, the following Scan instance should return the last 5 versions of values for a table, provided that the table is versioned.
>
>     Scan scan = new Scan();
>     scan.setMaxVersions(5);
>
>>From the Javadoc, it looks like what I want is in here (from Result)....
> getMap
>
> public NavigableMap<http://java.sun.com/javase/6/docs/api/java/util/NavigableMap.html?is-external=true><byte[],NavigableMap<http://java.sun.com/javase/6/docs/api/java/util/NavigableMap.html?is-external=true><byte[],NavigableMap<http://java.sun.com/javase/6/docs/api/java/util/NavigableMap.html?is-external=true><Long<http://java.sun.com/javase/6/docs/api/java/lang/Long.html?is-external=true>,byte[]>>> getMap()
> Map of families to all versions of its qualifiers and values.
>
> ...  but there is a fair amount of gymnastics involved to get the data.
>
> Suggestion:   can an example of iterating over this map (and contained maps) be added to the Javadoc?  Versioning is a powerful feature of HBase but using it isn't obvious in result processing.
>
> Thanks!
>
>
> Doug Meil
> Director of Engineering
> doug.meil@...
>
>

[Doug Meil-2]

Hi Jonathan,

The use-case in question is one column with multiple versions, either to table-display or plot (Y values over X timestamps), and since the insertion time into HBase is good enough for the X dimension, we don't need a second column.

After digging through my test program per your suggestion, I realized the error of my ways.  The history *does* actually come back with the KeyValues when requested.  Doh!  I was so focused on this Javadoc comment from getMap() "Map of families to all versions of its qualifiers and values." that I focused on that method instead of realizing that the data was right in front of me with KeyValue.

I do think that something like what you suggested "Result.getKeyValues(family, qualifier)" would still be useful for other versioned use-cases.

I'd still suggest a few blurbs in the Result Javadoc on obtaining history (and that the methods that return KeyValues contain history when requested).  It's "obvious" now that I understand it, but not so obvious from the existing Javadoc.

I'd be happy to help out with some of this documentation.

Thanks!


-----Original Message-----
From: Jonathan Gray [mailto:jlist@...]
Sent: Monday, October 26, 2009 12:13 PM
To: hbase-user@...
Subject: Re: Question/Suggestion: obtaining older versions of values

Personally, when I need to dig into a complex result with multiple
columns and versions, I iterate over the KeyValues directly rather than
messing with the Map-based return formats from Result.

In your example, are you just returning versions/values for a single
column?  Maybe we could add some methods that returned subsets of the
total set of KeyValues (for example, Result.getKeyValues(family,
qualifier) would return all versions of that column in the result in
KeyValue format).

Going to drop a note in HBASE-1937.

JG

Doug Meil wrote:

> Hi there-
>
> Question:  how does one obtain older versions of values from Result?
>
> As I understand it, the following Scan instance should return the last 5 versions of values for a table, provided that the table is versioned.
>
>     Scan scan = new Scan();
>     scan.setMaxVersions(5);
>
>>From the Javadoc, it looks like what I want is in here (from Result)....
> getMap
>
> public NavigableMap<http://java.sun.com/javase/6/docs/api/java/util/NavigableMap.html?is-external=true><byte[],NavigableMap<http://java.sun.com/javase/6/docs/api/java/util/NavigableMap.html?is-external=true><byte[],NavigableMap<http://java.sun.com/javase/6/docs/api/java/util/NavigableMap.html?is-external=true><Long<http://java.sun.com/javase/6/docs/api/java/lang/Long.html?is-external=true>,byte[]>>> getMap()
> Map of families to all versions of its qualifiers and values.
>
> ...  but there is a fair amount of gymnastics involved to get the data.
>
> Suggestion:   can an example of iterating over this map (and contained maps) be added to the Javadoc?  Versioning is a powerful feature of HBase but using it isn't obvious in result processing.
>
> Thanks!
>
>
> Doug Meil
> Director of Engineering
> doug.meil@...
>
>

[Jonathan Gray-2]

Doug,

It can be difficult for us to fully understand the perspective of a new
user as far as properly documenting and preventing confusion.

Maybe you could drop a couple suggested lines of javadoc into HBASE-1937
and I'd be happy to expand on them and get that committed asap.

I also think the getKeyValues methods would be useful and provide
additional areas of javadoc that would give more clarity as to how to
properly handle results.

Thanks for your help.

JG

Doug Meil wrote:

> Hi Jonathan,
>
> The use-case in question is one column with multiple versions, either to table-display or plot (Y values over X timestamps), and since the insertion time into HBase is good enough for the X dimension, we don't need a second column.
>
> After digging through my test program per your suggestion, I realized the error of my ways.  The history *does* actually come back with the KeyValues when requested.  Doh!  I was so focused on this Javadoc comment from getMap() "Map of families to all versions of its qualifiers and values." that I focused on that method instead of realizing that the data was right in front of me with KeyValue.
>
> I do think that something like what you suggested "Result.getKeyValues(family, qualifier)" would still be useful for other versioned use-cases.
>
> I'd still suggest a few blurbs in the Result Javadoc on obtaining history (and that the methods that return KeyValues contain history when requested).  It's "obvious" now that I understand it, but not so obvious from the existing Javadoc.
>
> I'd be happy to help out with some of this documentation.
>
> Thanks!
>
>
> -----Original Message-----
> From: Jonathan Gray [mailto:jlist@...]
> Sent: Monday, October 26, 2009 12:13 PM
> To: hbase-user@...
> Subject: Re: Question/Suggestion: obtaining older versions of values
>
> Personally, when I need to dig into a complex result with multiple
> columns and versions, I iterate over the KeyValues directly rather than
> messing with the Map-based return formats from Result.
>
> In your example, are you just returning versions/values for a single
> column?  Maybe we could add some methods that returned subsets of the
> total set of KeyValues (for example, Result.getKeyValues(family,
> qualifier) would return all versions of that column in the result in
> KeyValue format).
>
> Going to drop a note in HBASE-1937.
>
> JG
>
> Doug Meil wrote:
>> Hi there-
>>
>> Question:  how does one obtain older versions of values from Result?
>>
>> As I understand it, the following Scan instance should return the last 5 versions of values for a table, provided that the table is versioned.
>>
>>     Scan scan = new Scan();
>>     scan.setMaxVersions(5);
>>
>> >From the Javadoc, it looks like what I want is in here (from Result)....
>> getMap
>>
>> public NavigableMap<http://java.sun.com/javase/6/docs/api/java/util/NavigableMap.html?is-external=true><byte[],NavigableMap<http://java.sun.com/javase/6/docs/api/java/util/NavigableMap.html?is-external=true><byte[],NavigableMap<http://java.sun.com/javase/6/docs/api/java/util/NavigableMap.html?is-external=true><Long<http://java.sun.com/javase/6/docs/api/java/lang/Long.html?is-external=true>,byte[]>>> getMap()
>> Map of families to all versions of its qualifiers and values.
>>
>> ...  but there is a fair amount of gymnastics involved to get the data.
>>
>> Suggestion:   can an example of iterating over this map (and contained maps) be added to the Javadoc?  Versioning is a powerful feature of HBase but using it isn't obvious in result processing.
>>
>> Thanks!
>>
>>
>> Doug Meil
>> Director of Engineering
>> doug.meil@...
>>
>>
>

[stack-3]

Doug, if you file an issue with the javadoc changes only, we can apply that
now ahead of hbase-1937.
Thanks,
St.Ack

On Mon, Oct 26, 2009 at 10:53 AM, Doug Meil
<doug.meil@...>wrote:

> Hi Jonathan,
>
> The use-case in question is one column with multiple versions, either to
> table-display or plot (Y values over X timestamps), and since the insertion
> time into HBase is good enough for the X dimension, we don't need a second
> column.
>
> After digging through my test program per your suggestion, I realized the
> error of my ways.  The history *does* actually come back with the KeyValues
> when requested.  Doh!  I was so focused on this Javadoc comment from
> getMap() "Map of families to all versions of its qualifiers and values."
> that I focused on that method instead of realizing that the data was right
> in front of me with KeyValue.
>
> I do think that something like what you suggested
> "Result.getKeyValues(family, qualifier)" would still be useful for other
> versioned use-cases.
>
> I'd still suggest a few blurbs in the Result Javadoc on obtaining history
> (and that the methods that return KeyValues contain history when requested).
>  It's "obvious" now that I understand it, but not so obvious from the
> existing Javadoc.
>
> I'd be happy to help out with some of this documentation.
>
> Thanks!
>
>
> -----Original Message-----
> From: Jonathan Gray [mailto:jlist@...]
> Sent: Monday, October 26, 2009 12:13 PM
> To: hbase-user@...
> Subject: Re: Question/Suggestion: obtaining older versions of values
>
> Personally, when I need to dig into a complex result with multiple
> columns and versions, I iterate over the KeyValues directly rather than
> messing with the Map-based return formats from Result.
>
> In your example, are you just returning versions/values for a single
> column?  Maybe we could add some methods that returned subsets of the
> total set of KeyValues (for example, Result.getKeyValues(family,
> qualifier) would return all versions of that column in the result in
> KeyValue format).
>
> Going to drop a note in HBASE-1937.
>
> JG
>
> Doug Meil wrote:
> > Hi there-
> >
> > Question:  how does one obtain older versions of values from Result?
> >
> > As I understand it, the following Scan instance should return the last 5
> versions of values for a table, provided that the table is versioned.
> >
> >     Scan scan = new Scan();
> >     scan.setMaxVersions(5);
> >
> >>From the Javadoc, it looks like what I want is in here (from Result)....
> > getMap
> >
> > public NavigableMap<
> http://java.sun.com/javase/6/docs/api/java/util/NavigableMap.html?is-external=true
> ><byte[],NavigableMap<
> http://java.sun.com/javase/6/docs/api/java/util/NavigableMap.html?is-external=true
> ><byte[],NavigableMap<
> http://java.sun.com/javase/6/docs/api/java/util/NavigableMap.html?is-external=true
> ><Long<
> http://java.sun.com/javase/6/docs/api/java/lang/Long.html?is-external=true>,byte[]>>>
> getMap()
> > Map of families to all versions of its qualifiers and values.
> >
> > ...  but there is a fair amount of gymnastics involved to get the data.
> >
> > Suggestion:   can an example of iterating over this map (and contained
> maps) be added to the Javadoc?  Versioning is a powerful feature of HBase
> but using it isn't obvious in result processing.
> >
> > Thanks!
> >
> >
> > Doug Meil
> > Director of Engineering
> > doug.meil@...
> >
> >
>

[Doug Meil-2]

Will do.  Thanks!

-----Original Message-----
From: saint.ack@... [mailto:saint.ack@...] On Behalf Of stack
Sent: Monday, October 26, 2009 1:59 PM
To: hbase-user@...
Subject: Re: Question/Suggestion: obtaining older versions of values

Doug, if you file an issue with the javadoc changes only, we can apply that
now ahead of hbase-1937.
Thanks,
St.Ack

On Mon, Oct 26, 2009 at 10:53 AM, Doug Meil
<doug.meil@...>wrote:

> Hi Jonathan,
>
> The use-case in question is one column with multiple versions, either to
> table-display or plot (Y values over X timestamps), and since the insertion
> time into HBase is good enough for the X dimension, we don't need a second
> column.
>
> After digging through my test program per your suggestion, I realized the
> error of my ways.  The history *does* actually come back with the KeyValues
> when requested.  Doh!  I was so focused on this Javadoc comment from
> getMap() "Map of families to all versions of its qualifiers and values."
> that I focused on that method instead of realizing that the data was right
> in front of me with KeyValue.
>
> I do think that something like what you suggested
> "Result.getKeyValues(family, qualifier)" would still be useful for other
> versioned use-cases.
>
> I'd still suggest a few blurbs in the Result Javadoc on obtaining history
> (and that the methods that return KeyValues contain history when requested).
>  It's "obvious" now that I understand it, but not so obvious from the
> existing Javadoc.
>
> I'd be happy to help out with some of this documentation.
>
> Thanks!
>
>
> -----Original Message-----
> From: Jonathan Gray [mailto:jlist@...]
> Sent: Monday, October 26, 2009 12:13 PM
> To: hbase-user@...
> Subject: Re: Question/Suggestion: obtaining older versions of values
>
> Personally, when I need to dig into a complex result with multiple
> columns and versions, I iterate over the KeyValues directly rather than
> messing with the Map-based return formats from Result.
>
> In your example, are you just returning versions/values for a single
> column?  Maybe we could add some methods that returned subsets of the
> total set of KeyValues (for example, Result.getKeyValues(family,
> qualifier) would return all versions of that column in the result in
> KeyValue format).
>
> Going to drop a note in HBASE-1937.
>
> JG
>
> Doug Meil wrote:
> > Hi there-
> >
> > Question:  how does one obtain older versions of values from Result?
> >
> > As I understand it, the following Scan instance should return the last 5
> versions of values for a table, provided that the table is versioned.
> >
> >     Scan scan = new Scan();
> >     scan.setMaxVersions(5);
> >
> >>From the Javadoc, it looks like what I want is in here (from Result)....
> > getMap
> >
> > public NavigableMap<
> http://java.sun.com/javase/6/docs/api/java/util/NavigableMap.html?is-external=true
> ><byte[],NavigableMap<
> http://java.sun.com/javase/6/docs/api/java/util/NavigableMap.html?is-external=true
> ><byte[],NavigableMap<
> http://java.sun.com/javase/6/docs/api/java/util/NavigableMap.html?is-external=true
> ><Long<
> http://java.sun.com/javase/6/docs/api/java/lang/Long.html?is-external=true>,byte[]>>>
> getMap()
> > Map of families to all versions of its qualifiers and values.
> >
> > ...  but there is a fair amount of gymnastics involved to get the data.
> >
> > Suggestion:   can an example of iterating over this map (and contained
> maps) be added to the Javadoc?  Versioning is a powerful feature of HBase
> but using it isn't obvious in result processing.
> >
> > Thanks!
> >
> >
> > Doug Meil
> > Director of Engineering
> > doug.meil@...
> >
> >
>