[jira] Created: (HADOOP-2636) [hbase] Make flusher less dumb

[hbase] Make flusher less dumb
------------------------------

                 Key: HADOOP-2636
                 URL: https://issues.apache.org/jira/browse/HADOOP-2636
             Project: Hadoop
          Issue Type: Improvement
          Components: contrib/hbase
            Reporter: stack
            Priority: Minor


When flusher runs -- its triggered when the sum of all Stores in a Region > a configurable max size -- we flush all Stores though a Store memcache might have but a few bytes.

I would think Stores should only dump their memcache disk if they have some substance.

The problem becomes more acute, the more families you have in a Region.

Possible behaviors would be to dump the biggest Store only, or only those Stores > 50% of max memcache size.  Behavior would vary dependent on the prompt that provoked the flush.  Would also log why the flush is running: optional or > max size.

This issue comes out of HADOOP-2621.



--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

     [ https://issues.apache.org/jira/browse/HADOOP-2636?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Jim Kellerman reassigned HADOOP-2636:
-------------------------------------

    Assignee: Jim Kellerman

> [hbase] Make flusher less dumb
> ------------------------------
>
>                 Key: HADOOP-2636
>                 URL: https://issues.apache.org/jira/browse/HADOOP-2636
>             Project: Hadoop
>          Issue Type: Improvement
>          Components: contrib/hbase
>            Reporter: stack
>            Assignee: Jim Kellerman
>            Priority: Minor
>
> When flusher runs -- its triggered when the sum of all Stores in a Region > a configurable max size -- we flush all Stores though a Store memcache might have but a few bytes.
> I would think Stores should only dump their memcache disk if they have some substance.
> The problem becomes more acute, the more families you have in a Region.
> Possible behaviors would be to dump the biggest Store only, or only those Stores > 50% of max memcache size.  Behavior would vary dependent on the prompt that provoked the flush.  Would also log why the flush is running: optional or > max size.
> This issue comes out of HADOOP-2621.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

     [ https://issues.apache.org/jira/browse/HADOOP-2636?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Jim Kellerman updated HADOOP-2636:
----------------------------------

    Summary: [hbase] Make cache flush triggering less simplistic  (was: [hbase] Make flusher less dumb)

> [hbase] Make cache flush triggering less simplistic
> ---------------------------------------------------
>
>                 Key: HADOOP-2636
>                 URL: https://issues.apache.org/jira/browse/HADOOP-2636
>             Project: Hadoop
>          Issue Type: Improvement
>          Components: contrib/hbase
>            Reporter: stack
>            Assignee: Jim Kellerman
>            Priority: Minor
>
> When flusher runs -- its triggered when the sum of all Stores in a Region > a configurable max size -- we flush all Stores though a Store memcache might have but a few bytes.
> I would think Stores should only dump their memcache disk if they have some substance.
> The problem becomes more acute, the more families you have in a Region.
> Possible behaviors would be to dump the biggest Store only, or only those Stores > 50% of max memcache size.  Behavior would vary dependent on the prompt that provoked the flush.  Would also log why the flush is running: optional or > max size.
> This issue comes out of HADOOP-2621.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

    [ https://issues.apache.org/jira/browse/HADOOP-2636?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12559938#action_12559938 ]

Jim Kellerman commented on HADOOP-2636:
---------------------------------------

Better yet, move triggering of cache flush to the store level instead of the region level. Same for compactions.

Split still has to happen at the region level because it is the region that embodies the concept of row range. However the split could be triggered by a single store reaching the split threshold.

> [hbase] Make flusher less dumb
> ------------------------------
>
>                 Key: HADOOP-2636
>                 URL: https://issues.apache.org/jira/browse/HADOOP-2636
>             Project: Hadoop
>          Issue Type: Improvement
>          Components: contrib/hbase
>            Reporter: stack
>            Priority: Minor
>
> When flusher runs -- its triggered when the sum of all Stores in a Region > a configurable max size -- we flush all Stores though a Store memcache might have but a few bytes.
> I would think Stores should only dump their memcache disk if they have some substance.
> The problem becomes more acute, the more families you have in a Region.
> Possible behaviors would be to dump the biggest Store only, or only those Stores > 50% of max memcache size.  Behavior would vary dependent on the prompt that provoked the flush.  Would also log why the flush is running: optional or > max size.
> This issue comes out of HADOOP-2621.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

    [ https://issues.apache.org/jira/browse/HADOOP-2636?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12560016#action_12560016 ]

stack commented on HADOOP-2636:
-------------------------------

+1 on doing things at store level.

Hard part is keeping account of all the memcaches in all the Stores on all the Regions on an HRS, but thats another issue (smile).

> [hbase] Make cache flush triggering less simplistic
> ---------------------------------------------------
>
>                 Key: HADOOP-2636
>                 URL: https://issues.apache.org/jira/browse/HADOOP-2636
>             Project: Hadoop
>          Issue Type: Improvement
>          Components: contrib/hbase
>            Reporter: stack
>            Assignee: Jim Kellerman
>            Priority: Minor
>
> When flusher runs -- its triggered when the sum of all Stores in a Region > a configurable max size -- we flush all Stores though a Store memcache might have but a few bytes.
> I would think Stores should only dump their memcache disk if they have some substance.
> The problem becomes more acute, the more families you have in a Region.
> Possible behaviors would be to dump the biggest Store only, or only those Stores > 50% of max memcache size.  Behavior would vary dependent on the prompt that provoked the flush.  Would also log why the flush is running: optional or > max size.
> This issue comes out of HADOOP-2621.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

    [ https://issues.apache.org/jira/browse/HADOOP-2636?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12560054#action_12560054 ]

Billy Pearson commented on HADOOP-2636:
---------------------------------------

+1

> [hbase] Make cache flush triggering less simplistic
> ---------------------------------------------------
>
>                 Key: HADOOP-2636
>                 URL: https://issues.apache.org/jira/browse/HADOOP-2636
>             Project: Hadoop
>          Issue Type: Improvement
>          Components: contrib/hbase
>            Reporter: stack
>            Assignee: Jim Kellerman
>            Priority: Minor
>
> When flusher runs -- its triggered when the sum of all Stores in a Region > a configurable max size -- we flush all Stores though a Store memcache might have but a few bytes.
> I would think Stores should only dump their memcache disk if they have some substance.
> The problem becomes more acute, the more families you have in a Region.
> Possible behaviors would be to dump the biggest Store only, or only those Stores > 50% of max memcache size.  Behavior would vary dependent on the prompt that provoked the flush.  Would also log why the flush is running: optional or > max size.
> This issue comes out of HADOOP-2621.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

    [ https://issues.apache.org/jira/browse/HADOOP-2636?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12561369#action_12561369 ]

stack commented on HADOOP-2636:
-------------------------------

.bg [Copied from HADOOP-2674] Optional cache flushing and zero length flushes will be removed in HADOOP-2636

Why will optional cache flushing be removed?  Will anything be done in its place?

> [hbase] Make cache flush triggering less simplistic
> ---------------------------------------------------
>
>                 Key: HADOOP-2636
>                 URL: https://issues.apache.org/jira/browse/HADOOP-2636
>             Project: Hadoop
>          Issue Type: Improvement
>          Components: contrib/hbase
>            Reporter: stack
>            Assignee: Jim Kellerman
>            Priority: Minor
>
> When flusher runs -- its triggered when the sum of all Stores in a Region > a configurable max size -- we flush all Stores though a Store memcache might have but a few bytes.
> I would think Stores should only dump their memcache disk if they have some substance.
> The problem becomes more acute, the more families you have in a Region.
> Possible behaviors would be to dump the biggest Store only, or only those Stores > 50% of max memcache size.  Behavior would vary dependent on the prompt that provoked the flush.  Would also log why the flush is running: optional or > max size.
> This issue comes out of HADOOP-2621.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

    [ https://issues.apache.org/jira/browse/HADOOP-2636?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12561371#action_12561371 ]

stack commented on HADOOP-2636:
-------------------------------

.bq [Copied from HADOOP-2674] ... size reported is the mapfile size.  In HADOOP-2636, it will be the size of the memcache.

For kicks, you might log both memcache size -- in bytes and count -- and resultant log file size.

> [hbase] Make cache flush triggering less simplistic
> ---------------------------------------------------
>
>                 Key: HADOOP-2636
>                 URL: https://issues.apache.org/jira/browse/HADOOP-2636
>             Project: Hadoop
>          Issue Type: Improvement
>          Components: contrib/hbase
>            Reporter: stack
>            Assignee: Jim Kellerman
>            Priority: Minor
>
> When flusher runs -- its triggered when the sum of all Stores in a Region > a configurable max size -- we flush all Stores though a Store memcache might have but a few bytes.
> I would think Stores should only dump their memcache disk if they have some substance.
> The problem becomes more acute, the more families you have in a Region.
> Possible behaviors would be to dump the biggest Store only, or only those Stores > 50% of max memcache size.  Behavior would vary dependent on the prompt that provoked the flush.  Would also log why the flush is running: optional or > max size.
> This issue comes out of HADOOP-2621.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

    [ https://issues.apache.org/jira/browse/HADOOP-2636?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12561385#action_12561385 ]

Jim Kellerman commented on HADOOP-2636:
---------------------------------------

> Why will optional cache flushing be removed? Will anything be done in its place?

Optional cache flushing will be removed as it will no longer be necessary. Cache flushes will happen when the cache fills or when the region is closed. All the information is in the HLog anyway.

> For kicks, you might log both memcache size - in bytes and count - and resultant log file size.

+1

> [hbase] Make cache flush triggering less simplistic
> ---------------------------------------------------
>
>                 Key: HADOOP-2636
>                 URL: https://issues.apache.org/jira/browse/HADOOP-2636
>             Project: Hadoop
>          Issue Type: Improvement
>          Components: contrib/hbase
>            Reporter: stack
>            Assignee: Jim Kellerman
>            Priority: Minor
>
> When flusher runs -- its triggered when the sum of all Stores in a Region > a configurable max size -- we flush all Stores though a Store memcache might have but a few bytes.
> I would think Stores should only dump their memcache disk if they have some substance.
> The problem becomes more acute, the more families you have in a Region.
> Possible behaviors would be to dump the biggest Store only, or only those Stores > 50% of max memcache size.  Behavior would vary dependent on the prompt that provoked the flush.  Would also log why the flush is running: optional or > max size.
> This issue comes out of HADOOP-2621.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

     [ https://issues.apache.org/jira/browse/HADOOP-2636?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Jim Kellerman updated HADOOP-2636:
----------------------------------

        Fix Version/s: 0.17.0
             Priority: Major  (was: Minor)
    Affects Version/s: 0.16.0

> [hbase] Make cache flush triggering less simplistic
> ---------------------------------------------------
>
>                 Key: HADOOP-2636
>                 URL: https://issues.apache.org/jira/browse/HADOOP-2636
>             Project: Hadoop
>          Issue Type: Improvement
>          Components: contrib/hbase
>    Affects Versions: 0.16.0
>            Reporter: stack
>            Assignee: Jim Kellerman
>             Fix For: 0.17.0
>
>
> When flusher runs -- its triggered when the sum of all Stores in a Region > a configurable max size -- we flush all Stores though a Store memcache might have but a few bytes.
> I would think Stores should only dump their memcache disk if they have some substance.
> The problem becomes more acute, the more families you have in a Region.
> Possible behaviors would be to dump the biggest Store only, or only those Stores > 50% of max memcache size.  Behavior would vary dependent on the prompt that provoked the flush.  Would also log why the flush is running: optional or > max size.
> This issue comes out of HADOOP-2621.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.