|

»

»

»

Hadoop lucene-dev

[jira] Created: (HADOOP-2673) Flusher flushing back to back when it should not be

View: New views

8 Messages — Rating Filter: Alert me

	[jira] Created: (HADOOP-2673) Flusher flushing back to back when it should not be by JIRA jira@apache.org :: Rate this Message: Reply to Author \| View Threaded \| Show Only this Message Flusher flushing back to back when it should not be --------------------------------------------------- Key: HADOOP-2673 URL: https://issues.apache.org/jira/browse/HADOOP-2673 Project: Hadoop Issue Type: Bug Components: contrib/hbase Environment: hadoop svn 613782 Reporter: Billy Pearson Fix For: 0.16.0 Attachments: log.log settings: hbase.regionserver.optionalcacheflushinterval = 300000 (5 mins) hbase.hregion.memcache.flush.size = 67108864 (64MB) I am attaching a log for this everything works fine until the first flush of the memcache. but once it flushes the first time correctly it flushes back to back many times even when it does not exceed one of the two limits above. I have not found but one spot where we update the variable LastFlushTime except when the server starts so this could be the problem but I may be missing it in the code somewhere. If it is getting updated then we need to look if something could be blocking it from updating after a memcache flush. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
	[jira] Updated: (HADOOP-2673) Flusher flushing back to back when it should not be by JIRA jira@apache.org :: Rate this Message: Reply to Author \| View Threaded \| Show Only this Message [ https://issues.apache.org/jira/browse/HADOOP-2673?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Billy Pearson updated HADOOP-2673: ---------------------------------- Attachment: log.log attached snip of log > Flusher flushing back to back when it should not be > --------------------------------------------------- > > Key: HADOOP-2673 > URL: https://issues.apache.org/jira/browse/HADOOP-2673 > Project: Hadoop > Issue Type: Bug > Components: contrib/hbase > Environment: hadoop svn 613782 > Reporter: Billy Pearson > Fix For: 0.16.0 > > Attachments: log.log > > > settings: > hbase.regionserver.optionalcacheflushinterval = 300000 (5 mins) > hbase.hregion.memcache.flush.size = 67108864 (64MB) > I am attaching a log for this everything works fine until the first flush of the memcache. but once it flushes the first time correctly it flushes back to back many times even when it does not exceed one of the two limits above. > I have not found but one spot where we update the variable LastFlushTime except when the server starts so this could be the problem but I may be missing it in the code somewhere. If it is getting updated then we need to look if something could be blocking it from updating after a memcache flush. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
	[jira] Issue Comment Edited: (HADOOP-2673) Flusher flushing back to back when it should not be by JIRA jira@apache.org :: Rate this Message: Reply to Author \| View Threaded \| Show Only this Message [ https://issues.apache.org/jira/browse/HADOOP-2673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12561247#action_12561247 ] viper799 edited comment on HADOOP-2673 at 1/21/08 11:40 PM: ----------------------------------------------------------------- just adding some stats about this problem {code} 2008-01-22 01:07:31,944 DEBUG org.apache.hadoop.hbase.HStore: started compaction of 432 files using hdfs://10.0.0.1:9000/gfs_storage/hadoop-root/hbase/webdata/compaction.dir for 540402979/in_rank 2008-01-22 01:23:17,384 DEBUG org.apache.hadoop.hbase.HStore: moving 540402979/in_rank/8016207671941404930 in hdfs://10.0.0.1:9000/gfs_storage/hadoop-root/hbase/webdata/compaction.dir to 540402979/in_rank/8587356094792327071 in hdfs://10.0.0.1:9000/gfs_storage/hadoop-root/hbase/webdata for 540402979/in_rank 2008-01-22 01:23:22,605 DEBUG org.apache.hadoop.hbase.HStore: started compaction of 328 files using hdfs://10.0.0.1:9000/gfs_storage/hadoop-root/hbase/webdata/compaction.dir for 540402979/rank_visual 2008-01-22 01:23:31,248 DEBUG org.apache.hadoop.hbase.HStore: moving 540402979/rank_visual/4003180598508023113 in hdfs://10.0.0.1:9000/gfs_storage/hadoop-root/hbase/webdata/compaction.dir to 540402979/rank_visual/5001502002807562425 in hdfs://10.0.0.1:9000/gfs_storage/hadoop-root/hbase/webdata for 540402979/rank_visual 2008-01-22 01:23:34,438 INFO org.apache.hadoop.hbase.HRegion: compaction completed on region webdata,,1200970130265. Took 57mins, 10sec 2008-01-22 01:23:34,439 DEBUG org.apache.hadoop.hbase.HStore: compaction for HStore 540402979/size needed. 2008-01-22 01:23:34,439 DEBUG org.apache.hadoop.hbase.HRegion: 540402979/size needs compaction 2008-01-22 01:23:34,439 INFO org.apache.hadoop.hbase.HRegion: starting compaction on region webdata,,1200970130265 2008-01-22 01:23:34,439 DEBUG org.apache.hadoop.hbase.HStore: started compaction of 323 files using hdfs://10.0.0.1:9000/gfs_storage/hadoop-root/hbase/webdata/compaction.dir for 540402979/size {code} after finished compacting 432 files on the last column the compaction took 57min and 10 sec in that time there was 323 more flushes to the first column. was (Author: viper799): just adding some stats about this problem {code} 2008-01-22 01:07:31,944 DEBUG org.apache.hadoop.hbase.HStore: started compaction of 432 files using hdfs://10.0.0.1:9000/gfs_storage/hadoop-root/hbase/webdata/compaction.dir for 540402979/in_rank 2008-01-22 01:23:17,384 DEBUG org.apache.hadoop.hbase.HStore: moving 540402979/in_rank/8016207671941404930 in hdfs://10.0.0.1:9000/gfs_storage/hadoop-root/hbase/webdata/compaction.dir to 540402979/in_rank/8587356094792327071 in hdfs://10.0.0.1:9000/gfs_storage/hadoop-root/hbase/webdata for 540402979/in_rank 2008-01-22 01:23:22,605 DEBUG org.apache.hadoop.hbase.HStore: started compaction of 328 files using hdfs://10.0.0.1:9000/gfs_storage/hadoop-root/hbase/webdata/compaction.dir for 540402979/rank_visual 2008-01-22 01:23:31,248 DEBUG org.apache.hadoop.hbase.HStore: moving 540402979/rank_visual/4003180598508023113 in hdfs://10.0.0.1:9000/gfs_storage/hadoop-root/hbase/webdata/compaction.dir to 540402979/rank_visual/5001502002807562425 in hdfs://10.0.0.1:9000/gfs_storage/hadoop-root/hbase/webdata for 540402979/rank_visual 2008-01-22 01:23:34,438 INFO org.apache.hadoop.hbase.HRegion: compaction completed on region webdata,,1200970130265. Took 57mins, 10sec 2008-01-22 01:23:34,439 DEBUG org.apache.hadoop.hbase.HStore: compaction for HStore 540402979/size needed. 2008-01-22 01:23:34,439 DEBUG org.apache.hadoop.hbase.HRegion: 540402979/size needs compaction 2008-01-22 01:23:34,439 INFO org.apache.hadoop.hbase.HRegion: starting compaction on region webdata,,1200970130265 2008-01-22 01:23:34,439 DEBUG org.apache.hadoop.hbase.HStore: started compaction of 323 files using hdfs://10.0.0.1:9000/gfs_storage/hadoop-root/hbase/webdata/compaction.dir for 540402979/size {code} after finished compacting 432 files on the last column the compaction took 57min and 10 sec in that time there was 323 more flushes. > Flusher flushing back to back when it should not be > --------------------------------------------------- > > Key: HADOOP-2673 > URL: https://issues.apache.org/jira/browse/HADOOP-2673 > Project: Hadoop > Issue Type: Bug > Components: contrib/hbase > Environment: hadoop svn 613782 > Reporter: Billy Pearson > Fix For: 0.16.0 > > Attachments: log.log > > > settings: > hbase.regionserver.optionalcacheflushinterval = 300000 (5 mins) > hbase.hregion.memcache.flush.size = 67108864 (64MB) > I am attaching a log for this everything works fine until the first flush of the memcache. but once it flushes the first time correctly it flushes back to back many times even when it does not exceed one of the two limits above. > I have not found but one spot where we update the variable LastFlushTime except when the server starts so this could be the problem but I may be missing it in the code somewhere. If it is getting updated then we need to look if something could be blocking it from updating after a memcache flush. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
	[jira] Commented: (HADOOP-2673) Flusher flushing back to back when it should not be by JIRA jira@apache.org :: Rate this Message: Reply to Author \| View Threaded \| Show Only this Message [ https://issues.apache.org/jira/browse/HADOOP-2673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12561247#action_12561247 ] Billy Pearson commented on HADOOP-2673: --------------------------------------- just adding some stats about this problem {code} 2008-01-22 01:07:31,944 DEBUG org.apache.hadoop.hbase.HStore: started compaction of 432 files using hdfs://10.0.0.1:9000/gfs_storage/hadoop-root/hbase/webdata/compaction.dir for 540402979/in_rank 2008-01-22 01:23:17,384 DEBUG org.apache.hadoop.hbase.HStore: moving 540402979/in_rank/8016207671941404930 in hdfs://10.0.0.1:9000/gfs_storage/hadoop-root/hbase/webdata/compaction.dir to 540402979/in_rank/8587356094792327071 in hdfs://10.0.0.1:9000/gfs_storage/hadoop-root/hbase/webdata for 540402979/in_rank 2008-01-22 01:23:22,605 DEBUG org.apache.hadoop.hbase.HStore: started compaction of 328 files using hdfs://10.0.0.1:9000/gfs_storage/hadoop-root/hbase/webdata/compaction.dir for 540402979/rank_visual 2008-01-22 01:23:31,248 DEBUG org.apache.hadoop.hbase.HStore: moving 540402979/rank_visual/4003180598508023113 in hdfs://10.0.0.1:9000/gfs_storage/hadoop-root/hbase/webdata/compaction.dir to 540402979/rank_visual/5001502002807562425 in hdfs://10.0.0.1:9000/gfs_storage/hadoop-root/hbase/webdata for 540402979/rank_visual 2008-01-22 01:23:34,438 INFO org.apache.hadoop.hbase.HRegion: compaction completed on region webdata,,1200970130265. Took 57mins, 10sec 2008-01-22 01:23:34,439 DEBUG org.apache.hadoop.hbase.HStore: compaction for HStore 540402979/size needed. 2008-01-22 01:23:34,439 DEBUG org.apache.hadoop.hbase.HRegion: 540402979/size needs compaction 2008-01-22 01:23:34,439 INFO org.apache.hadoop.hbase.HRegion: starting compaction on region webdata,,1200970130265 2008-01-22 01:23:34,439 DEBUG org.apache.hadoop.hbase.HStore: started compaction of 323 files using hdfs://10.0.0.1:9000/gfs_storage/hadoop-root/hbase/webdata/compaction.dir for 540402979/size {code} after finished compacting 432 files on the last column the compaction took 57min and 10 sec in that time there was 323 more flushes. > Flusher flushing back to back when it should not be > --------------------------------------------------- > > Key: HADOOP-2673 > URL: https://issues.apache.org/jira/browse/HADOOP-2673 > Project: Hadoop > Issue Type: Bug > Components: contrib/hbase > Environment: hadoop svn 613782 > Reporter: Billy Pearson > Fix For: 0.16.0 > > Attachments: log.log > > > settings: > hbase.regionserver.optionalcacheflushinterval = 300000 (5 mins) > hbase.hregion.memcache.flush.size = 67108864 (64MB) > I am attaching a log for this everything works fine until the first flush of the memcache. but once it flushes the first time correctly it flushes back to back many times even when it does not exceed one of the two limits above. > I have not found but one spot where we update the variable LastFlushTime except when the server starts so this could be the problem but I may be missing it in the code somewhere. If it is getting updated then we need to look if something could be blocking it from updating after a memcache flush. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
	[jira] Issue Comment Edited: (HADOOP-2673) Flusher flushing back to back when it should not be by JIRA jira@apache.org :: Rate this Message: Reply to Author \| View Threaded \| Show Only this Message [ https://issues.apache.org/jira/browse/HADOOP-2673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12561247#action_12561247 ] viper799 edited comment on HADOOP-2673 at 1/21/08 11:54 PM: ----------------------------------------------------------------- just adding some stats about this problem {code} 2008-01-22 01:07:31,944 DEBUG org.apache.hadoop.hbase.HStore: started compaction of 432 files using hdfs://10.0.0.1:9000/gfs_storage/hadoop-root/hbase/webdata/compaction.dir for 540402979/in_rank 2008-01-22 01:23:17,384 DEBUG org.apache.hadoop.hbase.HStore: moving 540402979/in_rank/8016207671941404930 in hdfs://10.0.0.1:9000/gfs_storage/hadoop-root/hbase/webdata/compaction.dir to 540402979/in_rank/8587356094792327071 in hdfs://10.0.0.1:9000/gfs_storage/hadoop-root/hbase/webdata for 540402979/in_rank 2008-01-22 01:23:22,605 DEBUG org.apache.hadoop.hbase.HStore: started compaction of 328 files using hdfs://10.0.0.1:9000/gfs_storage/hadoop-root/hbase/webdata/compaction.dir for 540402979/rank_visual 2008-01-22 01:23:31,248 DEBUG org.apache.hadoop.hbase.HStore: moving 540402979/rank_visual/4003180598508023113 in hdfs://10.0.0.1:9000/gfs_storage/hadoop-root/hbase/webdata/compaction.dir to 540402979/rank_visual/5001502002807562425 in hdfs://10.0.0.1:9000/gfs_storage/hadoop-root/hbase/webdata for 540402979/rank_visual 2008-01-22 01:23:34,438 INFO org.apache.hadoop.hbase.HRegion: compaction completed on region webdata,,1200970130265. Took 57mins, 10sec 2008-01-22 01:23:34,439 DEBUG org.apache.hadoop.hbase.HStore: compaction for HStore 540402979/size needed. 2008-01-22 01:23:34,439 DEBUG org.apache.hadoop.hbase.HRegion: 540402979/size needs compaction 2008-01-22 01:23:34,439 INFO org.apache.hadoop.hbase.HRegion: starting compaction on region webdata,,1200970130265 2008-01-22 01:23:34,439 DEBUG org.apache.hadoop.hbase.HStore: started compaction of 323 files using hdfs://10.0.0.1:9000/gfs_storage/hadoop-root/hbase/webdata/compaction.dir for 540402979/size {code} after finished compacting 432 files on the last column the compaction took 57min and 10 sec in that time there was 323 more flushes to the first column. the job stop writeing to the table at 01:23:34 was (Author: viper799): just adding some stats about this problem {code} 2008-01-22 01:07:31,944 DEBUG org.apache.hadoop.hbase.HStore: started compaction of 432 files using hdfs://10.0.0.1:9000/gfs_storage/hadoop-root/hbase/webdata/compaction.dir for 540402979/in_rank 2008-01-22 01:23:17,384 DEBUG org.apache.hadoop.hbase.HStore: moving 540402979/in_rank/8016207671941404930 in hdfs://10.0.0.1:9000/gfs_storage/hadoop-root/hbase/webdata/compaction.dir to 540402979/in_rank/8587356094792327071 in hdfs://10.0.0.1:9000/gfs_storage/hadoop-root/hbase/webdata for 540402979/in_rank 2008-01-22 01:23:22,605 DEBUG org.apache.hadoop.hbase.HStore: started compaction of 328 files using hdfs://10.0.0.1:9000/gfs_storage/hadoop-root/hbase/webdata/compaction.dir for 540402979/rank_visual 2008-01-22 01:23:31,248 DEBUG org.apache.hadoop.hbase.HStore: moving 540402979/rank_visual/4003180598508023113 in hdfs://10.0.0.1:9000/gfs_storage/hadoop-root/hbase/webdata/compaction.dir to 540402979/rank_visual/5001502002807562425 in hdfs://10.0.0.1:9000/gfs_storage/hadoop-root/hbase/webdata for 540402979/rank_visual 2008-01-22 01:23:34,438 INFO org.apache.hadoop.hbase.HRegion: compaction completed on region webdata,,1200970130265. Took 57mins, 10sec 2008-01-22 01:23:34,439 DEBUG org.apache.hadoop.hbase.HStore: compaction for HStore 540402979/size needed. 2008-01-22 01:23:34,439 DEBUG org.apache.hadoop.hbase.HRegion: 540402979/size needs compaction 2008-01-22 01:23:34,439 INFO org.apache.hadoop.hbase.HRegion: starting compaction on region webdata,,1200970130265 2008-01-22 01:23:34,439 DEBUG org.apache.hadoop.hbase.HStore: started compaction of 323 files using hdfs://10.0.0.1:9000/gfs_storage/hadoop-root/hbase/webdata/compaction.dir for 540402979/size {code} after finished compacting 432 files on the last column the compaction took 57min and 10 sec in that time there was 323 more flushes to the first column. > Flusher flushing back to back when it should not be > --------------------------------------------------- > > Key: HADOOP-2673 > URL: https://issues.apache.org/jira/browse/HADOOP-2673 > Project: Hadoop > Issue Type: Bug > Components: contrib/hbase > Environment: hadoop svn 613782 > Reporter: Billy Pearson > Fix For: 0.16.0 > > Attachments: log.log > > > settings: > hbase.regionserver.optionalcacheflushinterval = 300000 (5 mins) > hbase.hregion.memcache.flush.size = 67108864 (64MB) > I am attaching a log for this everything works fine until the first flush of the memcache. but once it flushes the first time correctly it flushes back to back many times even when it does not exceed one of the two limits above. > I have not found but one spot where we update the variable LastFlushTime except when the server starts so this could be the problem but I may be missing it in the code somewhere. If it is getting updated then we need to look if something could be blocking it from updating after a memcache flush. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
	[jira] Issue Comment Edited: (HADOOP-2673) Flusher flushing back to back when it should not be by JIRA jira@apache.org :: Rate this Message: Reply to Author \| View Threaded \| Show Only this Message [ https://issues.apache.org/jira/browse/HADOOP-2673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12561247#action_12561247 ] viper799 edited comment on HADOOP-2673 at 1/21/08 11:54 PM: ----------------------------------------------------------------- just adding some stats about this problem {code} 2008-01-22 01:07:31,944 DEBUG org.apache.hadoop.hbase.HStore: started compaction of 432 files using hdfs://10.0.0.1:9000/gfs_storage/hadoop-root/hbase/webdata/compaction.dir for 540402979/in_rank 2008-01-22 01:23:17,384 DEBUG org.apache.hadoop.hbase.HStore: moving 540402979/in_rank/8016207671941404930 in hdfs://10.0.0.1:9000/gfs_storage/hadoop-root/hbase/webdata/compaction.dir to 540402979/in_rank/8587356094792327071 in hdfs://10.0.0.1:9000/gfs_storage/hadoop-root/hbase/webdata for 540402979/in_rank 2008-01-22 01:23:22,605 DEBUG org.apache.hadoop.hbase.HStore: started compaction of 328 files using hdfs://10.0.0.1:9000/gfs_storage/hadoop-root/hbase/webdata/compaction.dir for 540402979/rank_visual 2008-01-22 01:23:31,248 DEBUG org.apache.hadoop.hbase.HStore: moving 540402979/rank_visual/4003180598508023113 in hdfs://10.0.0.1:9000/gfs_storage/hadoop-root/hbase/webdata/compaction.dir to 540402979/rank_visual/5001502002807562425 in hdfs://10.0.0.1:9000/gfs_storage/hadoop-root/hbase/webdata for 540402979/rank_visual 2008-01-22 01:23:34,438 INFO org.apache.hadoop.hbase.HRegion: compaction completed on region webdata,,1200970130265. Took 57mins, 10sec 2008-01-22 01:23:34,439 DEBUG org.apache.hadoop.hbase.HStore: compaction for HStore 540402979/size needed. 2008-01-22 01:23:34,439 DEBUG org.apache.hadoop.hbase.HRegion: 540402979/size needs compaction 2008-01-22 01:23:34,439 INFO org.apache.hadoop.hbase.HRegion: starting compaction on region webdata,,1200970130265 2008-01-22 01:23:34,439 DEBUG org.apache.hadoop.hbase.HStore: started compaction of 323 files using hdfs://10.0.0.1:9000/gfs_storage/hadoop-root/hbase/webdata/compaction.dir for 540402979/size {code} after finished compacting 432 files on the last column the compaction took 57min and 10 sec in that time there was 323 more flushes to the first column. the job stop writeing to the table at 01:04:35 was (Author: viper799): just adding some stats about this problem {code} 2008-01-22 01:07:31,944 DEBUG org.apache.hadoop.hbase.HStore: started compaction of 432 files using hdfs://10.0.0.1:9000/gfs_storage/hadoop-root/hbase/webdata/compaction.dir for 540402979/in_rank 2008-01-22 01:23:17,384 DEBUG org.apache.hadoop.hbase.HStore: moving 540402979/in_rank/8016207671941404930 in hdfs://10.0.0.1:9000/gfs_storage/hadoop-root/hbase/webdata/compaction.dir to 540402979/in_rank/8587356094792327071 in hdfs://10.0.0.1:9000/gfs_storage/hadoop-root/hbase/webdata for 540402979/in_rank 2008-01-22 01:23:22,605 DEBUG org.apache.hadoop.hbase.HStore: started compaction of 328 files using hdfs://10.0.0.1:9000/gfs_storage/hadoop-root/hbase/webdata/compaction.dir for 540402979/rank_visual 2008-01-22 01:23:31,248 DEBUG org.apache.hadoop.hbase.HStore: moving 540402979/rank_visual/4003180598508023113 in hdfs://10.0.0.1:9000/gfs_storage/hadoop-root/hbase/webdata/compaction.dir to 540402979/rank_visual/5001502002807562425 in hdfs://10.0.0.1:9000/gfs_storage/hadoop-root/hbase/webdata for 540402979/rank_visual 2008-01-22 01:23:34,438 INFO org.apache.hadoop.hbase.HRegion: compaction completed on region webdata,,1200970130265. Took 57mins, 10sec 2008-01-22 01:23:34,439 DEBUG org.apache.hadoop.hbase.HStore: compaction for HStore 540402979/size needed. 2008-01-22 01:23:34,439 DEBUG org.apache.hadoop.hbase.HRegion: 540402979/size needs compaction 2008-01-22 01:23:34,439 INFO org.apache.hadoop.hbase.HRegion: starting compaction on region webdata,,1200970130265 2008-01-22 01:23:34,439 DEBUG org.apache.hadoop.hbase.HStore: started compaction of 323 files using hdfs://10.0.0.1:9000/gfs_storage/hadoop-root/hbase/webdata/compaction.dir for 540402979/size {code} after finished compacting 432 files on the last column the compaction took 57min and 10 sec in that time there was 323 more flushes to the first column. the job stop writeing to the table at 01:23:34 > Flusher flushing back to back when it should not be > --------------------------------------------------- > > Key: HADOOP-2673 > URL: https://issues.apache.org/jira/browse/HADOOP-2673 > Project: Hadoop > Issue Type: Bug > Components: contrib/hbase > Environment: hadoop svn 613782 > Reporter: Billy Pearson > Fix For: 0.16.0 > > Attachments: log.log > > > settings: > hbase.regionserver.optionalcacheflushinterval = 300000 (5 mins) > hbase.hregion.memcache.flush.size = 67108864 (64MB) > I am attaching a log for this everything works fine until the first flush of the memcache. but once it flushes the first time correctly it flushes back to back many times even when it does not exceed one of the two limits above. > I have not found but one spot where we update the variable LastFlushTime except when the server starts so this could be the problem but I may be missing it in the code somewhere. If it is getting updated then we need to look if something could be blocking it from updating after a memcache flush. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
	[jira] Issue Comment Edited: (HADOOP-2673) Flusher flushing back to back when it should not be by JIRA jira@apache.org :: Rate this Message: Reply to Author \| View Threaded \| Show Only this Message [ https://issues.apache.org/jira/browse/HADOOP-2673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12561247#action_12561247 ] viper799 edited comment on HADOOP-2673 at 1/21/08 11:56 PM: ----------------------------------------------------------------- just adding some stats about this problem {code} 2008-01-22 01:07:31,944 DEBUG org.apache.hadoop.hbase.HStore: started compaction of 432 files using hdfs://10.0.0.1:9000/gfs_storage/hadoop-root/hbase/webdata/compaction.dir for 540402979/in_rank 2008-01-22 01:23:17,384 DEBUG org.apache.hadoop.hbase.HStore: moving 540402979/in_rank/8016207671941404930 in hdfs://10.0.0.1:9000/gfs_storage/hadoop-root/hbase/webdata/compaction.dir to 540402979/in_rank/8587356094792327071 in hdfs://10.0.0.1:9000/gfs_storage/hadoop-root/hbase/webdata for 540402979/in_rank 2008-01-22 01:23:22,605 DEBUG org.apache.hadoop.hbase.HStore: started compaction of 328 files using hdfs://10.0.0.1:9000/gfs_storage/hadoop-root/hbase/webdata/compaction.dir for 540402979/rank_visual 2008-01-22 01:23:31,248 DEBUG org.apache.hadoop.hbase.HStore: moving 540402979/rank_visual/4003180598508023113 in hdfs://10.0.0.1:9000/gfs_storage/hadoop-root/hbase/webdata/compaction.dir to 540402979/rank_visual/5001502002807562425 in hdfs://10.0.0.1:9000/gfs_storage/hadoop-root/hbase/webdata for 540402979/rank_visual 2008-01-22 01:23:34,438 INFO org.apache.hadoop.hbase.HRegion: compaction completed on region webdata,,1200970130265. Took 57mins, 10sec 2008-01-22 01:23:34,439 DEBUG org.apache.hadoop.hbase.HStore: compaction for HStore 540402979/size needed. 2008-01-22 01:23:34,439 DEBUG org.apache.hadoop.hbase.HRegion: 540402979/size needs compaction 2008-01-22 01:23:34,439 INFO org.apache.hadoop.hbase.HRegion: starting compaction on region webdata,,1200970130265 2008-01-22 01:23:34,439 DEBUG org.apache.hadoop.hbase.HStore: started compaction of 323 files using hdfs://10.0.0.1:9000/gfs_storage/hadoop-root/hbase/webdata/compaction.dir for 540402979/size {code} after finished compacting 432 files on the last column the compaction took 57min and 10 sec in that time there was 323 more flushes to the first column. the job stop writeing to the table at 01:04:35. so with my match that works out to be 8.5 flushes a min was (Author: viper799): just adding some stats about this problem {code} 2008-01-22 01:07:31,944 DEBUG org.apache.hadoop.hbase.HStore: started compaction of 432 files using hdfs://10.0.0.1:9000/gfs_storage/hadoop-root/hbase/webdata/compaction.dir for 540402979/in_rank 2008-01-22 01:23:17,384 DEBUG org.apache.hadoop.hbase.HStore: moving 540402979/in_rank/8016207671941404930 in hdfs://10.0.0.1:9000/gfs_storage/hadoop-root/hbase/webdata/compaction.dir to 540402979/in_rank/8587356094792327071 in hdfs://10.0.0.1:9000/gfs_storage/hadoop-root/hbase/webdata for 540402979/in_rank 2008-01-22 01:23:22,605 DEBUG org.apache.hadoop.hbase.HStore: started compaction of 328 files using hdfs://10.0.0.1:9000/gfs_storage/hadoop-root/hbase/webdata/compaction.dir for 540402979/rank_visual 2008-01-22 01:23:31,248 DEBUG org.apache.hadoop.hbase.HStore: moving 540402979/rank_visual/4003180598508023113 in hdfs://10.0.0.1:9000/gfs_storage/hadoop-root/hbase/webdata/compaction.dir to 540402979/rank_visual/5001502002807562425 in hdfs://10.0.0.1:9000/gfs_storage/hadoop-root/hbase/webdata for 540402979/rank_visual 2008-01-22 01:23:34,438 INFO org.apache.hadoop.hbase.HRegion: compaction completed on region webdata,,1200970130265. Took 57mins, 10sec 2008-01-22 01:23:34,439 DEBUG org.apache.hadoop.hbase.HStore: compaction for HStore 540402979/size needed. 2008-01-22 01:23:34,439 DEBUG org.apache.hadoop.hbase.HRegion: 540402979/size needs compaction 2008-01-22 01:23:34,439 INFO org.apache.hadoop.hbase.HRegion: starting compaction on region webdata,,1200970130265 2008-01-22 01:23:34,439 DEBUG org.apache.hadoop.hbase.HStore: started compaction of 323 files using hdfs://10.0.0.1:9000/gfs_storage/hadoop-root/hbase/webdata/compaction.dir for 540402979/size {code} after finished compacting 432 files on the last column the compaction took 57min and 10 sec in that time there was 323 more flushes to the first column. the job stop writeing to the table at 01:04:35 > Flusher flushing back to back when it should not be > --------------------------------------------------- > > Key: HADOOP-2673 > URL: https://issues.apache.org/jira/browse/HADOOP-2673 > Project: Hadoop > Issue Type: Bug > Components: contrib/hbase > Environment: hadoop svn 613782 > Reporter: Billy Pearson > Fix For: 0.16.0 > > Attachments: log.log > > > settings: > hbase.regionserver.optionalcacheflushinterval = 300000 (5 mins) > hbase.hregion.memcache.flush.size = 67108864 (64MB) > I am attaching a log for this everything works fine until the first flush of the memcache. but once it flushes the first time correctly it flushes back to back many times even when it does not exceed one of the two limits above. > I have not found but one spot where we update the variable LastFlushTime except when the server starts so this could be the problem but I may be missing it in the code somewhere. If it is getting updated then we need to look if something could be blocking it from updating after a memcache flush. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
	[jira] Assigned: (HADOOP-2673) Flusher flushing back to back when it should not be by JIRA jira@apache.org :: Rate this Message: Reply to Author \| View Threaded \| Show Only this Message [ https://issues.apache.org/jira/browse/HADOOP-2673?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jim Kellerman reassigned HADOOP-2673: ------------------------------------- Assignee: Jim Kellerman > Flusher flushing back to back when it should not be > --------------------------------------------------- > > Key: HADOOP-2673 > URL: https://issues.apache.org/jira/browse/HADOOP-2673 > Project: Hadoop Core > Issue Type: Bug > Components: contrib/hbase > Environment: hadoop svn 613782 > Reporter: Billy Pearson > Assignee: Jim Kellerman > Attachments: log.log > > > settings: > hbase.regionserver.optionalcacheflushinterval = 300000 (5 mins) > hbase.hregion.memcache.flush.size = 67108864 (64MB) > I am attaching a log for this everything works fine until the first flush of the memcache. but once it flushes the first time correctly it flushes back to back many times even when it does not exceed one of the two limits above. > I have not found but one spot where we update the variable LastFlushTime except when the server starts so this could be the problem but I may be missing it in the code somewhere. If it is getting updated then we need to look if something could be blocking it from updating after a memcache flush. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.