|

»

»

»

Hadoop lucene-dev

[jira] Created: (HADOOP-2688) [hbase] IllegalArgumentException processing a shutdown

View: New views

5 Messages — Rating Filter: Alert me

	[jira] Created: (HADOOP-2688) [hbase] IllegalArgumentException processing a shutdown by JIRA jira@apache.org :: Rate this Message: Reply to Author \| View Threaded \| Show Only this Message [hbase] IllegalArgumentException processing a shutdown ------------------------------------------------------ Key: HADOOP-2688 URL: https://issues.apache.org/jira/browse/HADOOP-2688 Project: Hadoop Issue Type: Bug Components: contrib/hbase Reporter: stack Priority: Minor {code} 2008-01-23 02:38:54,537 WARN org.apache.hadoop.hbase.HMaster: Processing pending operations: ProcessServerShutdown of XX.XX.XX.128:60020 java.lang.IllegalArgumentException: Can't build a writable with empty bytes array at org.apache.hadoop.hbase.util.Writables.getWritable(Writables.java:81) at org.apache.hadoop.hbase.HMaster$ProcessServerShutdown.scanMetaRegion(HMaster.java:2045) at org.apache.hadoop.hbase.HMaster$ProcessServerShutdown.process(HMaster.java:2230) at org.apache.hadoop.hbase.HMaster.run(HMaster.java:1059) {code} -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
	[jira] Updated: (HADOOP-2688) [hbase] IllegalArgumentException processing a shutdown stops server going down and results in millions of lines of output by JIRA jira@apache.org :: Rate this Message: Reply to Author \| View Threaded \| Show Only this Message [ https://issues.apache.org/jira/browse/HADOOP-2688?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HADOOP-2688: -------------------------- Priority: Blocker (was: Minor) Fix Version/s: 0.16.0 Summary: [hbase] IllegalArgumentException processing a shutdown stops server going down and results in millions of lines of output (was: [hbase] IllegalArgumentException processing a shutdown) This exception makes it so master never goes down and runs for ever emitting millions of lines of logs. Chatting w/ Jim, this is a blocker. Here is more context around the exception given above: {code} 2008-01-23 00:00:00,058 DEBUG org.apache.hadoop.hbase.HMaster: shutdown scanner looking at enwiki_test10k_meta,by4qSvb84c-zYyH55XoDok==,1197597636063 2008-01-23 00:00:00,058 DEBUG org.apache.hadoop.hbase.HMaster: Server name XX.XX.XX.119:60020 is not same as XX.XX.XX.128:60020: Passing 2008-01-23 00:00:00,058 DEBUG org.apache.hadoop.hbase.HMaster: shutdown scanner looking at enwiki_test10k_meta,jxq8LhQ49PGgSe5NYxxWYF==,1197597545783 2008-01-23 00:00:00,059 DEBUG org.apache.hadoop.hbase.HMaster: Server name XX.XX.XX.176:60020 is not same as XX.XX.XX.128:60020: Passing 2008-01-23 00:00:00,059 DEBUG org.apache.hadoop.hbase.HMaster: shutdown scanner looking at enwiki_test10k_meta,s-2mdgCPAfon8meNI4WdL-==,1197597545783 2008-01-23 00:00:00,059 DEBUG org.apache.hadoop.hbase.HMaster: Server name XX.XX.XX.223:60020 is not same as XX.XX.XX.128:60020: Passing 2008-01-23 00:00:00,059 DEBUG org.apache.hadoop.hbase.HMaster: shutdown scanner looking at enwiki_uploader_test_meta,,1197591146685 2008-01-23 00:00:00,059 DEBUG org.apache.hadoop.hbase.HMaster: Server name XX.XX.XX.95:60020 is not same as XX.XX.XX.128:60020: Passing 2008-01-23 00:00:00,060 DEBUG org.apache.hadoop.hbase.HMaster: shutdown scanner looking at mediawiki_test_article,enwiki_080103.-OEwmocs7e86-ixv9hx9WV==,1200357394687 2008-01-23 00:00:00,061 WARN org.apache.hadoop.hbase.HMaster: Processing pending operations: ProcessServerShutdown of XX.XX.XX.128:60020 java.lang.IllegalArgumentException: Can't build a writable with empty bytes array at org.apache.hadoop.hbase.util.Writables.getWritable(Writables.java:81) at org.apache.hadoop.hbase.HMaster$ProcessServerShutdown.scanMetaRegion(HMaster.java:2045) at org.apache.hadoop.hbase.HMaster$ProcessServerShutdown.process(HMaster.java:2230) at org.apache.hadoop.hbase.HMaster.run(HMaster.java:1059) 2008-01-23 00:00:00,061 DEBUG org.apache.hadoop.hbase.HMaster: Main processing loop: ProcessServerShutdown of XX.XX.XX.128:60020 2008-01-23 00:00:00,061 INFO org.apache.hadoop.hbase.HMaster: process shutdown of server 208.76.45.128:60020: logSplit: false, rootRescanned: false, numberOfMetaRegions: 1, onlineMetaRegions.size(): 1 2008-01-23 00:00:00,062 DEBUG org.apache.hadoop.hbase.HMaster: process server shutdown scanning root region onXX.XX.XX121 2008-01-23 00:00:00,075 DEBUG org.apache.hadoop.hbase.HMaster: shutdown scanner looking at .META.,,1 2008-01-23 00:00:00,075 DEBUG org.apache.hadoop.hbase.HMaster: Server name XX.XX.XX.97:60020 is not same as XX.XX.XX.128:60020: Passing 2008-01-23 00:00:00,076 DEBUG org.apache.hadoop.hbase.HMaster: process server shutdown scanning root region on XX.XX.XX.121 finished HMaster 2008-01-23 00:00:00,076 DEBUG org.apache.hadoop.hbase.HMaster: numberOfMetaRegions: 1, onlineMetaRegions.size(): 1 2008-01-23 00:00:00,076 DEBUG org.apache.hadoop.hbase.HMaster: process server shutdown scanning .META.,,1 on XX.XX.XX.97:60020 HMaster 2008-01-23 00:00:00,094 DEBUG org.apache.hadoop.hbase.HMaster: shutdown scanner looking at coral_mappers_1_0_test_article,,1200598809448 {code} > [hbase] IllegalArgumentException processing a shutdown stops server going down and results in millions of lines of output > ------------------------------------------------------------------------------------------------------------------------- > > Key: HADOOP-2688 > URL: https://issues.apache.org/jira/browse/HADOOP-2688 > Project: Hadoop > Issue Type: Bug > Components: contrib/hbase > Reporter: stack > Priority: Blocker > Fix For: 0.16.0 > > > {code} > 2008-01-23 02:38:54,537 WARN org.apache.hadoop.hbase.HMaster: Processing pending operations: ProcessServerShutdown of XX.XX.XX.128:60020 > java.lang.IllegalArgumentException: Can't build a writable with empty bytes array > at org.apache.hadoop.hbase.util.Writables.getWritable(Writables.java:81) > at org.apache.hadoop.hbase.HMaster$ProcessServerShutdown.scanMetaRegion(HMaster.java:2045) > at org.apache.hadoop.hbase.HMaster$ProcessServerShutdown.process(HMaster.java:2230) > at org.apache.hadoop.hbase.HMaster.run(HMaster.java:1059) > {code} -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
	[jira] Updated: (HADOOP-2688) [hbase] IllegalArgumentException processing a shutdown stops server going down and results in millions of lines of output by JIRA jira@apache.org :: Rate this Message: Reply to Author \| View Threaded \| Show Only this Message [ https://issues.apache.org/jira/browse/HADOOP-2688?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HADOOP-2688: -------------------------- Attachment: emptywritable.patch Jim, please review this patch. In the name of minimal changes this close to release, I did not change HMaster.run making it stay up whatever the exception. M src/contrib/hbase/src/java/org/apache/hadoop/hbase/HStoreFile.java (getReader): Remove synchronize M src/contrib/hbase/src/java/org/apache/hadoop/hbase/HMaster.java Refactor all meta scans to use two new utility methods that add checks for empty cells - the cause of this issue. (getHRegionInfo, getRowMap): Added utility methods for converting HbaseMapWritable to map keyed by column and a get of COL_REGIONINFO that does check for empty cell. > [hbase] IllegalArgumentException processing a shutdown stops server going down and results in millions of lines of output > ------------------------------------------------------------------------------------------------------------------------- > > Key: HADOOP-2688 > URL: https://issues.apache.org/jira/browse/HADOOP-2688 > Project: Hadoop Core > Issue Type: Bug > Components: contrib/hbase > Reporter: stack > Priority: Blocker > Fix For: 0.16.0 > > Attachments: emptywritable.patch > > > {code} > 2008-01-23 02:38:54,537 WARN org.apache.hadoop.hbase.HMaster: Processing pending operations: ProcessServerShutdown of XX.XX.XX.128:60020 > java.lang.IllegalArgumentException: Can't build a writable with empty bytes array > at org.apache.hadoop.hbase.util.Writables.getWritable(Writables.java:81) > at org.apache.hadoop.hbase.HMaster$ProcessServerShutdown.scanMetaRegion(HMaster.java:2045) > at org.apache.hadoop.hbase.HMaster$ProcessServerShutdown.process(HMaster.java:2230) > at org.apache.hadoop.hbase.HMaster.run(HMaster.java:1059) > {code} -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
	[jira] Updated: (HADOOP-2688) [hbase] IllegalArgumentException processing a shutdown stops server going down and results in millions of lines of output by JIRA jira@apache.org :: Rate this Message: Reply to Author \| View Threaded \| Show Only this Message [ https://issues.apache.org/jira/browse/HADOOP-2688?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HADOOP-2688: -------------------------- Attachment: emptywritable-v2.patch v2 addresses jim comment (Made RowMap private) > [hbase] IllegalArgumentException processing a shutdown stops server going down and results in millions of lines of output > ------------------------------------------------------------------------------------------------------------------------- > > Key: HADOOP-2688 > URL: https://issues.apache.org/jira/browse/HADOOP-2688 > Project: Hadoop Core > Issue Type: Bug > Components: contrib/hbase > Reporter: stack > Priority: Blocker > Fix For: 0.16.0 > > Attachments: emptywritable-v2.patch, emptywritable.patch > > > {code} > 2008-01-23 02:38:54,537 WARN org.apache.hadoop.hbase.HMaster: Processing pending operations: ProcessServerShutdown of XX.XX.XX.128:60020 > java.lang.IllegalArgumentException: Can't build a writable with empty bytes array > at org.apache.hadoop.hbase.util.Writables.getWritable(Writables.java:81) > at org.apache.hadoop.hbase.HMaster$ProcessServerShutdown.scanMetaRegion(HMaster.java:2045) > at org.apache.hadoop.hbase.HMaster$ProcessServerShutdown.process(HMaster.java:2230) > at org.apache.hadoop.hbase.HMaster.run(HMaster.java:1059) > {code} -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
	[jira] Updated: (HADOOP-2688) [hbase] IllegalArgumentException processing a shutdown stops server going down and results in millions of lines of output by JIRA jira@apache.org :: Rate this Message: Reply to Author \| View Threaded \| Show Only this Message [ https://issues.apache.org/jira/browse/HADOOP-2688?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HADOOP-2688: -------------------------- Status: Patch Available (was: Open) Passes tests locally. Passing to hudson. > [hbase] IllegalArgumentException processing a shutdown stops server going down and results in millions of lines of output > ------------------------------------------------------------------------------------------------------------------------- > > Key: HADOOP-2688 > URL: https://issues.apache.org/jira/browse/HADOOP-2688 > Project: Hadoop Core > Issue Type: Bug > Components: contrib/hbase > Reporter: stack > Priority: Blocker > Fix For: 0.16.0 > > Attachments: emptywritable-v2.patch, emptywritable.patch > > > {code} > 2008-01-23 02:38:54,537 WARN org.apache.hadoop.hbase.HMaster: Processing pending operations: ProcessServerShutdown of XX.XX.XX.128:60020 > java.lang.IllegalArgumentException: Can't build a writable with empty bytes array > at org.apache.hadoop.hbase.util.Writables.getWritable(Writables.java:81) > at org.apache.hadoop.hbase.HMaster$ProcessServerShutdown.scanMetaRegion(HMaster.java:2045) > at org.apache.hadoop.hbase.HMaster$ProcessServerShutdown.process(HMaster.java:2230) > at org.apache.hadoop.hbase.HMaster.run(HMaster.java:1059) > {code} -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.