|

»

»

Hadoop lucene-dev

[jira] Created: (HADOOP-1195) NullPointerException in FSNamesystem due to getDatanode() return value is not checked

View: New views

10 Messages — Rating Filter: Alert me

	[jira] Created: (HADOOP-1195) NullPointerException in FSNamesystem due to getDatanode() return value is not checked by JIRA jira@apache.org :: Rate this Message: Reply to Author \| View Threaded \| Show Only this Message NullPointerException in FSNamesystem due to getDatanode() return value is not checked ------------------------------------------------------------------------------------- Key: HADOOP-1195 URL: https://issues.apache.org/jira/browse/HADOOP-1195 Project: Hadoop Issue Type: Bug Components: dfs Reporter: Konstantin Shvachko Fix For: 0.13.0 FSNamesystem.getDatanode( nodeID ) returns null if the node is not found. There are several places where we do not check wether the returned node is null, e.g. FSNamesystem.processReport() FSNamesystem.setDatanodeDead() FSNamesystem.invalidateBlock() FSNamesystem.verifyNodeRegistration() I'm getting the following NPE: 07/04/02 17:31:50 WARN dfs.DataNode: org.apache.hadoop.ipc.RemoteException: java.io.IOException: java.lang.NullPointerException at org.apache.hadoop.dfs.FSNamesystem.shouldNodeShutdown(FSNamesystem.java:3306) at org.apache.hadoop.dfs.FSNamesystem.processReport(FSNamesystem.java:2012) at org.apache.hadoop.dfs.NameNode.blockReport(NameNode.java:654) at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:585) at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:341) at org.apache.hadoop.ipc.Server$Handler.run(Server.java:580) at org.apache.hadoop.ipc.Client.call(Client.java:473) at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:164) at org.apache.hadoop.dfs.$Proxy1.blockReport(Unknown Source) at org.apache.hadoop.dfs.DataNode.offerService(DataNode.java:460) at org.apache.hadoop.dfs.DataNode.run(DataNode.java:1095) at org.apache.hadoop.dfs.MiniDFSCluster$DataNodeRunner.run(MiniDFSCluster.java:160) at java.lang.Thread.run(Thread.java:595) -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
	[jira] Assigned: (HADOOP-1195) NullPointerException in FSNamesystem due to getDatanode() return value is not checked by JIRA jira@apache.org :: Rate this Message: Reply to Author \| View Threaded \| Show Only this Message [ https://issues.apache.org/jira/browse/HADOOP-1195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] dhruba borthakur reassigned HADOOP-1195: ---------------------------------------- Assignee: dhruba borthakur > NullPointerException in FSNamesystem due to getDatanode() return value is not checked > ------------------------------------------------------------------------------------- > > Key: HADOOP-1195 > URL: https://issues.apache.org/jira/browse/HADOOP-1195 > Project: Hadoop > Issue Type: Bug > Components: dfs > Reporter: Konstantin Shvachko > Assignee: dhruba borthakur > > FSNamesystem.getDatanode( nodeID ) returns null if the node is not found. > There are several places where we do not check wether the returned node is null, e.g. > FSNamesystem.processReport() > FSNamesystem.setDatanodeDead() > FSNamesystem.invalidateBlock() > FSNamesystem.verifyNodeRegistration() > I'm getting the following NPE: > 07/04/02 17:31:50 WARN dfs.DataNode: org.apache.hadoop.ipc.RemoteException: java.io.IOException: java.lang.NullPointerException > at org.apache.hadoop.dfs.FSNamesystem.shouldNodeShutdown(FSNamesystem.java:3306) > at org.apache.hadoop.dfs.FSNamesystem.processReport(FSNamesystem.java:2012) > at org.apache.hadoop.dfs.NameNode.blockReport(NameNode.java:654) > at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source) > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) > at java.lang.reflect.Method.invoke(Method.java:585) > at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:341) > at org.apache.hadoop.ipc.Server$Handler.run(Server.java:580) > at org.apache.hadoop.ipc.Client.call(Client.java:473) > at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:164) > at org.apache.hadoop.dfs.$Proxy1.blockReport(Unknown Source) > at org.apache.hadoop.dfs.DataNode.offerService(DataNode.java:460) > at org.apache.hadoop.dfs.DataNode.run(DataNode.java:1095) > at org.apache.hadoop.dfs.MiniDFSCluster$DataNodeRunner.run(MiniDFSCluster.java:160) > at java.lang.Thread.run(Thread.java:595) -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
	[jira] Updated: (HADOOP-1195) NullPointerException in FSNamesystem due to getDatanode() return value is not checked by JIRA jira@apache.org :: Rate this Message: Reply to Author \| View Threaded \| Show Only this Message [ https://issues.apache.org/jira/browse/HADOOP-1195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] dhruba borthakur updated HADOOP-1195: ------------------------------------- Attachment: datanodeNullPointer.patch Check for null return value before using DatanodeDescriptor. > NullPointerException in FSNamesystem due to getDatanode() return value is not checked > ------------------------------------------------------------------------------------- > > Key: HADOOP-1195 > URL: https://issues.apache.org/jira/browse/HADOOP-1195 > Project: Hadoop > Issue Type: Bug > Components: dfs > Reporter: Konstantin Shvachko > Assignee: dhruba borthakur > Fix For: 0.16.0 > > Attachments: datanodeNullPointer.patch > > > FSNamesystem.getDatanode( nodeID ) returns null if the node is not found. > There are several places where we do not check wether the returned node is null, e.g. > FSNamesystem.processReport() > FSNamesystem.setDatanodeDead() > FSNamesystem.invalidateBlock() > FSNamesystem.verifyNodeRegistration() > I'm getting the following NPE: > 07/04/02 17:31:50 WARN dfs.DataNode: org.apache.hadoop.ipc.RemoteException: java.io.IOException: java.lang.NullPointerException > at org.apache.hadoop.dfs.FSNamesystem.shouldNodeShutdown(FSNamesystem.java:3306) > at org.apache.hadoop.dfs.FSNamesystem.processReport(FSNamesystem.java:2012) > at org.apache.hadoop.dfs.NameNode.blockReport(NameNode.java:654) > at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source) > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) > at java.lang.reflect.Method.invoke(Method.java:585) > at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:341) > at org.apache.hadoop.ipc.Server$Handler.run(Server.java:580) > at org.apache.hadoop.ipc.Client.call(Client.java:473) > at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:164) > at org.apache.hadoop.dfs.$Proxy1.blockReport(Unknown Source) > at org.apache.hadoop.dfs.DataNode.offerService(DataNode.java:460) > at org.apache.hadoop.dfs.DataNode.run(DataNode.java:1095) > at org.apache.hadoop.dfs.MiniDFSCluster$DataNodeRunner.run(MiniDFSCluster.java:160) > at java.lang.Thread.run(Thread.java:595) -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
	[jira] Updated: (HADOOP-1195) NullPointerException in FSNamesystem due to getDatanode() return value is not checked by JIRA jira@apache.org :: Rate this Message: Reply to Author \| View Threaded \| Show Only this Message [ https://issues.apache.org/jira/browse/HADOOP-1195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] dhruba borthakur updated HADOOP-1195: ------------------------------------- Fix Version/s: 0.16.0 Status: Patch Available (was: Open) > NullPointerException in FSNamesystem due to getDatanode() return value is not checked > ------------------------------------------------------------------------------------- > > Key: HADOOP-1195 > URL: https://issues.apache.org/jira/browse/HADOOP-1195 > Project: Hadoop > Issue Type: Bug > Components: dfs > Reporter: Konstantin Shvachko > Assignee: dhruba borthakur > Fix For: 0.16.0 > > Attachments: datanodeNullPointer.patch > > > FSNamesystem.getDatanode( nodeID ) returns null if the node is not found. > There are several places where we do not check wether the returned node is null, e.g. > FSNamesystem.processReport() > FSNamesystem.setDatanodeDead() > FSNamesystem.invalidateBlock() > FSNamesystem.verifyNodeRegistration() > I'm getting the following NPE: > 07/04/02 17:31:50 WARN dfs.DataNode: org.apache.hadoop.ipc.RemoteException: java.io.IOException: java.lang.NullPointerException > at org.apache.hadoop.dfs.FSNamesystem.shouldNodeShutdown(FSNamesystem.java:3306) > at org.apache.hadoop.dfs.FSNamesystem.processReport(FSNamesystem.java:2012) > at org.apache.hadoop.dfs.NameNode.blockReport(NameNode.java:654) > at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source) > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) > at java.lang.reflect.Method.invoke(Method.java:585) > at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:341) > at org.apache.hadoop.ipc.Server$Handler.run(Server.java:580) > at org.apache.hadoop.ipc.Client.call(Client.java:473) > at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:164) > at org.apache.hadoop.dfs.$Proxy1.blockReport(Unknown Source) > at org.apache.hadoop.dfs.DataNode.offerService(DataNode.java:460) > at org.apache.hadoop.dfs.DataNode.run(DataNode.java:1095) > at org.apache.hadoop.dfs.MiniDFSCluster$DataNodeRunner.run(MiniDFSCluster.java:160) > at java.lang.Thread.run(Thread.java:595) -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
	[jira] Commented: (HADOOP-1195) NullPointerException in FSNamesystem due to getDatanode() return value is not checked by JIRA jira@apache.org :: Rate this Message: Reply to Author \| View Threaded \| Show Only this Message [ https://issues.apache.org/jira/browse/HADOOP-1195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12560580#action_12560580 ] Hadoop QA commented on HADOOP-1195: ----------------------------------- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12373473/datanodeNullPointer.patch against trunk revision r613115. @author +1. The patch does not contain any @author tags. javadoc +1. The javadoc tool did not generate any warning messages. javac +1. The applied patch does not generate any new compiler warnings. findbugs +1. The patch does not introduce any new Findbugs warnings. core tests +1. The patch passed core unit tests. contrib tests -1. The patch failed contrib unit tests. Test results: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/1642/testReport/ Findbugs warnings: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/1642/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/1642/artifact/trunk/build/test/checkstyle-errors.html Console output: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/1642/console This message is automatically generated. > NullPointerException in FSNamesystem due to getDatanode() return value is not checked > ------------------------------------------------------------------------------------- > > Key: HADOOP-1195 > URL: https://issues.apache.org/jira/browse/HADOOP-1195 > Project: Hadoop > Issue Type: Bug > Components: dfs > Reporter: Konstantin Shvachko > Assignee: dhruba borthakur > Fix For: 0.16.0 > > Attachments: datanodeNullPointer.patch > > > FSNamesystem.getDatanode( nodeID ) returns null if the node is not found. > There are several places where we do not check wether the returned node is null, e.g. > FSNamesystem.processReport() > FSNamesystem.setDatanodeDead() > FSNamesystem.invalidateBlock() > FSNamesystem.verifyNodeRegistration() > I'm getting the following NPE: > 07/04/02 17:31:50 WARN dfs.DataNode: org.apache.hadoop.ipc.RemoteException: java.io.IOException: java.lang.NullPointerException > at org.apache.hadoop.dfs.FSNamesystem.shouldNodeShutdown(FSNamesystem.java:3306) > at org.apache.hadoop.dfs.FSNamesystem.processReport(FSNamesystem.java:2012) > at org.apache.hadoop.dfs.NameNode.blockReport(NameNode.java:654) > at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source) > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) > at java.lang.reflect.Method.invoke(Method.java:585) > at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:341) > at org.apache.hadoop.ipc.Server$Handler.run(Server.java:580) > at org.apache.hadoop.ipc.Client.call(Client.java:473) > at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:164) > at org.apache.hadoop.dfs.$Proxy1.blockReport(Unknown Source) > at org.apache.hadoop.dfs.DataNode.offerService(DataNode.java:460) > at org.apache.hadoop.dfs.DataNode.run(DataNode.java:1095) > at org.apache.hadoop.dfs.MiniDFSCluster$DataNodeRunner.run(MiniDFSCluster.java:160) > at java.lang.Thread.run(Thread.java:595) -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
	[jira] Commented: (HADOOP-1195) NullPointerException in FSNamesystem due to getDatanode() return value is not checked by JIRA jira@apache.org :: Rate this Message: Reply to Author \| View Threaded \| Show Only this Message [ https://issues.apache.org/jira/browse/HADOOP-1195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12561413#action_12561413 ] Raghu Angadi commented on HADOOP-1195: -------------------------------------- In all these cases is there valid reasons for expecting null? For. e.g. I am not sure if setDatanodeDead() should handle null.. its an internal member. > NullPointerException in FSNamesystem due to getDatanode() return value is not checked > ------------------------------------------------------------------------------------- > > Key: HADOOP-1195 > URL: https://issues.apache.org/jira/browse/HADOOP-1195 > Project: Hadoop > Issue Type: Bug > Components: dfs > Reporter: Konstantin Shvachko > Assignee: dhruba borthakur > Fix For: 0.16.0 > > Attachments: datanodeNullPointer.patch > > > FSNamesystem.getDatanode( nodeID ) returns null if the node is not found. > There are several places where we do not check wether the returned node is null, e.g. > FSNamesystem.processReport() > FSNamesystem.setDatanodeDead() > FSNamesystem.invalidateBlock() > FSNamesystem.verifyNodeRegistration() > I'm getting the following NPE: > 07/04/02 17:31:50 WARN dfs.DataNode: org.apache.hadoop.ipc.RemoteException: java.io.IOException: java.lang.NullPointerException > at org.apache.hadoop.dfs.FSNamesystem.shouldNodeShutdown(FSNamesystem.java:3306) > at org.apache.hadoop.dfs.FSNamesystem.processReport(FSNamesystem.java:2012) > at org.apache.hadoop.dfs.NameNode.blockReport(NameNode.java:654) > at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source) > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) > at java.lang.reflect.Method.invoke(Method.java:585) > at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:341) > at org.apache.hadoop.ipc.Server$Handler.run(Server.java:580) > at org.apache.hadoop.ipc.Client.call(Client.java:473) > at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:164) > at org.apache.hadoop.dfs.$Proxy1.blockReport(Unknown Source) > at org.apache.hadoop.dfs.DataNode.offerService(DataNode.java:460) > at org.apache.hadoop.dfs.DataNode.run(DataNode.java:1095) > at org.apache.hadoop.dfs.MiniDFSCluster$DataNodeRunner.run(MiniDFSCluster.java:160) > at java.lang.Thread.run(Thread.java:595) -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
	[jira] Commented: (HADOOP-1195) NullPointerException in FSNamesystem due to getDatanode() return value is not checked by JIRA jira@apache.org :: Rate this Message: Reply to Author \| View Threaded \| Show Only this Message [ https://issues.apache.org/jira/browse/HADOOP-1195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12561431#action_12561431 ] Raghu Angadi commented on HADOOP-1195: -------------------------------------- +1 for 1st and 3rd hunks in the patch. > NullPointerException in FSNamesystem due to getDatanode() return value is not checked > ------------------------------------------------------------------------------------- > > Key: HADOOP-1195 > URL: https://issues.apache.org/jira/browse/HADOOP-1195 > Project: Hadoop > Issue Type: Bug > Components: dfs > Reporter: Konstantin Shvachko > Assignee: dhruba borthakur > Fix For: 0.16.0 > > Attachments: datanodeNullPointer.patch > > > FSNamesystem.getDatanode( nodeID ) returns null if the node is not found. > There are several places where we do not check wether the returned node is null, e.g. > FSNamesystem.processReport() > FSNamesystem.setDatanodeDead() > FSNamesystem.invalidateBlock() > FSNamesystem.verifyNodeRegistration() > I'm getting the following NPE: > 07/04/02 17:31:50 WARN dfs.DataNode: org.apache.hadoop.ipc.RemoteException: java.io.IOException: java.lang.NullPointerException > at org.apache.hadoop.dfs.FSNamesystem.shouldNodeShutdown(FSNamesystem.java:3306) > at org.apache.hadoop.dfs.FSNamesystem.processReport(FSNamesystem.java:2012) > at org.apache.hadoop.dfs.NameNode.blockReport(NameNode.java:654) > at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source) > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) > at java.lang.reflect.Method.invoke(Method.java:585) > at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:341) > at org.apache.hadoop.ipc.Server$Handler.run(Server.java:580) > at org.apache.hadoop.ipc.Client.call(Client.java:473) > at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:164) > at org.apache.hadoop.dfs.$Proxy1.blockReport(Unknown Source) > at org.apache.hadoop.dfs.DataNode.offerService(DataNode.java:460) > at org.apache.hadoop.dfs.DataNode.run(DataNode.java:1095) > at org.apache.hadoop.dfs.MiniDFSCluster$DataNodeRunner.run(MiniDFSCluster.java:160) > at java.lang.Thread.run(Thread.java:595) -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
	[jira] Commented: (HADOOP-1195) NullPointerException in FSNamesystem due to getDatanode() return value is not checked by JIRA jira@apache.org :: Rate this Message: Reply to Author \| View Threaded \| Show Only this Message [ https://issues.apache.org/jira/browse/HADOOP-1195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12561438#action_12561438 ] dhruba borthakur commented on HADOOP-1195: ------------------------------------------ setDatanodeDead() sets the dead bit on the datanode if it finds the datanode. Otherwise it does nothing. The current code in the "trunk" will throw a NPE if this condition occurs. The premise of this bug fix is that a "Null Pointer Exception" is very bad. The namenode code should be able to throw explicit exceptions on error conditions. > NullPointerException in FSNamesystem due to getDatanode() return value is not checked > ------------------------------------------------------------------------------------- > > Key: HADOOP-1195 > URL: https://issues.apache.org/jira/browse/HADOOP-1195 > Project: Hadoop > Issue Type: Bug > Components: dfs > Reporter: Konstantin Shvachko > Assignee: dhruba borthakur > Fix For: 0.16.0 > > Attachments: datanodeNullPointer.patch > > > FSNamesystem.getDatanode( nodeID ) returns null if the node is not found. > There are several places where we do not check wether the returned node is null, e.g. > FSNamesystem.processReport() > FSNamesystem.setDatanodeDead() > FSNamesystem.invalidateBlock() > FSNamesystem.verifyNodeRegistration() > I'm getting the following NPE: > 07/04/02 17:31:50 WARN dfs.DataNode: org.apache.hadoop.ipc.RemoteException: java.io.IOException: java.lang.NullPointerException > at org.apache.hadoop.dfs.FSNamesystem.shouldNodeShutdown(FSNamesystem.java:3306) > at org.apache.hadoop.dfs.FSNamesystem.processReport(FSNamesystem.java:2012) > at org.apache.hadoop.dfs.NameNode.blockReport(NameNode.java:654) > at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source) > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) > at java.lang.reflect.Method.invoke(Method.java:585) > at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:341) > at org.apache.hadoop.ipc.Server$Handler.run(Server.java:580) > at org.apache.hadoop.ipc.Client.call(Client.java:473) > at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:164) > at org.apache.hadoop.dfs.$Proxy1.blockReport(Unknown Source) > at org.apache.hadoop.dfs.DataNode.offerService(DataNode.java:460) > at org.apache.hadoop.dfs.DataNode.run(DataNode.java:1095) > at org.apache.hadoop.dfs.MiniDFSCluster$DataNodeRunner.run(MiniDFSCluster.java:160) > at java.lang.Thread.run(Thread.java:595) -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
	[jira] Commented: (HADOOP-1195) NullPointerException in FSNamesystem due to getDatanode() return value is not checked by JIRA jira@apache.org :: Rate this Message: Reply to Author \| View Threaded \| Show Only this Message [ https://issues.apache.org/jira/browse/HADOOP-1195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12561446#action_12561446 ] Raghu Angadi commented on HADOOP-1195: -------------------------------------- > The premise of this bug fix is that a "Null Pointer Exception" is very bad. IMHO it is bad when null is expected (e.g. user arguments, RPC calls), no need to check for null when the context does not expect a null. I mean it does not need cover up some other bug. But I am not strongly against it. Your choice. how about replacing the 2nd hunk with: {noformat} @@ -1967,8 +1967,7 @@ (now() - heartbeatExpireInterval)); } - void setDatanodeDead(DatanodeID nodeID) throws IOException { - DatanodeDescriptor node = getDatanode(nodeID); + private void setDatanodeDead(DatanodeDescriptor node) throws IOException { node.setLastUpdate(0); } {noformat} > NullPointerException in FSNamesystem due to getDatanode() return value is not checked > ------------------------------------------------------------------------------------- > > Key: HADOOP-1195 > URL: https://issues.apache.org/jira/browse/HADOOP-1195 > Project: Hadoop > Issue Type: Bug > Components: dfs > Reporter: Konstantin Shvachko > Assignee: dhruba borthakur > Fix For: 0.16.0 > > Attachments: datanodeNullPointer.patch > > > FSNamesystem.getDatanode( nodeID ) returns null if the node is not found. > There are several places where we do not check wether the returned node is null, e.g. > FSNamesystem.processReport() > FSNamesystem.setDatanodeDead() > FSNamesystem.invalidateBlock() > FSNamesystem.verifyNodeRegistration() > I'm getting the following NPE: > 07/04/02 17:31:50 WARN dfs.DataNode: org.apache.hadoop.ipc.RemoteException: java.io.IOException: java.lang.NullPointerException > at org.apache.hadoop.dfs.FSNamesystem.shouldNodeShutdown(FSNamesystem.java:3306) > at org.apache.hadoop.dfs.FSNamesystem.processReport(FSNamesystem.java:2012) > at org.apache.hadoop.dfs.NameNode.blockReport(NameNode.java:654) > at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source) > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) > at java.lang.reflect.Method.invoke(Method.java:585) > at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:341) > at org.apache.hadoop.ipc.Server$Handler.run(Server.java:580) > at org.apache.hadoop.ipc.Client.call(Client.java:473) > at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:164) > at org.apache.hadoop.dfs.$Proxy1.blockReport(Unknown Source) > at org.apache.hadoop.dfs.DataNode.offerService(DataNode.java:460) > at org.apache.hadoop.dfs.DataNode.run(DataNode.java:1095) > at org.apache.hadoop.dfs.MiniDFSCluster$DataNodeRunner.run(MiniDFSCluster.java:160) > at java.lang.Thread.run(Thread.java:595) -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
	[jira] Updated: (HADOOP-1195) NullPointerException in FSNamesystem due to getDatanode() return value is not checked by JIRA jira@apache.org :: Rate this Message: Reply to Author \| View Threaded \| Show Only this Message [ https://issues.apache.org/jira/browse/HADOOP-1195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] dhruba borthakur updated HADOOP-1195: ------------------------------------- Attachment: datanodeNullPointer.patch Incorporated Raghu's comments. > NullPointerException in FSNamesystem due to getDatanode() return value is not checked > ------------------------------------------------------------------------------------- > > Key: HADOOP-1195 > URL: https://issues.apache.org/jira/browse/HADOOP-1195 > Project: Hadoop > Issue Type: Bug > Components: dfs > Reporter: Konstantin Shvachko > Assignee: dhruba borthakur > Fix For: 0.16.0 > > Attachments: datanodeNullPointer.patch, datanodeNullPointer.patch > > > FSNamesystem.getDatanode( nodeID ) returns null if the node is not found. > There are several places where we do not check wether the returned node is null, e.g. > FSNamesystem.processReport() > FSNamesystem.setDatanodeDead() > FSNamesystem.invalidateBlock() > FSNamesystem.verifyNodeRegistration() > I'm getting the following NPE: > 07/04/02 17:31:50 WARN dfs.DataNode: org.apache.hadoop.ipc.RemoteException: java.io.IOException: java.lang.NullPointerException > at org.apache.hadoop.dfs.FSNamesystem.shouldNodeShutdown(FSNamesystem.java:3306) > at org.apache.hadoop.dfs.FSNamesystem.processReport(FSNamesystem.java:2012) > at org.apache.hadoop.dfs.NameNode.blockReport(NameNode.java:654) > at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source) > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) > at java.lang.reflect.Method.invoke(Method.java:585) > at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:341) > at org.apache.hadoop.ipc.Server$Handler.run(Server.java:580) > at org.apache.hadoop.ipc.Client.call(Client.java:473) > at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:164) > at org.apache.hadoop.dfs.$Proxy1.blockReport(Unknown Source) > at org.apache.hadoop.dfs.DataNode.offerService(DataNode.java:460) > at org.apache.hadoop.dfs.DataNode.run(DataNode.java:1095) > at org.apache.hadoop.dfs.MiniDFSCluster$DataNodeRunner.run(MiniDFSCluster.java:160) > at java.lang.Thread.run(Thread.java:595) -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.