sequence file does not detect corruption in key/value lengths
-------------------------------------------------------------
Key: HADOOP-2497
URL:
https://issues.apache.org/jira/browse/HADOOP-2497 Project: Hadoop
Issue Type: Bug
Reporter: eric baldeschwieler
From:
jason@...
Subject: Re: question on Hadoop configuration for non cpu intensive jobs - 0.15.1
Date: December 28, 2007 7:20:11 AM PST
To:
hadoop-user@...
Our OOM was being caused by a damaged sequence data file. We had assumed that the sequence files had checksums, which appears to be in correct.
The deserializer was reading a bad length out of the file and trying to allocate 4gig of ram.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
