|

»

»

Hadoop lucene-dev

[jira] Created: (HADOOP-2664) lzop-compatible CompresionCodec

View: New views

9 Messages — Rating Filter: Alert me

	[jira] Created: (HADOOP-2664) lzop-compatible CompresionCodec by JIRA jira@apache.org :: Rate this Message: Reply to Author \| View Threaded \| Show Only this Message lzop-compatible CompresionCodec ------------------------------- Key: HADOOP-2664 URL: https://issues.apache.org/jira/browse/HADOOP-2664 Project: Hadoop Issue Type: New Feature Components: io Reporter: Chris Douglas Assignee: Chris Douglas Fix For: 0.16.0 The current lzo codec is not compatible with the standard .lzo file format used by lzop. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
	[jira] Updated: (HADOOP-2664) lzop-compatible CompresionCodec by JIRA jira@apache.org :: Rate this Message: Reply to Author \| View Threaded \| Show Only this Message [ https://issues.apache.org/jira/browse/HADOOP-2664?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Douglas updated HADOOP-2664: ---------------------------------- Attachment: 2664-0.patch This patch adds lzop compatibility as an optional codec. On writes, it adds a generic header to .lzo files; on reads, it respects and confirms any block-checksum data specified in the header. It cannot be used with SequenceFiles. > lzop-compatible CompresionCodec > ------------------------------- > > Key: HADOOP-2664 > URL: https://issues.apache.org/jira/browse/HADOOP-2664 > Project: Hadoop > Issue Type: New Feature > Components: io > Reporter: Chris Douglas > Assignee: Chris Douglas > Fix For: 0.16.0 > > Attachments: 2664-0.patch > > > The current lzo codec is not compatible with the standard .lzo file format used by lzop. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
	[jira] Updated: (HADOOP-2664) lzop-compatible CompresionCodec by JIRA jira@apache.org :: Rate this Message: Reply to Author \| View Threaded \| Show Only this Message [ https://issues.apache.org/jira/browse/HADOOP-2664?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Douglas updated HADOOP-2664: ---------------------------------- Status: Patch Available (was: Open) > lzop-compatible CompresionCodec > ------------------------------- > > Key: HADOOP-2664 > URL: https://issues.apache.org/jira/browse/HADOOP-2664 > Project: Hadoop > Issue Type: New Feature > Components: io > Reporter: Chris Douglas > Assignee: Chris Douglas > Fix For: 0.16.0 > > Attachments: 2664-0.patch > > > The current lzo codec is not compatible with the standard .lzo file format used by lzop. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
	[jira] Commented: (HADOOP-2664) lzop-compatible CompresionCodec by JIRA jira@apache.org :: Rate this Message: Reply to Author \| View Threaded \| Show Only this Message [ https://issues.apache.org/jira/browse/HADOOP-2664?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12560842#action_12560842 ] Hadoop QA commented on HADOOP-2664: ----------------------------------- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12373587/2664-0.patch against trunk revision r613499. @author +1. The patch does not contain any @author tags. javadoc +1. The javadoc tool did not generate any warning messages. javac +1. The applied patch does not generate any new compiler warnings. findbugs -1. The patch appears to introduce 4 new Findbugs warnings. core tests +1. The patch passed core unit tests. contrib tests +1. The patch passed contrib unit tests. Test results: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/1662/testReport/ Findbugs warnings: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/1662/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/1662/artifact/trunk/build/test/checkstyle-errors.html Console output: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/1662/console This message is automatically generated. > lzop-compatible CompresionCodec > ------------------------------- > > Key: HADOOP-2664 > URL: https://issues.apache.org/jira/browse/HADOOP-2664 > Project: Hadoop > Issue Type: New Feature > Components: io > Reporter: Chris Douglas > Assignee: Chris Douglas > Fix For: 0.16.0 > > Attachments: 2664-0.patch > > > The current lzo codec is not compatible with the standard .lzo file format used by lzop. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
	[jira] Updated: (HADOOP-2664) lzop-compatible CompresionCodec by JIRA jira@apache.org :: Rate this Message: Reply to Author \| View Threaded \| Show Only this Message [ https://issues.apache.org/jira/browse/HADOOP-2664?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Douglas updated HADOOP-2664: ---------------------------------- Fix Version/s: (was: 0.16.0) 0.17.0 > lzop-compatible CompresionCodec > ------------------------------- > > Key: HADOOP-2664 > URL: https://issues.apache.org/jira/browse/HADOOP-2664 > Project: Hadoop > Issue Type: New Feature > Components: io > Reporter: Chris Douglas > Assignee: Chris Douglas > Fix For: 0.17.0 > > Attachments: 2664-0.patch, 2664-1.patch > > > The current lzo codec is not compatible with the standard .lzo file format used by lzop. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
	[jira] Updated: (HADOOP-2664) lzop-compatible CompresionCodec by JIRA jira@apache.org :: Rate this Message: Reply to Author \| View Threaded \| Show Only this Message [ https://issues.apache.org/jira/browse/HADOOP-2664?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Douglas updated HADOOP-2664: ---------------------------------- Attachment: 2664-1.patch Fixed findbugs warnings, bumped buffer to 256k (the size used by lzop) for the decompressor, changed the decompressor to the "safe" code to avoid crashing the JVM when it's too small, and added some documentation. I have some reservations about this patch (memory usage, thread safety if pooled, etc), so I'm pushing it to 0.17. > lzop-compatible CompresionCodec > ------------------------------- > > Key: HADOOP-2664 > URL: https://issues.apache.org/jira/browse/HADOOP-2664 > Project: Hadoop > Issue Type: New Feature > Components: io > Reporter: Chris Douglas > Assignee: Chris Douglas > Fix For: 0.17.0 > > Attachments: 2664-0.patch, 2664-1.patch > > > The current lzo codec is not compatible with the standard .lzo file format used by lzop. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
	[jira] Updated: (HADOOP-2664) lzop-compatible CompresionCodec by JIRA jira@apache.org :: Rate this Message: Reply to Author \| View Threaded \| Show Only this Message [ https://issues.apache.org/jira/browse/HADOOP-2664?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Douglas updated HADOOP-2664: ---------------------------------- Status: Open (was: Patch Available) > lzop-compatible CompresionCodec > ------------------------------- > > Key: HADOOP-2664 > URL: https://issues.apache.org/jira/browse/HADOOP-2664 > Project: Hadoop > Issue Type: New Feature > Components: io > Reporter: Chris Douglas > Assignee: Chris Douglas > Fix For: 0.17.0 > > Attachments: 2664-0.patch, 2664-1.patch > > > The current lzo codec is not compatible with the standard .lzo file format used by lzop. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
	[jira] Updated: (HADOOP-2664) lzop-compatible CompresionCodec by JIRA jira@apache.org :: Rate this Message: Reply to Author \| View Threaded \| Show Only this Message [ https://issues.apache.org/jira/browse/HADOOP-2664?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Douglas updated HADOOP-2664: ---------------------------------- Status: Patch Available (was: Open) > lzop-compatible CompresionCodec > ------------------------------- > > Key: HADOOP-2664 > URL: https://issues.apache.org/jira/browse/HADOOP-2664 > Project: Hadoop > Issue Type: New Feature > Components: io > Reporter: Chris Douglas > Assignee: Chris Douglas > Fix For: 0.17.0 > > Attachments: 2664-0.patch, 2664-1.patch > > > The current lzo codec is not compatible with the standard .lzo file format used by lzop. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
	[jira] Commented: (HADOOP-2664) lzop-compatible CompresionCodec by JIRA jira@apache.org :: Rate this Message: Reply to Author \| View Threaded \| Show Only this Message [ https://issues.apache.org/jira/browse/HADOOP-2664?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12561597#action_12561597 ] Hadoop QA commented on HADOOP-2664: ----------------------------------- +1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12373706/2664-1.patch against trunk revision r614413. @author +1. The patch does not contain any @author tags. javadoc +1. The javadoc tool did not generate any warning messages. javac +1. The applied patch does not generate any new compiler warnings. findbugs +1. The patch does not introduce any new Findbugs warnings. core tests +1. The patch passed core unit tests. contrib tests +1. The patch passed contrib unit tests. Test results: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/1680/testReport/ Findbugs warnings: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/1680/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/1680/artifact/trunk/build/test/checkstyle-errors.html Console output: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/1680/console This message is automatically generated. > lzop-compatible CompresionCodec > ------------------------------- > > Key: HADOOP-2664 > URL: https://issues.apache.org/jira/browse/HADOOP-2664 > Project: Hadoop > Issue Type: New Feature > Components: io > Reporter: Chris Douglas > Assignee: Chris Douglas > Fix For: 0.17.0 > > Attachments: 2664-0.patch, 2664-1.patch > > > The current lzo codec is not compatible with the standard .lzo file format used by lzop. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.