[jira] Created: (HADOOP-2596) add SequenceFile.createWriter() method that takes block size as parameter

add SequenceFile.createWriter() method that takes block size as parameter
-------------------------------------------------------------------------

                 Key: HADOOP-2596
                 URL: https://issues.apache.org/jira/browse/HADOOP-2596
             Project: Hadoop
          Issue Type: Improvement
          Components: io
         Environment: all
            Reporter: Alejandro Abdelnur
            Priority: Minor


Currently it is not possible to create a SequenceFile.Writer using a block size other than the default.

The createWriter() method should be overloaded with a signature receiving block size as parameter should be added to the the SequenceFile class.

With all the current signatures for this method there is a significant code duplication, if possible the createWriter() methods  should be refactored to avoid such duplication.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

     [ https://issues.apache.org/jira/browse/HADOOP-2596?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Alejandro Abdelnur reassigned HADOOP-2596:
------------------------------------------

    Assignee: Alejandro Abdelnur

> add SequenceFile.createWriter() method that takes block size as parameter
> -------------------------------------------------------------------------
>
>                 Key: HADOOP-2596
>                 URL: https://issues.apache.org/jira/browse/HADOOP-2596
>             Project: Hadoop
>          Issue Type: Improvement
>          Components: io
>         Environment: all
>            Reporter: Alejandro Abdelnur
>            Assignee: Alejandro Abdelnur
>            Priority: Minor
>             Fix For: 0.16.0
>
>         Attachments: patch2596.txt
>
>
> Currently it is not possible to create a SequenceFile.Writer using a block size other than the default.
> The createWriter() method should be overloaded with a signature receiving block size as parameter should be added to the the SequenceFile class.
> With all the current signatures for this method there is a significant code duplication, if possible the createWriter() methods  should be refactored to avoid such duplication.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

     [ https://issues.apache.org/jira/browse/HADOOP-2596?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Alejandro Abdelnur updated HADOOP-2596:
---------------------------------------

    Attachment: patch2596.txt

Added a most generic createWriter() method that takes buffersize, replication and blockSize. Refactored all other methods removing duplicated logic by calling more generic signature.

Added constructors to the SequenceFile writer implementations to support the new parameters above. Refactored writer constructors to remove duplicate logic by calling more generic signature.

> add SequenceFile.createWriter() method that takes block size as parameter
> -------------------------------------------------------------------------
>
>                 Key: HADOOP-2596
>                 URL: https://issues.apache.org/jira/browse/HADOOP-2596
>             Project: Hadoop
>          Issue Type: Improvement
>          Components: io
>         Environment: all
>            Reporter: Alejandro Abdelnur
>            Priority: Minor
>             Fix For: 0.16.0
>
>         Attachments: patch2596.txt
>
>
> Currently it is not possible to create a SequenceFile.Writer using a block size other than the default.
> The createWriter() method should be overloaded with a signature receiving block size as parameter should be added to the the SequenceFile class.
> With all the current signatures for this method there is a significant code duplication, if possible the createWriter() methods  should be refactored to avoid such duplication.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

     [ https://issues.apache.org/jira/browse/HADOOP-2596?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Alejandro Abdelnur updated HADOOP-2596:
---------------------------------------

    Fix Version/s: 0.16.0
           Status: Patch Available  (was: Open)

> add SequenceFile.createWriter() method that takes block size as parameter
> -------------------------------------------------------------------------
>
>                 Key: HADOOP-2596
>                 URL: https://issues.apache.org/jira/browse/HADOOP-2596
>             Project: Hadoop
>          Issue Type: Improvement
>          Components: io
>         Environment: all
>            Reporter: Alejandro Abdelnur
>            Assignee: Alejandro Abdelnur
>            Priority: Minor
>             Fix For: 0.16.0
>
>         Attachments: patch2596.txt
>
>
> Currently it is not possible to create a SequenceFile.Writer using a block size other than the default.
> The createWriter() method should be overloaded with a signature receiving block size as parameter should be added to the the SequenceFile class.
> With all the current signatures for this method there is a significant code duplication, if possible the createWriter() methods  should be refactored to avoid such duplication.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

    [ https://issues.apache.org/jira/browse/HADOOP-2596?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12560531#action_12560531 ]

Hadoop QA commented on HADOOP-2596:
-----------------------------------

-1 overall.  Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12373467/patch2596.txt
against trunk revision r613115.

    @author +1.  The patch does not contain any @author tags.

    javadoc +1.  The javadoc tool did not generate any warning messages.

    javac +1.  The applied patch does not generate any new compiler warnings.

    findbugs +1.  The patch does not introduce any new Findbugs warnings.

    core tests -1.  The patch failed core unit tests.

    contrib tests +1.  The patch passed contrib unit tests.

Test results: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/1640/testReport/
Findbugs warnings: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/1640/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Checkstyle results: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/1640/artifact/trunk/build/test/checkstyle-errors.html
Console output: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/1640/console

This message is automatically generated.

> add SequenceFile.createWriter() method that takes block size as parameter
> -------------------------------------------------------------------------
>
>                 Key: HADOOP-2596
>                 URL: https://issues.apache.org/jira/browse/HADOOP-2596
>             Project: Hadoop
>          Issue Type: Improvement
>          Components: io
>         Environment: all
>            Reporter: Alejandro Abdelnur
>            Assignee: Alejandro Abdelnur
>            Priority: Minor
>             Fix For: 0.16.0
>
>         Attachments: patch2596.txt
>
>
> Currently it is not possible to create a SequenceFile.Writer using a block size other than the default.
> The createWriter() method should be overloaded with a signature receiving block size as parameter should be added to the the SequenceFile class.
> With all the current signatures for this method there is a significant code duplication, if possible the createWriter() methods  should be refactored to avoid such duplication.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

     [ https://issues.apache.org/jira/browse/HADOOP-2596?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Arun C Murthy updated HADOOP-2596:
----------------------------------

    Status: Open  (was: Patch Available)

Re-submitting to hudson since an unrelated test failed...

> add SequenceFile.createWriter() method that takes block size as parameter
> -------------------------------------------------------------------------
>
>                 Key: HADOOP-2596
>                 URL: https://issues.apache.org/jira/browse/HADOOP-2596
>             Project: Hadoop
>          Issue Type: Improvement
>          Components: io
>         Environment: all
>            Reporter: Alejandro Abdelnur
>            Assignee: Alejandro Abdelnur
>            Priority: Minor
>             Fix For: 0.16.0
>
>         Attachments: patch2596.txt
>
>
> Currently it is not possible to create a SequenceFile.Writer using a block size other than the default.
> The createWriter() method should be overloaded with a signature receiving block size as parameter should be added to the the SequenceFile class.
> With all the current signatures for this method there is a significant code duplication, if possible the createWriter() methods  should be refactored to avoid such duplication.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

     [ https://issues.apache.org/jira/browse/HADOOP-2596?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Arun C Murthy updated HADOOP-2596:
----------------------------------

    Status: Patch Available  (was: Open)

> add SequenceFile.createWriter() method that takes block size as parameter
> -------------------------------------------------------------------------
>
>                 Key: HADOOP-2596
>                 URL: https://issues.apache.org/jira/browse/HADOOP-2596
>             Project: Hadoop
>          Issue Type: Improvement
>          Components: io
>         Environment: all
>            Reporter: Alejandro Abdelnur
>            Assignee: Alejandro Abdelnur
>            Priority: Minor
>             Fix For: 0.16.0
>
>         Attachments: patch2596.txt
>
>
> Currently it is not possible to create a SequenceFile.Writer using a block size other than the default.
> The createWriter() method should be overloaded with a signature receiving block size as parameter should be added to the the SequenceFile class.
> With all the current signatures for this method there is a significant code duplication, if possible the createWriter() methods  should be refactored to avoid such duplication.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

    [ https://issues.apache.org/jira/browse/HADOOP-2596?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12560800#action_12560800 ]

Hadoop QA commented on HADOOP-2596:
-----------------------------------

-1 overall.  Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12373467/patch2596.txt
against trunk revision r613499.

    @author +1.  The patch does not contain any @author tags.

    javadoc +1.  The javadoc tool did not generate any warning messages.

    javac +1.  The applied patch does not generate any new compiler warnings.

    findbugs +1.  The patch does not introduce any new Findbugs warnings.

    core tests -1.  The patch failed core unit tests.

    contrib tests +1.  The patch passed contrib unit tests.

Test results: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/1660/testReport/
Findbugs warnings: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/1660/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Checkstyle results: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/1660/artifact/trunk/build/test/checkstyle-errors.html
Console output: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/1660/console

This message is automatically generated.

> add SequenceFile.createWriter() method that takes block size as parameter
> -------------------------------------------------------------------------
>
>                 Key: HADOOP-2596
>                 URL: https://issues.apache.org/jira/browse/HADOOP-2596
>             Project: Hadoop
>          Issue Type: Improvement
>          Components: io
>         Environment: all
>            Reporter: Alejandro Abdelnur
>            Assignee: Alejandro Abdelnur
>            Priority: Minor
>             Fix For: 0.16.0
>
>         Attachments: patch2596.txt
>
>
> Currently it is not possible to create a SequenceFile.Writer using a block size other than the default.
> The createWriter() method should be overloaded with a signature receiving block size as parameter should be added to the the SequenceFile class.
> With all the current signatures for this method there is a significant code duplication, if possible the createWriter() methods  should be refactored to avoid such duplication.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

    [ https://issues.apache.org/jira/browse/HADOOP-2596?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12561795#action_12561795 ]

Arun C Murthy commented on HADOOP-2596:
---------------------------------------

Sigh, looks like another unrelated test failed.

I'm ok about this patch, but then Doug's concern about too many SequenceFile.createWriter methods (HADOOP-938) springs to my mind... thoughts? Doug?

> add SequenceFile.createWriter() method that takes block size as parameter
> -------------------------------------------------------------------------
>
>                 Key: HADOOP-2596
>                 URL: https://issues.apache.org/jira/browse/HADOOP-2596
>             Project: Hadoop Core
>          Issue Type: Improvement
>          Components: io
>         Environment: all
>            Reporter: Alejandro Abdelnur
>            Assignee: Alejandro Abdelnur
>            Priority: Minor
>             Fix For: 0.16.0
>
>         Attachments: patch2596.txt
>
>
> Currently it is not possible to create a SequenceFile.Writer using a block size other than the default.
> The createWriter() method should be overloaded with a signature receiving block size as parameter should be added to the the SequenceFile class.
> With all the current signatures for this method there is a significant code duplication, if possible the createWriter() methods  should be refactored to avoid such duplication.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

    [ https://issues.apache.org/jira/browse/HADOOP-2596?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12561853#action_12561853 ]

Doug Cutting commented on HADOOP-2596:
--------------------------------------

I don't think we ought to hold this up for HADOOP-938.  We do need to address the plethora of options, but that's a separate issue.

> add SequenceFile.createWriter() method that takes block size as parameter
> -------------------------------------------------------------------------
>
>                 Key: HADOOP-2596
>                 URL: https://issues.apache.org/jira/browse/HADOOP-2596
>             Project: Hadoop Core
>          Issue Type: Improvement
>          Components: io
>         Environment: all
>            Reporter: Alejandro Abdelnur
>            Assignee: Alejandro Abdelnur
>            Priority: Minor
>             Fix For: 0.16.0
>
>         Attachments: patch2596.txt
>
>
> Currently it is not possible to create a SequenceFile.Writer using a block size other than the default.
> The createWriter() method should be overloaded with a signature receiving block size as parameter should be added to the the SequenceFile class.
> With all the current signatures for this method there is a significant code duplication, if possible the createWriter() methods  should be refactored to avoid such duplication.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.