[jira] Created: (HADOOP-938) too many SequenceFile.createWriter() methods

too many SequenceFile.createWriter() methods
--------------------------------------------

                 Key: HADOOP-938
                 URL: https://issues.apache.org/jira/browse/HADOOP-938
             Project: Hadoop
          Issue Type: Improvement
            Reporter: Doug Cutting


There are too many SequenceFile.createWriter() method signatures.  This method has two required paramters: a Configuration and a Path.  It has one obsolete parameter: a FileSystem.  And it has five optional parameters: CompressionType, CompressionCodec, Progress, replication, and metadata.

We should remove the obsolete parameter and make all optional parameters into setters.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

    [ https://issues.apache.org/jira/browse/HADOOP-938?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12467663 ]

Doug Cutting commented on HADOOP-938:
-------------------------------------

I can think of a couple of ways to implement this:

1. Put options in a Configuration:

Configuration options = new Configuration(defaults);
SequenceFile.setCompressionType(options, Compress.BLOCK);
SequenceFile.Writer writer = SequenceFile.createWriter(options, path);

This would require making the Configuration constructor lighter weight.

2. Put options in a new parameter class:

SequenceFile.Options options = new SequenceFile.Options(defaults);
options.setCompressionType(options, Compress.BLOCK);
SequenceFile.Writer writer = SequenceFile.createWriter(options, path);

Thoughts?

> too many SequenceFile.createWriter() methods
> --------------------------------------------
>
>                 Key: HADOOP-938
>                 URL: https://issues.apache.org/jira/browse/HADOOP-938
>             Project: Hadoop
>          Issue Type: Improvement
>            Reporter: Doug Cutting
>
> There are too many SequenceFile.createWriter() method signatures.  This method has two required paramters: a Configuration and a Path.  It has one obsolete parameter: a FileSystem.  And it has five optional parameters: CompressionType, CompressionCodec, Progress, replication, and metadata.
> We should remove the obsolete parameter and make all optional parameters into setters.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

    [ https://issues.apache.org/jira/browse/HADOOP-938?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12467664 ]

Doug Judd commented on HADOOP-938:
----------------------------------

This should also support setting the replication factor as an optional setting.  See JIRA bug 907.

> too many SequenceFile.createWriter() methods
> --------------------------------------------
>
>                 Key: HADOOP-938
>                 URL: https://issues.apache.org/jira/browse/HADOOP-938
>             Project: Hadoop
>          Issue Type: Improvement
>            Reporter: Doug Cutting
>
> There are too many SequenceFile.createWriter() method signatures.  This method has two required paramters: a Configuration and a Path.  It has one obsolete parameter: a FileSystem.  And it has five optional parameters: CompressionType, CompressionCodec, Progress, replication, and metadata.
> We should remove the obsolete parameter and make all optional parameters into setters.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

    [ https://issues.apache.org/jira/browse/HADOOP-938?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12467748 ]

Andrzej Bialecki  commented on HADOOP-938:
------------------------------------------

IMHO it's better to use a specialized parameter class instead of opaque Configuration - it's much more light weight, and we can make it type-safe.

> too many SequenceFile.createWriter() methods
> --------------------------------------------
>
>                 Key: HADOOP-938
>                 URL: https://issues.apache.org/jira/browse/HADOOP-938
>             Project: Hadoop
>          Issue Type: Improvement
>            Reporter: Doug Cutting
>
> There are too many SequenceFile.createWriter() method signatures.  This method has two required paramters: a Configuration and a Path.  It has one obsolete parameter: a FileSystem.  And it has five optional parameters: CompressionType, CompressionCodec, Progress, replication, and metadata.
> We should remove the obsolete parameter and make all optional parameters into setters.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

     [ https://issues.apache.org/jira/browse/HADOOP-938?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Sameer Paranjpye updated HADOOP-938:
------------------------------------

    Component/s: io

> too many SequenceFile.createWriter() methods
> --------------------------------------------
>
>                 Key: HADOOP-938
>                 URL: https://issues.apache.org/jira/browse/HADOOP-938
>             Project: Hadoop
>          Issue Type: Improvement
>          Components: io
>            Reporter: Doug Cutting
>
> There are too many SequenceFile.createWriter() method signatures.  This method has two required paramters: a Configuration and a Path.  It has one obsolete parameter: a FileSystem.  And it has five optional parameters: CompressionType, CompressionCodec, Progress, replication, and metadata.
> We should remove the obsolete parameter and make all optional parameters into setters.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

    [ https://issues.apache.org/jira/browse/HADOOP-938?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12561852#action_12561852 ]

Doug Cutting commented on HADOOP-938:
-------------------------------------

Another approach would be to add an open() method, and use SequenceFile.Writer setter methods, e.g.:

SequenceFile.Writer writer = SequenceFile.createWriter(path);
writer.setCompressionType(Compress.BLOCK);
writer.setBlockSize(128*1024*1024);
writer.open();

> too many SequenceFile.createWriter() methods
> --------------------------------------------
>
>                 Key: HADOOP-938
>                 URL: https://issues.apache.org/jira/browse/HADOOP-938
>             Project: Hadoop Core
>          Issue Type: Improvement
>          Components: io
>            Reporter: Doug Cutting
>
> There are too many SequenceFile.createWriter() method signatures.  This method has two required paramters: a Configuration and a Path.  It has one obsolete parameter: a FileSystem.  And it has five optional parameters: CompressionType, CompressionCodec, Progress, replication, and metadata.
> We should remove the obsolete parameter and make all optional parameters into setters.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.