Hadoop Common
  1. Hadoop Common
  2. HADOOP-2596

add SequenceFile.createWriter() method that takes block size as parameter

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Minor Minor
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.16.0
    • Component/s: io
    • Labels:
      None
    • Environment:

      all

      Description

      Currently it is not possible to create a SequenceFile.Writer using a block size other than the default.

      The createWriter() method should be overloaded with a signature receiving block size as parameter should be added to the the SequenceFile class.

      With all the current signatures for this method there is a significant code duplication, if possible the createWriter() methods should be refactored to avoid such duplication.

      1. patch2596.txt
        13 kB
        Alejandro Abdelnur

        Activity

        Hide
        Hudson added a comment -
        Show
        Hudson added a comment - Integrated in Hadoop-trunk #378 (See http://hudson.zones.apache.org/hudson/job/Hadoop-trunk/378/ )
        Hide
        Arun C Murthy added a comment -

        I just committed this. Thanks, Alejandro!

        Show
        Arun C Murthy added a comment - I just committed this. Thanks, Alejandro!
        Hide
        Doug Cutting added a comment -

        I don't think we ought to hold this up for HADOOP-938. We do need to address the plethora of options, but that's a separate issue.

        Show
        Doug Cutting added a comment - I don't think we ought to hold this up for HADOOP-938 . We do need to address the plethora of options, but that's a separate issue.
        Hide
        Arun C Murthy added a comment -

        Sigh, looks like another unrelated test failed.

        I'm ok about this patch, but then Doug's concern about too many SequenceFile.createWriter methods (HADOOP-938) springs to my mind... thoughts? Doug?

        Show
        Arun C Murthy added a comment - Sigh, looks like another unrelated test failed. I'm ok about this patch, but then Doug's concern about too many SequenceFile.createWriter methods ( HADOOP-938 ) springs to my mind... thoughts? Doug?
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12373467/patch2596.txt
        against trunk revision r613499.

        @author +1. The patch does not contain any @author tags.

        javadoc +1. The javadoc tool did not generate any warning messages.

        javac +1. The applied patch does not generate any new compiler warnings.

        findbugs +1. The patch does not introduce any new Findbugs warnings.

        core tests -1. The patch failed core unit tests.

        contrib tests +1. The patch passed contrib unit tests.

        Test results: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/1660/testReport/
        Findbugs warnings: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/1660/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
        Checkstyle results: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/1660/artifact/trunk/build/test/checkstyle-errors.html
        Console output: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/1660/console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12373467/patch2596.txt against trunk revision r613499. @author +1. The patch does not contain any @author tags. javadoc +1. The javadoc tool did not generate any warning messages. javac +1. The applied patch does not generate any new compiler warnings. findbugs +1. The patch does not introduce any new Findbugs warnings. core tests -1. The patch failed core unit tests. contrib tests +1. The patch passed contrib unit tests. Test results: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/1660/testReport/ Findbugs warnings: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/1660/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/1660/artifact/trunk/build/test/checkstyle-errors.html Console output: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/1660/console This message is automatically generated.
        Hide
        Arun C Murthy added a comment -

        Re-submitting to hudson since an unrelated test failed...

        Show
        Arun C Murthy added a comment - Re-submitting to hudson since an unrelated test failed...
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12373467/patch2596.txt
        against trunk revision r613115.

        @author +1. The patch does not contain any @author tags.

        javadoc +1. The javadoc tool did not generate any warning messages.

        javac +1. The applied patch does not generate any new compiler warnings.

        findbugs +1. The patch does not introduce any new Findbugs warnings.

        core tests -1. The patch failed core unit tests.

        contrib tests +1. The patch passed contrib unit tests.

        Test results: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/1640/testReport/
        Findbugs warnings: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/1640/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
        Checkstyle results: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/1640/artifact/trunk/build/test/checkstyle-errors.html
        Console output: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/1640/console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12373467/patch2596.txt against trunk revision r613115. @author +1. The patch does not contain any @author tags. javadoc +1. The javadoc tool did not generate any warning messages. javac +1. The applied patch does not generate any new compiler warnings. findbugs +1. The patch does not introduce any new Findbugs warnings. core tests -1. The patch failed core unit tests. contrib tests +1. The patch passed contrib unit tests. Test results: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/1640/testReport/ Findbugs warnings: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/1640/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/1640/artifact/trunk/build/test/checkstyle-errors.html Console output: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/1640/console This message is automatically generated.
        Hide
        Alejandro Abdelnur added a comment -

        Added a most generic createWriter() method that takes buffersize, replication and blockSize. Refactored all other methods removing duplicated logic by calling more generic signature.

        Added constructors to the SequenceFile writer implementations to support the new parameters above. Refactored writer constructors to remove duplicate logic by calling more generic signature.

        Show
        Alejandro Abdelnur added a comment - Added a most generic createWriter() method that takes buffersize, replication and blockSize. Refactored all other methods removing duplicated logic by calling more generic signature. Added constructors to the SequenceFile writer implementations to support the new parameters above. Refactored writer constructors to remove duplicate logic by calling more generic signature.

          People

          • Assignee:
            Alejandro Abdelnur
            Reporter:
            Alejandro Abdelnur
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development