Hadoop Common
  1. Hadoop Common
  2. HADOOP-2596

add SequenceFile.createWriter() method that takes block size as parameter

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Minor Minor
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.16.0
    • Component/s: io
    • Labels:
      None
    • Environment:

      all

      Description

      Currently it is not possible to create a SequenceFile.Writer using a block size other than the default.

      The createWriter() method should be overloaded with a signature receiving block size as parameter should be added to the the SequenceFile class.

      With all the current signatures for this method there is a significant code duplication, if possible the createWriter() methods should be refactored to avoid such duplication.

      1. patch2596.txt
        13 kB
        Alejandro Abdelnur

        Activity

        Alejandro Abdelnur created issue -
        Hide
        Alejandro Abdelnur added a comment -

        Added a most generic createWriter() method that takes buffersize, replication and blockSize. Refactored all other methods removing duplicated logic by calling more generic signature.

        Added constructors to the SequenceFile writer implementations to support the new parameters above. Refactored writer constructors to remove duplicate logic by calling more generic signature.

        Show
        Alejandro Abdelnur added a comment - Added a most generic createWriter() method that takes buffersize, replication and blockSize. Refactored all other methods removing duplicated logic by calling more generic signature. Added constructors to the SequenceFile writer implementations to support the new parameters above. Refactored writer constructors to remove duplicate logic by calling more generic signature.
        Alejandro Abdelnur made changes -
        Field Original Value New Value
        Attachment patch2596.txt [ 12373467 ]
        Alejandro Abdelnur made changes -
        Assignee Alejandro Abdelnur [ tucu00 ]
        Alejandro Abdelnur made changes -
        Fix Version/s 0.16.0 [ 12312740 ]
        Status Open [ 1 ] Patch Available [ 10002 ]
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12373467/patch2596.txt
        against trunk revision r613115.

        @author +1. The patch does not contain any @author tags.

        javadoc +1. The javadoc tool did not generate any warning messages.

        javac +1. The applied patch does not generate any new compiler warnings.

        findbugs +1. The patch does not introduce any new Findbugs warnings.

        core tests -1. The patch failed core unit tests.

        contrib tests +1. The patch passed contrib unit tests.

        Test results: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/1640/testReport/
        Findbugs warnings: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/1640/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
        Checkstyle results: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/1640/artifact/trunk/build/test/checkstyle-errors.html
        Console output: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/1640/console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12373467/patch2596.txt against trunk revision r613115. @author +1. The patch does not contain any @author tags. javadoc +1. The javadoc tool did not generate any warning messages. javac +1. The applied patch does not generate any new compiler warnings. findbugs +1. The patch does not introduce any new Findbugs warnings. core tests -1. The patch failed core unit tests. contrib tests +1. The patch passed contrib unit tests. Test results: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/1640/testReport/ Findbugs warnings: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/1640/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/1640/artifact/trunk/build/test/checkstyle-errors.html Console output: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/1640/console This message is automatically generated.
        Hide
        Arun C Murthy added a comment -

        Re-submitting to hudson since an unrelated test failed...

        Show
        Arun C Murthy added a comment - Re-submitting to hudson since an unrelated test failed...
        Arun C Murthy made changes -
        Status Patch Available [ 10002 ] Open [ 1 ]
        Arun C Murthy made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12373467/patch2596.txt
        against trunk revision r613499.

        @author +1. The patch does not contain any @author tags.

        javadoc +1. The javadoc tool did not generate any warning messages.

        javac +1. The applied patch does not generate any new compiler warnings.

        findbugs +1. The patch does not introduce any new Findbugs warnings.

        core tests -1. The patch failed core unit tests.

        contrib tests +1. The patch passed contrib unit tests.

        Test results: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/1660/testReport/
        Findbugs warnings: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/1660/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
        Checkstyle results: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/1660/artifact/trunk/build/test/checkstyle-errors.html
        Console output: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/1660/console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12373467/patch2596.txt against trunk revision r613499. @author +1. The patch does not contain any @author tags. javadoc +1. The javadoc tool did not generate any warning messages. javac +1. The applied patch does not generate any new compiler warnings. findbugs +1. The patch does not introduce any new Findbugs warnings. core tests -1. The patch failed core unit tests. contrib tests +1. The patch passed contrib unit tests. Test results: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/1660/testReport/ Findbugs warnings: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/1660/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/1660/artifact/trunk/build/test/checkstyle-errors.html Console output: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/1660/console This message is automatically generated.
        Hide
        Arun C Murthy added a comment -

        Sigh, looks like another unrelated test failed.

        I'm ok about this patch, but then Doug's concern about too many SequenceFile.createWriter methods (HADOOP-938) springs to my mind... thoughts? Doug?

        Show
        Arun C Murthy added a comment - Sigh, looks like another unrelated test failed. I'm ok about this patch, but then Doug's concern about too many SequenceFile.createWriter methods ( HADOOP-938 ) springs to my mind... thoughts? Doug?
        Hide
        Doug Cutting added a comment -

        I don't think we ought to hold this up for HADOOP-938. We do need to address the plethora of options, but that's a separate issue.

        Show
        Doug Cutting added a comment - I don't think we ought to hold this up for HADOOP-938 . We do need to address the plethora of options, but that's a separate issue.
        Hide
        Arun C Murthy added a comment -

        I just committed this. Thanks, Alejandro!

        Show
        Arun C Murthy added a comment - I just committed this. Thanks, Alejandro!
        Arun C Murthy made changes -
        Status Patch Available [ 10002 ] Resolved [ 5 ]
        Resolution Fixed [ 1 ]
        Hide
        Hudson added a comment -
        Show
        Hudson added a comment - Integrated in Hadoop-trunk #378 (See http://hudson.zones.apache.org/hudson/job/Hadoop-trunk/378/ )
        Nigel Daley made changes -
        Status Resolved [ 5 ] Closed [ 6 ]
        Transition Time In Source Status Execution Times Last Executer Last Execution Date
        Patch Available Patch Available Open Open
        19h 49m 1 Arun C Murthy 19/Jan/08 00:31
        Open Open Patch Available Patch Available
        4d 16m 2 Arun C Murthy 19/Jan/08 00:32
        Patch Available Patch Available Resolved Resolved
        5d 1h 22m 1 Arun C Murthy 24/Jan/08 01:54
        Resolved Resolved Closed Closed
        15d 21h 43m 1 Nigel Daley 08/Feb/08 23:38

          People

          • Assignee:
            Alejandro Abdelnur
            Reporter:
            Alejandro Abdelnur
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development