Hadoop Common
  1. Hadoop Common
  2. HADOOP-6856

SequenceFile and MapFile need cleanup to remove redundant constructors

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.22.0
    • Component/s: io
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      Currently there are 2 public SequenceFile.Reader constructors, 3 public SequenceFile.Writer constructors, 9 public SequenceFile.createWriter, 2 public MapFile.Reader constructors, and 8 public MapFile.Writer constructors. All of with various historical combinations of parameters that don't cover the entire space.

      All of this makes it very difficult to add new optional parameters to SequenceFile and MapFile.

      I'd like change to the style of FileContext.create with option parameters. I'll implement one public SequenceFile.Reader constructor and one public SequenceFile.createWriter and implement all of the current variants based on those two. I'll do the same for MapFile.Reader and MapFile.Writer including passing parameters down to the underlying SequenceFile.

      1. h-6856.patch
        90 kB
        Owen O'Malley
      2. h-6856.patch
        82 kB
        Owen O'Malley

        Issue Links

          Activity

          Hide
          Owen O'Malley added a comment -

          This patch adds the new constructors and makes all of the old (now deprecated) constructors call the new one.

          Show
          Owen O'Malley added a comment - This patch adds the new constructors and makes all of the old (now deprecated) constructors call the new one.
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12449138/h-6856.patch
          against trunk revision 962677.

          +1 @author. The patch does not contain any @author tags.

          -1 tests included. The patch doesn't appear to include any new or modified tests.
          Please justify why no new tests are needed for this patch.
          Also please list what manual steps were performed to verify this patch.

          -1 javadoc. The javadoc tool appears to have generated 1 warning messages.

          -1 javac. The applied patch generated 1065 javac compiler warnings (more than the trunk's current 1017 warnings).

          -1 findbugs. The patch appears to introduce 1 new Findbugs warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed core unit tests.

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h1.grid.sp2.yahoo.net/85/testReport/
          Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h1.grid.sp2.yahoo.net/85/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
          Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h1.grid.sp2.yahoo.net/85/artifact/trunk/build/test/checkstyle-errors.html
          Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h1.grid.sp2.yahoo.net/85/console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12449138/h-6856.patch against trunk revision 962677. +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. -1 javadoc. The javadoc tool appears to have generated 1 warning messages. -1 javac. The applied patch generated 1065 javac compiler warnings (more than the trunk's current 1017 warnings). -1 findbugs. The patch appears to introduce 1 new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed core unit tests. +1 contrib tests. The patch passed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h1.grid.sp2.yahoo.net/85/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h1.grid.sp2.yahoo.net/85/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h1.grid.sp2.yahoo.net/85/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h1.grid.sp2.yahoo.net/85/console This message is automatically generated.
          Hide
          Owen O'Malley added a comment -

          Fixed the javac and javadoc warnings.

          Show
          Owen O'Malley added a comment - Fixed the javac and javadoc warnings.
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12449147/h-6856.patch
          against trunk revision 962677.

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 2 new or modified tests.

          -1 javadoc. The javadoc tool appears to have generated 1 warning messages.

          -1 javac. The applied patch generated 1035 javac compiler warnings (more than the trunk's current 1017 warnings).

          +1 findbugs. The patch does not introduce any new Findbugs warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 core tests. The patch failed core unit tests.

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h1.grid.sp2.yahoo.net/86/testReport/
          Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h1.grid.sp2.yahoo.net/86/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
          Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h1.grid.sp2.yahoo.net/86/artifact/trunk/build/test/checkstyle-errors.html
          Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h1.grid.sp2.yahoo.net/86/console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12449147/h-6856.patch against trunk revision 962677. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 2 new or modified tests. -1 javadoc. The javadoc tool appears to have generated 1 warning messages. -1 javac. The applied patch generated 1035 javac compiler warnings (more than the trunk's current 1017 warnings). +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed core unit tests. +1 contrib tests. The patch passed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h1.grid.sp2.yahoo.net/86/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h1.grid.sp2.yahoo.net/86/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h1.grid.sp2.yahoo.net/86/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h1.grid.sp2.yahoo.net/86/console This message is automatically generated.
          Hide
          Chris Douglas added a comment -

          +1 I really like the Option pattern, here. It would be interesting to see it applied in FileContext

          HADOOP-6885 fixed the javadoc warnings; could you regenerate?

          Show
          Chris Douglas added a comment - +1 I really like the Option pattern, here. It would be interesting to see it applied in FileContext HADOOP-6885 fixed the javadoc warnings; could you regenerate?
          Hide
          Owen O'Malley added a comment -

          I just committed this.

          Show
          Owen O'Malley added a comment - I just committed this.
          Hide
          Konstantin Boudnik added a comment -

          I wonder if all tests were executed in the light of this HADOOP-6989?

          Show
          Konstantin Boudnik added a comment - I wonder if all tests were executed in the light of this HADOOP-6989 ?
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Common-trunk-Commit #395 (See https://hudson.apache.org/hudson/job/Hadoop-Common-trunk-Commit/395/)

          Show
          Hudson added a comment - Integrated in Hadoop-Common-trunk-Commit #395 (See https://hudson.apache.org/hudson/job/Hadoop-Common-trunk-Commit/395/ )
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Common-trunk #489 (See https://hudson.apache.org/hudson/job/Hadoop-Common-trunk/489/)

          Show
          Hudson added a comment - Integrated in Hadoop-Common-trunk #489 (See https://hudson.apache.org/hudson/job/Hadoop-Common-trunk/489/ )
          Hide
          Nigel Daley added a comment -

          Owen, given MAPREDUCE-2121, did this deprecate a constructor and change the semantics of the deprecated constructor?

          Show
          Nigel Daley added a comment - Owen, given MAPREDUCE-2121 , did this deprecate a constructor and change the semantics of the deprecated constructor?

            People

            • Assignee:
              Owen O'Malley
              Reporter:
              Owen O'Malley
            • Votes:
              1 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development