Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-8395

Text shell command unnecessarily demands that a SequenceFile's key class be WritableComparable

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Trivial
    • Resolution: Fixed
    • Affects Version/s: 2.0.0-alpha
    • Fix Version/s: 3.0.0-alpha1
    • Component/s: util
    • Labels:

      Description

      Text from Display set of Shell commands (hadoop fs -text), has a strict subclass check for a sequence-file-header loaded key class to be a subclass of WritableComparable.

      The sequence file writer itself has no such checks (one can create sequence files with just plain writable keys, comparable is needed for sequence file's sorter alone, which not all of them use always), and hence its not reasonable for Text command to carry it either.

      We should relax the check and simply just check for "Writable", not "WritableComparable".

        Activity

        Hide
        hadoopqa Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12526563/HADOOP-8395.patch
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        -1 tests included. The patch doesn't appear to include any new or modified tests.
        Please justify why no new tests are needed for this patch.
        Also please list what manual steps were performed to verify this patch.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 eclipse:eclipse. The patch built with eclipse:eclipse.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 core tests. The patch passed unit tests in hadoop-common-project/hadoop-common.

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/986//testReport/
        Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/986//console

        This message is automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12526563/HADOOP-8395.patch against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 javadoc. The javadoc tool did not generate any warning messages. +1 eclipse:eclipse. The patch built with eclipse:eclipse. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in hadoop-common-project/hadoop-common. +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/986//testReport/ Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/986//console This message is automatically generated.
        Hide
        tlipcon Todd Lipcon added a comment -

        Seems reasonable. +1 assuming you tested this manually.

        Show
        tlipcon Todd Lipcon added a comment - Seems reasonable. +1 assuming you tested this manually.
        Hide
        qwertymaniac Harsh J added a comment -

        Thanks for checking Todd.

        Yes, I manually tested the Text command with these two types of files (one used a WritableComparable set, the other uses Writable sets), just to make sure the subclass checks don't break anything.

        As WritableComparable subclasses Writable, this does not break anything, only relaxes the false requirement.

        Committing to trunk shortly.

        Show
        qwertymaniac Harsh J added a comment - Thanks for checking Todd. Yes, I manually tested the Text command with these two types of files (one used a WritableComparable set, the other uses Writable sets), just to make sure the subclass checks don't break anything. As WritableComparable subclasses Writable, this does not break anything, only relaxes the false requirement. Committing to trunk shortly.
        Hide
        hudson Hudson added a comment -

        Integrated in Hadoop-Common-trunk-Commit #2237 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/2237/)
        HADOOP-8395. Text shell command unnecessarily demands that a SequenceFile's key class be WritableComparable (harsh) (Revision 1337449)

        Result = SUCCESS
        harsh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1337449
        Files :

        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/shell/Display.java
        Show
        hudson Hudson added a comment - Integrated in Hadoop-Common-trunk-Commit #2237 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/2237/ ) HADOOP-8395 . Text shell command unnecessarily demands that a SequenceFile's key class be WritableComparable (harsh) (Revision 1337449) Result = SUCCESS harsh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1337449 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/shell/Display.java
        Hide
        hudson Hudson added a comment -

        Integrated in Hadoop-Hdfs-trunk-Commit #2311 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/2311/)
        HADOOP-8395. Text shell command unnecessarily demands that a SequenceFile's key class be WritableComparable (harsh) (Revision 1337449)

        Result = SUCCESS
        harsh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1337449
        Files :

        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/shell/Display.java
        Show
        hudson Hudson added a comment - Integrated in Hadoop-Hdfs-trunk-Commit #2311 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/2311/ ) HADOOP-8395 . Text shell command unnecessarily demands that a SequenceFile's key class be WritableComparable (harsh) (Revision 1337449) Result = SUCCESS harsh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1337449 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/shell/Display.java
        Hide
        qwertymaniac Harsh J added a comment -

        Committed to trunk.

        Show
        qwertymaniac Harsh J added a comment - Committed to trunk.
        Hide
        hudson Hudson added a comment -

        Integrated in Hadoop-Mapreduce-trunk-Commit #2254 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/2254/)
        HADOOP-8395. Text shell command unnecessarily demands that a SequenceFile's key class be WritableComparable (harsh) (Revision 1337449)

        Result = ABORTED
        harsh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1337449
        Files :

        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/shell/Display.java
        Show
        hudson Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk-Commit #2254 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/2254/ ) HADOOP-8395 . Text shell command unnecessarily demands that a SequenceFile's key class be WritableComparable (harsh) (Revision 1337449) Result = ABORTED harsh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1337449 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/shell/Display.java
        Hide
        hudson Hudson added a comment -

        Integrated in Hadoop-Hdfs-trunk #1041 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1041/)
        HADOOP-8395. Text shell command unnecessarily demands that a SequenceFile's key class be WritableComparable (harsh) (Revision 1337449)

        Result = FAILURE
        harsh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1337449
        Files :

        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/shell/Display.java
        Show
        hudson Hudson added a comment - Integrated in Hadoop-Hdfs-trunk #1041 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1041/ ) HADOOP-8395 . Text shell command unnecessarily demands that a SequenceFile's key class be WritableComparable (harsh) (Revision 1337449) Result = FAILURE harsh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1337449 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/shell/Display.java
        Hide
        hudson Hudson added a comment -

        Integrated in Hadoop-Mapreduce-trunk #1077 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1077/)
        HADOOP-8395. Text shell command unnecessarily demands that a SequenceFile's key class be WritableComparable (harsh) (Revision 1337449)

        Result = SUCCESS
        harsh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1337449
        Files :

        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/shell/Display.java
        Show
        hudson Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk #1077 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1077/ ) HADOOP-8395 . Text shell command unnecessarily demands that a SequenceFile's key class be WritableComparable (harsh) (Revision 1337449) Result = SUCCESS harsh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1337449 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/shell/Display.java

          People

          • Assignee:
            qwertymaniac Harsh J
            Reporter:
            qwertymaniac Harsh J
          • Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development