Details

    • Type: Bug
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 3.0.0-alpha4
    • Component/s: fs
    • Labels:
      None

      Description

      Some frameworks rely on Java serialization to pass objects between processes. FileStatus is a common argument, but it only supports Writable serialization without special handling.

        Issue Links

          Activity

          Hide
          hudson Hudson added a comment -

          SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #11195 (See https://builds.apache.org/job/Hadoop-trunk-Commit/11195/)
          HADOOP-13895. Make FileStatus Serializable (cdouglas: rev 59c5f18784121f04030d1d0982f2e2285688ee11)

          • (edit) hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestFileStatus.java
          • (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/BlockLocation.java
          • (edit) hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocol/FsPermissionExtension.java
          • (edit) hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/fs/HdfsBlockLocation.java
          • (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FileStatus.java
          • (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/permission/FsCreateModes.java
          • (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/permission/FsPermission.java
          Show
          hudson Hudson added a comment - SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #11195 (See https://builds.apache.org/job/Hadoop-trunk-Commit/11195/ ) HADOOP-13895 . Make FileStatus Serializable (cdouglas: rev 59c5f18784121f04030d1d0982f2e2285688ee11) (edit) hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestFileStatus.java (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/BlockLocation.java (edit) hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocol/FsPermissionExtension.java (edit) hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/fs/HdfsBlockLocation.java (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FileStatus.java (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/permission/FsCreateModes.java (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/permission/FsPermission.java
          Hide
          chris.douglas Chris Douglas added a comment -

          Thanks, Steve.

          I committed this.

          Show
          chris.douglas Chris Douglas added a comment - Thanks, Steve. I committed this.
          Hide
          stevel@apache.org Steve Loughran added a comment -

          LGTM

          +1

          Show
          stevel@apache.org Steve Loughran added a comment - LGTM +1
          Hide
          chris.douglas Chris Douglas added a comment -

          Steve Loughran Would you mind taking a quick look?

          Show
          chris.douglas Chris Douglas added a comment - Steve Loughran Would you mind taking a quick look?
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 15s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
          0 mvndep 0m 15s Maven dependency ordering for branch
          +1 mvninstall 6m 44s trunk passed
          +1 compile 9m 26s trunk passed
          +1 checkstyle 1m 33s trunk passed
          +1 mvnsite 1m 42s trunk passed
          +1 mvneclipse 0m 36s trunk passed
          +1 findbugs 2m 59s trunk passed
          +1 javadoc 1m 17s trunk passed
          0 mvndep 0m 18s Maven dependency ordering for patch
          +1 mvninstall 1m 11s the patch passed
          +1 compile 9m 11s the patch passed
          +1 javac 9m 11s the patch passed
          +1 checkstyle 1m 36s the patch passed
          +1 mvnsite 1m 45s the patch passed
          +1 mvneclipse 0m 42s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 findbugs 3m 31s the patch passed
          +1 javadoc 1m 30s the patch passed
          -1 unit 8m 52s hadoop-common in the patch failed.
          +1 unit 1m 14s hadoop-hdfs-client in the patch passed.
          +1 asflicense 0m 38s The patch does not generate ASF License warnings.
          78m 53s



          Reason Tests
          Failed junit tests hadoop.security.token.delegation.web.TestWebDelegationToken



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:a9ad5d6
          JIRA Issue HADOOP-13895
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12842867/HADOOP-13895.000.patch
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux 01996ea65a86 3.13.0-103-generic #150-Ubuntu SMP Thu Nov 24 10:34:17 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / f66f618
          Default Java 1.8.0_111
          findbugs v3.0.0
          unit https://builds.apache.org/job/PreCommit-HADOOP-Build/11252/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common.txt
          Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/11252/testReport/
          modules C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs-client U: .
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/11252/console
          Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 15s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. 0 mvndep 0m 15s Maven dependency ordering for branch +1 mvninstall 6m 44s trunk passed +1 compile 9m 26s trunk passed +1 checkstyle 1m 33s trunk passed +1 mvnsite 1m 42s trunk passed +1 mvneclipse 0m 36s trunk passed +1 findbugs 2m 59s trunk passed +1 javadoc 1m 17s trunk passed 0 mvndep 0m 18s Maven dependency ordering for patch +1 mvninstall 1m 11s the patch passed +1 compile 9m 11s the patch passed +1 javac 9m 11s the patch passed +1 checkstyle 1m 36s the patch passed +1 mvnsite 1m 45s the patch passed +1 mvneclipse 0m 42s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 3m 31s the patch passed +1 javadoc 1m 30s the patch passed -1 unit 8m 52s hadoop-common in the patch failed. +1 unit 1m 14s hadoop-hdfs-client in the patch passed. +1 asflicense 0m 38s The patch does not generate ASF License warnings. 78m 53s Reason Tests Failed junit tests hadoop.security.token.delegation.web.TestWebDelegationToken Subsystem Report/Notes Docker Image:yetus/hadoop:a9ad5d6 JIRA Issue HADOOP-13895 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12842867/HADOOP-13895.000.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 01996ea65a86 3.13.0-103-generic #150-Ubuntu SMP Thu Nov 24 10:34:17 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / f66f618 Default Java 1.8.0_111 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-HADOOP-Build/11252/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common.txt Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/11252/testReport/ modules C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs-client U: . Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/11252/console Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          chris.douglas Chris Douglas added a comment -

          Andrew Wang, Steve Loughran please review

          Show
          chris.douglas Chris Douglas added a comment - Andrew Wang , Steve Loughran please review
          Hide
          chris.douglas Chris Douglas added a comment -

          Moved relevant Serializable code from HDFS-6984.

          Changed FileStatus to be Serializable, per Steve Loughran's suggestion. This cascaded to a few other classes, which I halted at HdfsBlockLocation (changing the final ref to transient). Looking through its usage this is probably correct, since the fields not redundant with BlockLocation are things like tokens, which are internal to DFSClient.

          Show
          chris.douglas Chris Douglas added a comment - Moved relevant Serializable code from HDFS-6984 . Changed FileStatus to be Serializable , per Steve Loughran 's suggestion. This cascaded to a few other classes, which I halted at HdfsBlockLocation (changing the final ref to transient). Looking through its usage this is probably correct, since the fields not redundant with BlockLocation are things like tokens, which are internal to DFSClient.

            People

            • Assignee:
              chris.douglas Chris Douglas
              Reporter:
              chris.douglas Chris Douglas
            • Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development