Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-7997

The first non-existing xattr should also throw IOException

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: 2.6.0
    • Fix Version/s: 2.8.0, 3.0.0-alpha1
    • Component/s: None
    • Labels:
      None
    • Target Version/s:
    • Hadoop Flags:
      Reviewed

      Description

      We use the following code snippet to get/set xattrs. However, if there are no xattrs have ever been set, the first getXAttr returns null and the second one just throws exception with message like "At least one of the attributes provided was not found.". This is not expected, we believe they should behave in the same way - i.e either both getXAttr returns null or both getXAttr throw exception with the message "... not found". We will provide a patch to make them both throw exception.

      ....
      attrValueNM = fs.getXAttr(path, "nm");
      if (attrValueNM == null) {
      fs.setXAttr("nm", DEFAULT_VALUE);
      }
      attrValueNN = fs.getXAttr(path, "nn");
      if (attrValueNN == null) {
      fs.setXAttr("nn", DEFAULT_VALUE);
      }
      ....

        Activity

        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Mapreduce-trunk #2100 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2100/)
        HDFS-7997. The first non-existing xattr should also throw IOException. (zhouyingchao via yliu) (yliu: rev 3df61f303a8c0f5105661dc86fc3a34a60c49066)

        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirXAttrOp.java
        • hadoop-hdfs-project/hadoop-hdfs/src/test/resources/testXAttrConf.xml
        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk #2100 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2100/ ) HDFS-7997 . The first non-existing xattr should also throw IOException. (zhouyingchao via yliu) (yliu: rev 3df61f303a8c0f5105661dc86fc3a34a60c49066) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirXAttrOp.java hadoop-hdfs-project/hadoop-hdfs/src/test/resources/testXAttrConf.xml hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #141 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/141/)
        HDFS-7997. The first non-existing xattr should also throw IOException. (zhouyingchao via yliu) (yliu: rev 3df61f303a8c0f5105661dc86fc3a34a60c49066)

        • hadoop-hdfs-project/hadoop-hdfs/src/test/resources/testXAttrConf.xml
        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirXAttrOp.java
        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #141 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/141/ ) HDFS-7997 . The first non-existing xattr should also throw IOException. (zhouyingchao via yliu) (yliu: rev 3df61f303a8c0f5105661dc86fc3a34a60c49066) hadoop-hdfs-project/hadoop-hdfs/src/test/resources/testXAttrConf.xml hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirXAttrOp.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #150 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/150/)
        HDFS-7997. The first non-existing xattr should also throw IOException. (zhouyingchao via yliu) (yliu: rev 3df61f303a8c0f5105661dc86fc3a34a60c49066)

        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • hadoop-hdfs-project/hadoop-hdfs/src/test/resources/testXAttrConf.xml
        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirXAttrOp.java
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #150 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/150/ ) HDFS-7997 . The first non-existing xattr should also throw IOException. (zhouyingchao via yliu) (yliu: rev 3df61f303a8c0f5105661dc86fc3a34a60c49066) hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/test/resources/testXAttrConf.xml hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirXAttrOp.java
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Hdfs-trunk #2082 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2082/)
        HDFS-7997. The first non-existing xattr should also throw IOException. (zhouyingchao via yliu) (yliu: rev 3df61f303a8c0f5105661dc86fc3a34a60c49066)

        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • hadoop-hdfs-project/hadoop-hdfs/src/test/resources/testXAttrConf.xml
        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirXAttrOp.java
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk #2082 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2082/ ) HDFS-7997 . The first non-existing xattr should also throw IOException. (zhouyingchao via yliu) (yliu: rev 3df61f303a8c0f5105661dc86fc3a34a60c49066) hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/test/resources/testXAttrConf.xml hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirXAttrOp.java
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Yarn-trunk #884 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/884/)
        HDFS-7997. The first non-existing xattr should also throw IOException. (zhouyingchao via yliu) (yliu: rev 3df61f303a8c0f5105661dc86fc3a34a60c49066)

        • hadoop-hdfs-project/hadoop-hdfs/src/test/resources/testXAttrConf.xml
        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirXAttrOp.java
        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk #884 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/884/ ) HDFS-7997 . The first non-existing xattr should also throw IOException. (zhouyingchao via yliu) (yliu: rev 3df61f303a8c0f5105661dc86fc3a34a60c49066) hadoop-hdfs-project/hadoop-hdfs/src/test/resources/testXAttrConf.xml hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirXAttrOp.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #150 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/150/)
        HDFS-7997. The first non-existing xattr should also throw IOException. (zhouyingchao via yliu) (yliu: rev 3df61f303a8c0f5105661dc86fc3a34a60c49066)

        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • hadoop-hdfs-project/hadoop-hdfs/src/test/resources/testXAttrConf.xml
        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirXAttrOp.java
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #150 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/150/ ) HDFS-7997 . The first non-existing xattr should also throw IOException. (zhouyingchao via yliu) (yliu: rev 3df61f303a8c0f5105661dc86fc3a34a60c49066) hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/test/resources/testXAttrConf.xml hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirXAttrOp.java
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-trunk-Commit #7475 (See https://builds.apache.org/job/Hadoop-trunk-Commit/7475/)
        HDFS-7997. The first non-existing xattr should also throw IOException. (zhouyingchao via yliu) (yliu: rev 3df61f303a8c0f5105661dc86fc3a34a60c49066)

        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirXAttrOp.java
        • hadoop-hdfs-project/hadoop-hdfs/src/test/resources/testXAttrConf.xml
        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-trunk-Commit #7475 (See https://builds.apache.org/job/Hadoop-trunk-Commit/7475/ ) HDFS-7997 . The first non-existing xattr should also throw IOException. (zhouyingchao via yliu) (yliu: rev 3df61f303a8c0f5105661dc86fc3a34a60c49066) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirXAttrOp.java hadoop-hdfs-project/hadoop-hdfs/src/test/resources/testXAttrConf.xml hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        Hide
        hitliuyi Yi Liu added a comment -

        Committed to trunk and branch-2

        Show
        hitliuyi Yi Liu added a comment - Committed to trunk and branch-2
        Hide
        sinago zhouyingchao added a comment -

        Thank you for pointing it out. Actually we are using the name of "user.xxx", the pseudo code snippet here is just used to explain the issue.

        Show
        sinago zhouyingchao added a comment - Thank you for pointing it out. Actually we are using the name of "user.xxx", the pseudo code snippet here is just used to explain the issue.
        Hide
        hitliuyi Yi Liu added a comment -

        zhouyingchao, the example you gave is not correct, the xattr should be prefixed with user/trusted/security/system/raw. But the patch looks good.

        Show
        hitliuyi Yi Liu added a comment - zhouyingchao , the example you gave is not correct, the xattr should be prefixed with user/trusted/security/system/raw. But the patch looks good.
        Hide
        sinago zhouyingchao added a comment -

        The failed test case is not related to the change. I just verified that it cannot pass without the xattr changes.

        Show
        sinago zhouyingchao added a comment - The failed test case is not related to the change. I just verified that it cannot pass without the xattr changes.
        Hide
        hadoopqa Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12707713/HDFS-7997-001.patch
        against trunk revision af618f2.

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 1 new or modified test files.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 javadoc. There were no new javadoc warning messages.

        +1 eclipse:eclipse. The patch built with eclipse:eclipse.

        +1 findbugs. The patch does not introduce any new Findbugs (version 2.0.3) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        -1 core tests. The patch failed these unit tests in hadoop-hdfs-project/hadoop-hdfs:

        org.apache.hadoop.hdfs.server.namenode.snapshot.TestOpenFilesWithSnapshot

        Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/10085//testReport/
        Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/10085//console

        This message is automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12707713/HDFS-7997-001.patch against trunk revision af618f2. +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 1 new or modified test files. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . There were no new javadoc warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 2.0.3) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. -1 core tests . The patch failed these unit tests in hadoop-hdfs-project/hadoop-hdfs: org.apache.hadoop.hdfs.server.namenode.snapshot.TestOpenFilesWithSnapshot Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/10085//testReport/ Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/10085//console This message is automatically generated.
        Hide
        sinago zhouyingchao added a comment -

        Test with " -Dtest=TestXAttrCLI,TestXAttrWithSnapshot,TestNameNodeXAttr,TestFileContextXAttr,FSXAttrBaseTest,TestFSImageWithXAttr,TestXAttrConfigFlag,TestXAttrsWithHA,TestWebHDFSXAttr,TestXAttr,TestViewFileSystemWithXAttrs,TestViewFsWithXAttrs "

        Show
        sinago zhouyingchao added a comment - Test with " -Dtest=TestXAttrCLI,TestXAttrWithSnapshot,TestNameNodeXAttr,TestFileContextXAttr,FSXAttrBaseTest,TestFSImageWithXAttr,TestXAttrConfigFlag,TestXAttrsWithHA,TestWebHDFSXAttr,TestXAttr,TestViewFileSystemWithXAttrs,TestViewFsWithXAttrs "

          People

          • Assignee:
            sinago zhouyingchao
            Reporter:
            sinago zhouyingchao
          • Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development