Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-9286

HttpFs does not parse ACL syntax correctly for operation REMOVEACLENTRIES

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.6.0
    • Fix Version/s: 2.8.0, 3.0.0-alpha1
    • Component/s: fs
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      Output from WebHdfs:
      curl -X PUT "http://weichiu.vpc.cloudera.com:50070/webhdfs/v1/a?aclspec=group:user:&op=REMOVEACLENTRIES&user.name=weichiu"

      Output from HttpFs:
      curl -X PUT "http://weichiu.vpc.cloudera.com:14000/webhdfs/v1/a?aclspec=group:user:&op=REMOVEACLENTRIES&user.name=weichiu"

      {"RemoteException":{"message":"Invalid <aclSpec> : group:user:","exception":"HadoopIllegalArgumentException","javaClassName":"org.apache.hadoop.HadoopIllegalArgumentException"}}

      The exception is thrown because the ACL string "group:user:" is not recognized by HttpFs.
      Effectively, what this means is that the behavior of HttpFs is not consistent with that of WebHdfs.

      Additionally, command line does accept the ACL string. For example:
      hdfs dfds -setfactl -x group:yyy: /a

      Bug is reproducible if httpfs and acl are enabled, and reproducible on single-node cluster configuration.

      To reproduce, add into core-site.xml:
      <property>
      <name>dfs.webhdfs.enabled</name>
      <value>true</value>
      </property>
      <property>
      <name>dfs.namenode.acls.enabled</name>
      <value>true</value>
      </property>
      <property>
      <name>hadoop.proxyuser.#HTTPFSUSER#.hosts</name>
      <value>httpfs-host.foo.com</value>
      </property>
      <property>
      <name>hadoop.proxyuser.#HTTPFSUSER#.groups</name>
      <value>*</value>
      </property>

      restart name node, data node and httpfs daemon

      Credit to Romain Rigaux for reporting the issue.

      1. HDFS-9286.001.patch
        2 kB
        Wei-Chiu Chuang
      2. HDFS-9286.002.patch
        5 kB
        Wei-Chiu Chuang

        Activity

        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #529 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/529/)
        HDFS-9286. HttpFs does not parse ACL syntax correctly for operation (cnauroth: rev 124a412a3711bd42eaeebe531376004c739a15d6)

        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • hadoop-hdfs-project/hadoop-hdfs-httpfs/src/main/java/org/apache/hadoop/fs/http/server/FSOperations.java
        • hadoop-hdfs-project/hadoop-hdfs-httpfs/src/test/java/org/apache/hadoop/fs/http/server/TestHttpFSServerNoACLs.java
        • hadoop-hdfs-project/hadoop-hdfs-httpfs/src/test/java/org/apache/hadoop/fs/http/server/TestHttpFSServer.java
        • hadoop-hdfs-project/hadoop-hdfs-httpfs/src/test/java/org/apache/hadoop/fs/http/client/BaseTestHttpFSWith.java
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #529 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/529/ ) HDFS-9286 . HttpFs does not parse ACL syntax correctly for operation (cnauroth: rev 124a412a3711bd42eaeebe531376004c739a15d6) hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs-httpfs/src/main/java/org/apache/hadoop/fs/http/server/FSOperations.java hadoop-hdfs-project/hadoop-hdfs-httpfs/src/test/java/org/apache/hadoop/fs/http/server/TestHttpFSServerNoACLs.java hadoop-hdfs-project/hadoop-hdfs-httpfs/src/test/java/org/apache/hadoop/fs/http/server/TestHttpFSServer.java hadoop-hdfs-project/hadoop-hdfs-httpfs/src/test/java/org/apache/hadoop/fs/http/client/BaseTestHttpFSWith.java
        Hide
        cnauroth Chris Nauroth added a comment -

        It's fun working on this stuff.

        Hehe. Yeah, I'm hooked too.

        Show
        cnauroth Chris Nauroth added a comment - It's fun working on this stuff. Hehe. Yeah, I'm hooked too.
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Mapreduce-trunk #2518 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2518/)
        HDFS-9286. HttpFs does not parse ACL syntax correctly for operation (cnauroth: rev 124a412a3711bd42eaeebe531376004c739a15d6)

        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • hadoop-hdfs-project/hadoop-hdfs-httpfs/src/main/java/org/apache/hadoop/fs/http/server/FSOperations.java
        • hadoop-hdfs-project/hadoop-hdfs-httpfs/src/test/java/org/apache/hadoop/fs/http/server/TestHttpFSServer.java
        • hadoop-hdfs-project/hadoop-hdfs-httpfs/src/test/java/org/apache/hadoop/fs/http/client/BaseTestHttpFSWith.java
        • hadoop-hdfs-project/hadoop-hdfs-httpfs/src/test/java/org/apache/hadoop/fs/http/server/TestHttpFSServerNoACLs.java
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk #2518 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2518/ ) HDFS-9286 . HttpFs does not parse ACL syntax correctly for operation (cnauroth: rev 124a412a3711bd42eaeebe531376004c739a15d6) hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs-httpfs/src/main/java/org/apache/hadoop/fs/http/server/FSOperations.java hadoop-hdfs-project/hadoop-hdfs-httpfs/src/test/java/org/apache/hadoop/fs/http/server/TestHttpFSServer.java hadoop-hdfs-project/hadoop-hdfs-httpfs/src/test/java/org/apache/hadoop/fs/http/client/BaseTestHttpFSWith.java hadoop-hdfs-project/hadoop-hdfs-httpfs/src/test/java/org/apache/hadoop/fs/http/server/TestHttpFSServerNoACLs.java
        Hide
        jojochuang Wei-Chiu Chuang added a comment -

        Chris Nauroth Thanks for the review! It's fun working on this stuff.

        Show
        jojochuang Wei-Chiu Chuang added a comment - Chris Nauroth Thanks for the review! It's fun working on this stuff.
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #572 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/572/)
        HDFS-9286. HttpFs does not parse ACL syntax correctly for operation (cnauroth: rev 124a412a3711bd42eaeebe531376004c739a15d6)

        • hadoop-hdfs-project/hadoop-hdfs-httpfs/src/test/java/org/apache/hadoop/fs/http/client/BaseTestHttpFSWith.java
        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • hadoop-hdfs-project/hadoop-hdfs-httpfs/src/main/java/org/apache/hadoop/fs/http/server/FSOperations.java
        • hadoop-hdfs-project/hadoop-hdfs-httpfs/src/test/java/org/apache/hadoop/fs/http/server/TestHttpFSServerNoACLs.java
        • hadoop-hdfs-project/hadoop-hdfs-httpfs/src/test/java/org/apache/hadoop/fs/http/server/TestHttpFSServer.java
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #572 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/572/ ) HDFS-9286 . HttpFs does not parse ACL syntax correctly for operation (cnauroth: rev 124a412a3711bd42eaeebe531376004c739a15d6) hadoop-hdfs-project/hadoop-hdfs-httpfs/src/test/java/org/apache/hadoop/fs/http/client/BaseTestHttpFSWith.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs-httpfs/src/main/java/org/apache/hadoop/fs/http/server/FSOperations.java hadoop-hdfs-project/hadoop-hdfs-httpfs/src/test/java/org/apache/hadoop/fs/http/server/TestHttpFSServerNoACLs.java hadoop-hdfs-project/hadoop-hdfs-httpfs/src/test/java/org/apache/hadoop/fs/http/server/TestHttpFSServer.java
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Yarn-trunk #1308 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/1308/)
        HDFS-9286. HttpFs does not parse ACL syntax correctly for operation (cnauroth: rev 124a412a3711bd42eaeebe531376004c739a15d6)

        • hadoop-hdfs-project/hadoop-hdfs-httpfs/src/main/java/org/apache/hadoop/fs/http/server/FSOperations.java
        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • hadoop-hdfs-project/hadoop-hdfs-httpfs/src/test/java/org/apache/hadoop/fs/http/server/TestHttpFSServer.java
        • hadoop-hdfs-project/hadoop-hdfs-httpfs/src/test/java/org/apache/hadoop/fs/http/server/TestHttpFSServerNoACLs.java
        • hadoop-hdfs-project/hadoop-hdfs-httpfs/src/test/java/org/apache/hadoop/fs/http/client/BaseTestHttpFSWith.java
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk #1308 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/1308/ ) HDFS-9286 . HttpFs does not parse ACL syntax correctly for operation (cnauroth: rev 124a412a3711bd42eaeebe531376004c739a15d6) hadoop-hdfs-project/hadoop-hdfs-httpfs/src/main/java/org/apache/hadoop/fs/http/server/FSOperations.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs-httpfs/src/test/java/org/apache/hadoop/fs/http/server/TestHttpFSServer.java hadoop-hdfs-project/hadoop-hdfs-httpfs/src/test/java/org/apache/hadoop/fs/http/server/TestHttpFSServerNoACLs.java hadoop-hdfs-project/hadoop-hdfs-httpfs/src/test/java/org/apache/hadoop/fs/http/client/BaseTestHttpFSWith.java
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Hdfs-trunk #2465 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2465/)
        HDFS-9286. HttpFs does not parse ACL syntax correctly for operation (cnauroth: rev 124a412a3711bd42eaeebe531376004c739a15d6)

        • hadoop-hdfs-project/hadoop-hdfs-httpfs/src/test/java/org/apache/hadoop/fs/http/server/TestHttpFSServerNoACLs.java
        • hadoop-hdfs-project/hadoop-hdfs-httpfs/src/test/java/org/apache/hadoop/fs/http/client/BaseTestHttpFSWith.java
        • hadoop-hdfs-project/hadoop-hdfs-httpfs/src/main/java/org/apache/hadoop/fs/http/server/FSOperations.java
        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • hadoop-hdfs-project/hadoop-hdfs-httpfs/src/test/java/org/apache/hadoop/fs/http/server/TestHttpFSServer.java
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk #2465 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2465/ ) HDFS-9286 . HttpFs does not parse ACL syntax correctly for operation (cnauroth: rev 124a412a3711bd42eaeebe531376004c739a15d6) hadoop-hdfs-project/hadoop-hdfs-httpfs/src/test/java/org/apache/hadoop/fs/http/server/TestHttpFSServerNoACLs.java hadoop-hdfs-project/hadoop-hdfs-httpfs/src/test/java/org/apache/hadoop/fs/http/client/BaseTestHttpFSWith.java hadoop-hdfs-project/hadoop-hdfs-httpfs/src/main/java/org/apache/hadoop/fs/http/server/FSOperations.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs-httpfs/src/test/java/org/apache/hadoop/fs/http/server/TestHttpFSServer.java
        Hide
        hudson Hudson added a comment -

        SUCCESS: Integrated in Hadoop-Yarn-trunk-Java8 #586 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/586/)
        HDFS-9286. HttpFs does not parse ACL syntax correctly for operation (cnauroth: rev 124a412a3711bd42eaeebe531376004c739a15d6)

        • hadoop-hdfs-project/hadoop-hdfs-httpfs/src/test/java/org/apache/hadoop/fs/http/client/BaseTestHttpFSWith.java
        • hadoop-hdfs-project/hadoop-hdfs-httpfs/src/test/java/org/apache/hadoop/fs/http/server/TestHttpFSServer.java
        • hadoop-hdfs-project/hadoop-hdfs-httpfs/src/test/java/org/apache/hadoop/fs/http/server/TestHttpFSServerNoACLs.java
        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • hadoop-hdfs-project/hadoop-hdfs-httpfs/src/main/java/org/apache/hadoop/fs/http/server/FSOperations.java
        Show
        hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-Yarn-trunk-Java8 #586 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/586/ ) HDFS-9286 . HttpFs does not parse ACL syntax correctly for operation (cnauroth: rev 124a412a3711bd42eaeebe531376004c739a15d6) hadoop-hdfs-project/hadoop-hdfs-httpfs/src/test/java/org/apache/hadoop/fs/http/client/BaseTestHttpFSWith.java hadoop-hdfs-project/hadoop-hdfs-httpfs/src/test/java/org/apache/hadoop/fs/http/server/TestHttpFSServer.java hadoop-hdfs-project/hadoop-hdfs-httpfs/src/test/java/org/apache/hadoop/fs/http/server/TestHttpFSServerNoACLs.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs-httpfs/src/main/java/org/apache/hadoop/fs/http/server/FSOperations.java
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-trunk-Commit #8693 (See https://builds.apache.org/job/Hadoop-trunk-Commit/8693/)
        HDFS-9286. HttpFs does not parse ACL syntax correctly for operation (cnauroth: rev 124a412a3711bd42eaeebe531376004c739a15d6)

        • hadoop-hdfs-project/hadoop-hdfs-httpfs/src/test/java/org/apache/hadoop/fs/http/server/TestHttpFSServerNoACLs.java
        • hadoop-hdfs-project/hadoop-hdfs-httpfs/src/test/java/org/apache/hadoop/fs/http/client/BaseTestHttpFSWith.java
        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • hadoop-hdfs-project/hadoop-hdfs-httpfs/src/test/java/org/apache/hadoop/fs/http/server/TestHttpFSServer.java
        • hadoop-hdfs-project/hadoop-hdfs-httpfs/src/main/java/org/apache/hadoop/fs/http/server/FSOperations.java
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-trunk-Commit #8693 (See https://builds.apache.org/job/Hadoop-trunk-Commit/8693/ ) HDFS-9286 . HttpFs does not parse ACL syntax correctly for operation (cnauroth: rev 124a412a3711bd42eaeebe531376004c739a15d6) hadoop-hdfs-project/hadoop-hdfs-httpfs/src/test/java/org/apache/hadoop/fs/http/server/TestHttpFSServerNoACLs.java hadoop-hdfs-project/hadoop-hdfs-httpfs/src/test/java/org/apache/hadoop/fs/http/client/BaseTestHttpFSWith.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs-httpfs/src/test/java/org/apache/hadoop/fs/http/server/TestHttpFSServer.java hadoop-hdfs-project/hadoop-hdfs-httpfs/src/main/java/org/apache/hadoop/fs/http/server/FSOperations.java
        Hide
        cnauroth Chris Nauroth added a comment -

        I have committed this to trunk and branch-2. Wei-Chiu Chuang, thank you for contributing the patch. Romain Rigaux, thank you for finding and reporting the bug.

        Show
        cnauroth Chris Nauroth added a comment - I have committed this to trunk and branch-2. Wei-Chiu Chuang , thank you for contributing the patch. Romain Rigaux , thank you for finding and reporting the bug.
        Hide
        cnauroth Chris Nauroth added a comment -

        +1 for the patch. I'll commit this in a few minutes.

        Show
        cnauroth Chris Nauroth added a comment - +1 for the patch. I'll commit this in a few minutes.
        Hide
        hadoopqa Hadoop QA added a comment -



        +1 overall



        Vote Subsystem Runtime Comment
        0 pre-patch 16m 15s Pre-patch trunk compilation is healthy.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 tests included 0m 0s The patch appears to include 3 new or modified test files.
        +1 javac 7m 54s There were no new javac warning messages.
        +1 javadoc 10m 15s There were no new javadoc warning messages.
        +1 release audit 0m 23s The applied patch does not increase the total number of release audit warnings.
        +1 checkstyle 0m 28s There were no new checkstyle issues.
        +1 whitespace 0m 0s The patch has no lines that end in whitespace.
        +1 install 1m 27s mvn install still works.
        +1 eclipse:eclipse 0m 34s The patch built with eclipse:eclipse.
        +1 findbugs 0m 54s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
        +1 native 3m 13s Pre-build of native portion
        +1 hdfs tests 3m 38s Tests passed in hadoop-hdfs-httpfs.
            45m 4s  



        Subsystem Report/Notes
        Patch URL http://issues.apache.org/jira/secure/attachment/12768124/HDFS-9286.002.patch
        Optional Tests javadoc javac unit findbugs checkstyle
        git revision trunk / 0fce5f9
        hadoop-hdfs-httpfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/13135/artifact/patchprocess/testrun_hadoop-hdfs-httpfs.txt
        Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/13135/testReport/
        Java 1.7.0_55
        uname Linux asf903.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/13135/console

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - +1 overall Vote Subsystem Runtime Comment 0 pre-patch 16m 15s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 3 new or modified test files. +1 javac 7m 54s There were no new javac warning messages. +1 javadoc 10m 15s There were no new javadoc warning messages. +1 release audit 0m 23s The applied patch does not increase the total number of release audit warnings. +1 checkstyle 0m 28s There were no new checkstyle issues. +1 whitespace 0m 0s The patch has no lines that end in whitespace. +1 install 1m 27s mvn install still works. +1 eclipse:eclipse 0m 34s The patch built with eclipse:eclipse. +1 findbugs 0m 54s The patch does not introduce any new Findbugs (version 3.0.0) warnings. +1 native 3m 13s Pre-build of native portion +1 hdfs tests 3m 38s Tests passed in hadoop-hdfs-httpfs.     45m 4s   Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12768124/HDFS-9286.002.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / 0fce5f9 hadoop-hdfs-httpfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/13135/artifact/patchprocess/testrun_hadoop-hdfs-httpfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/13135/testReport/ Java 1.7.0_55 uname Linux asf903.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-HDFS-Build/13135/console This message was automatically generated.
        Hide
        jojochuang Wei-Chiu Chuang added a comment -

        Fixed all test cases which use the wrong ACL format to remove an ACL entry.

        Show
        jojochuang Wei-Chiu Chuang added a comment - Fixed all test cases which use the wrong ACL format to remove an ACL entry.
        Hide
        jojochuang Wei-Chiu Chuang added a comment -

        rev 1. fixed the bug in parsing ACL, and fixed the test case (the test case did not use the correct ACL string to remove the ACL entry)

        Show
        jojochuang Wei-Chiu Chuang added a comment - rev 1. fixed the bug in parsing ACL, and fixed the test case (the test case did not use the correct ACL string to remove the ACL entry)

          People

          • Assignee:
            jojochuang Wei-Chiu Chuang
            Reporter:
            jojochuang Wei-Chiu Chuang
          • Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development