Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-10481

HTTPFS server should correctly impersonate as end user to open file

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 0.23.1
    • Fix Version/s: 2.8.0, 3.0.0-alpha1
    • Component/s: httpfs
    • Labels:
      None
    • Target Version/s:

      Description

      Seen issues where applications like oozie/hue connecting to httpfs to read an encryption zone will end up being rejected by the KMS, because logged in as httpfs instead of the end user.

      The issue exists since the initial feature HDFS-2178.

      1. HDFS-10481.01.patch
        2 kB
        Xiao Chen
      2. HDFS-10481.02.patch
        11 kB
        Xiao Chen

        Activity

        Hide
        xiaochen Xiao Chen added a comment -

        Attaching a patch to fix this. Manually tested the existing issue is fixed after this patch.

        Sanity checked the code, only OPEN needs be fixed here, since we're special-casing it for streaming read. All other actions execute through FileSystemAccessService#execute, which correctly impersonates.

        Show
        xiaochen Xiao Chen added a comment - Attaching a patch to fix this. Manually tested the existing issue is fixed after this patch. Sanity checked the code, only OPEN needs be fixed here, since we're special-casing it for streaming read. All other actions execute through FileSystemAccessService#execute , which correctly impersonates.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 21s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
        +1 mvninstall 5m 56s trunk passed
        +1 compile 0m 14s trunk passed
        +1 checkstyle 0m 18s trunk passed
        +1 mvnsite 0m 23s trunk passed
        +1 mvneclipse 0m 13s trunk passed
        +1 findbugs 0m 23s trunk passed
        +1 javadoc 0m 13s trunk passed
        +1 mvninstall 0m 27s the patch passed
        +1 compile 0m 12s the patch passed
        +1 javac 0m 12s the patch passed
        -1 checkstyle 0m 16s hadoop-hdfs-project/hadoop-hdfs-httpfs: The patch generated 15 new + 398 unchanged - 6 fixed = 413 total (was 404)
        +1 mvnsite 0m 21s the patch passed
        +1 mvneclipse 0m 7s the patch passed
        +1 whitespace 0m 1s The patch has no whitespace issues.
        +1 findbugs 0m 28s the patch passed
        +1 javadoc 0m 10s the patch passed
        +1 unit 3m 19s hadoop-hdfs-httpfs in the patch passed.
        +1 asflicense 0m 16s The patch does not generate ASF License warnings.
        14m 47s



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:2c91fd8
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12807910/HDFS-10481.01.patch
        JIRA Issue HDFS-10481
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux d4f4af6848ae 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 97e2449
        Default Java 1.8.0_91
        findbugs v3.0.0
        checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/15641/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs-httpfs.txt
        Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/15641/testReport/
        modules C: hadoop-hdfs-project/hadoop-hdfs-httpfs U: hadoop-hdfs-project/hadoop-hdfs-httpfs
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/15641/console
        Powered by Apache Yetus 0.3.0 http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 21s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 mvninstall 5m 56s trunk passed +1 compile 0m 14s trunk passed +1 checkstyle 0m 18s trunk passed +1 mvnsite 0m 23s trunk passed +1 mvneclipse 0m 13s trunk passed +1 findbugs 0m 23s trunk passed +1 javadoc 0m 13s trunk passed +1 mvninstall 0m 27s the patch passed +1 compile 0m 12s the patch passed +1 javac 0m 12s the patch passed -1 checkstyle 0m 16s hadoop-hdfs-project/hadoop-hdfs-httpfs: The patch generated 15 new + 398 unchanged - 6 fixed = 413 total (was 404) +1 mvnsite 0m 21s the patch passed +1 mvneclipse 0m 7s the patch passed +1 whitespace 0m 1s The patch has no whitespace issues. +1 findbugs 0m 28s the patch passed +1 javadoc 0m 10s the patch passed +1 unit 3m 19s hadoop-hdfs-httpfs in the patch passed. +1 asflicense 0m 16s The patch does not generate ASF License warnings. 14m 47s Subsystem Report/Notes Docker Image:yetus/hadoop:2c91fd8 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12807910/HDFS-10481.01.patch JIRA Issue HDFS-10481 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux d4f4af6848ae 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 97e2449 Default Java 1.8.0_91 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/15641/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs-httpfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/15641/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs-httpfs U: hadoop-hdfs-project/hadoop-hdfs-httpfs Console output https://builds.apache.org/job/PreCommit-HDFS-Build/15641/console Powered by Apache Yetus 0.3.0 http://yetus.apache.org This message was automatically generated.
        Hide
        xiaochen Xiao Chen added a comment -

        Patch 2 to make checkstyle happy. (It's the switch indentation that's in original code).

        I didn't add any unit test for this because I manually tested (similar to the fix in HADOOP-12787). It will be an end-to-end testing anyways.

        Show
        xiaochen Xiao Chen added a comment - Patch 2 to make checkstyle happy. (It's the switch indentation that's in original code). I didn't add any unit test for this because I manually tested (similar to the fix in HADOOP-12787 ). It will be an end-to-end testing anyways.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 17m 36s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
        +1 mvninstall 7m 46s trunk passed
        +1 compile 0m 18s trunk passed
        +1 checkstyle 0m 20s trunk passed
        +1 mvnsite 0m 27s trunk passed
        +1 mvneclipse 0m 11s trunk passed
        +1 findbugs 0m 28s trunk passed
        +1 javadoc 0m 15s trunk passed
        +1 mvninstall 0m 32s the patch passed
        +1 compile 0m 14s the patch passed
        +1 javac 0m 14s the patch passed
        -1 checkstyle 0m 15s hadoop-hdfs-project/hadoop-hdfs-httpfs: The patch generated 13 new + 268 unchanged - 136 fixed = 281 total (was 404)
        +1 mvnsite 0m 23s the patch passed
        +1 mvneclipse 0m 8s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 findbugs 0m 34s the patch passed
        +1 javadoc 0m 12s the patch passed
        +1 unit 3m 57s hadoop-hdfs-httpfs in the patch passed.
        +1 asflicense 0m 18s The patch does not generate ASF License warnings.
        35m 20s



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:2c91fd8
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12808024/HDFS-10481.02.patch
        JIRA Issue HDFS-10481
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux 700b2b5d147f 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 097baaa
        Default Java 1.8.0_91
        findbugs v3.0.0
        checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/15644/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs-httpfs.txt
        Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/15644/testReport/
        modules C: hadoop-hdfs-project/hadoop-hdfs-httpfs U: hadoop-hdfs-project/hadoop-hdfs-httpfs
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/15644/console
        Powered by Apache Yetus 0.3.0 http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 17m 36s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 mvninstall 7m 46s trunk passed +1 compile 0m 18s trunk passed +1 checkstyle 0m 20s trunk passed +1 mvnsite 0m 27s trunk passed +1 mvneclipse 0m 11s trunk passed +1 findbugs 0m 28s trunk passed +1 javadoc 0m 15s trunk passed +1 mvninstall 0m 32s the patch passed +1 compile 0m 14s the patch passed +1 javac 0m 14s the patch passed -1 checkstyle 0m 15s hadoop-hdfs-project/hadoop-hdfs-httpfs: The patch generated 13 new + 268 unchanged - 136 fixed = 281 total (was 404) +1 mvnsite 0m 23s the patch passed +1 mvneclipse 0m 8s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 0m 34s the patch passed +1 javadoc 0m 12s the patch passed +1 unit 3m 57s hadoop-hdfs-httpfs in the patch passed. +1 asflicense 0m 18s The patch does not generate ASF License warnings. 35m 20s Subsystem Report/Notes Docker Image:yetus/hadoop:2c91fd8 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12808024/HDFS-10481.02.patch JIRA Issue HDFS-10481 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 700b2b5d147f 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 097baaa Default Java 1.8.0_91 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/15644/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs-httpfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/15644/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs-httpfs U: hadoop-hdfs-project/hadoop-hdfs-httpfs Console output https://builds.apache.org/job/PreCommit-HDFS-Build/15644/console Powered by Apache Yetus 0.3.0 http://yetus.apache.org This message was automatically generated.
        Hide
        andrew.wang Andrew Wang added a comment -

        Nice work here Xiao, +1 LGTM and thanks for the contribution.

        Committed back through 2.8.0.

        Show
        andrew.wang Andrew Wang added a comment - Nice work here Xiao, +1 LGTM and thanks for the contribution. Committed back through 2.8.0.
        Hide
        hudson Hudson added a comment -

        SUCCESS: Integrated in Hadoop-trunk-Commit #9912 (See https://builds.apache.org/job/Hadoop-trunk-Commit/9912/)
        HDFS-10481. HTTPFS server should correctly impersonate as end user to (wang: rev 47e0321ee91149331e6ae72e7caa41d1de078b6c)

        • hadoop-hdfs-project/hadoop-hdfs-httpfs/src/main/java/org/apache/hadoop/fs/http/server/HttpFSServer.java
        Show
        hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-trunk-Commit #9912 (See https://builds.apache.org/job/Hadoop-trunk-Commit/9912/ ) HDFS-10481 . HTTPFS server should correctly impersonate as end user to (wang: rev 47e0321ee91149331e6ae72e7caa41d1de078b6c) hadoop-hdfs-project/hadoop-hdfs-httpfs/src/main/java/org/apache/hadoop/fs/http/server/HttpFSServer.java

          People

          • Assignee:
            xiaochen Xiao Chen
            Reporter:
            xiaochen Xiao Chen
          • Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development