Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-5033

Bad error message for fs -put/copyFromLocal if user doesn't have permissions to read the source

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: 2.0.3-alpha, 3.0.0-alpha1
    • Fix Version/s: 3.0.0-alpha1
    • Component/s: None
    • Labels:
    • Release Note:
      "Permission denied" error message when unable to read local file for -put/copyFromLocal

      Description

      fs -put/copyFromLocal shows a "No such file or directory" error when the user doesn't have permissions to read the source file/directory. Saying "Permission Denied" is more useful to the user.

      1. HDFS-5033.001.patch
        50 kB
        Darrell Taylor

        Activity

        Hide
        sarutak Kousuke Saruta added a comment -

        Hi Karthik,

        I think the message which means "Permission Denied" should not be displayed on the client side from the view point of the security.
        Instead, that should be logged on the server side as audit log.

        How do you think?

        Show
        sarutak Kousuke Saruta added a comment - Hi Karthik, I think the message which means "Permission Denied" should not be displayed on the client side from the view point of the security. Instead, that should be logged on the server side as audit log. How do you think?
        Hide
        kkambatl Karthik Kambatla (Inactive) added a comment -

        The source for put/copyFromLocal is on the client. From a security POV, it should be okay to show the "Permission Denied" message.

        Show
        kkambatl Karthik Kambatla (Inactive) added a comment - The source for put/copyFromLocal is on the client. From a security POV, it should be okay to show the "Permission Denied" message.
        Hide
        sarutak Kousuke Saruta added a comment -

        "Permission Denied" lets malicious users know existence of files. I think if one doesn't allow to read a file, we shouldn't let them know.

        Show
        sarutak Kousuke Saruta added a comment - "Permission Denied" lets malicious users know existence of files. I think if one doesn't allow to read a file, we shouldn't let them know.
        Hide
        kkambatl Karthik Kambatla (Inactive) added a comment -

        On Linux (POSIX), this is what I see:

        kasha@keka:~/code/hadoop-trunk (trunk)$ ls -l testfile 
        -rw------- 1 root root 8 Jul 29 18:31 testfile
        kasha@keka:~/code/hadoop-trunk (trunk)$ cat testfile 
        cat: testfile: Permission denied
        

        When accessing local files/dirs through HDFS, the behavior should be the same as when using local file system directly.

        Show
        kkambatl Karthik Kambatla (Inactive) added a comment - On Linux (POSIX), this is what I see: kasha@keka:~/code/hadoop-trunk (trunk)$ ls -l testfile -rw------- 1 root root 8 Jul 29 18:31 testfile kasha@keka:~/code/hadoop-trunk (trunk)$ cat testfile cat: testfile: Permission denied When accessing local files/dirs through HDFS, the behavior should be the same as when using local file system directly.
        Hide
        sarutak Kousuke Saruta added a comment -

        I think the behavior should be same as the local file system too.
        I agree with you.

        Show
        sarutak Kousuke Saruta added a comment - I think the behavior should be same as the local file system too. I agree with you.
        Hide
        d4rr3ll Darrell Taylor added a comment -

        -001

        • Low level change in FileUtil.java
        • New test in TestDFSShell.java
        Show
        d4rr3ll Darrell Taylor added a comment - -001 Low level change in FileUtil.java New test in TestDFSShell.java
        Hide
        hadoopqa Hadoop QA added a comment -



        +1 overall



        Vote Subsystem Runtime Comment
        0 pre-patch 14m 32s Pre-patch trunk compilation is healthy.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 tests included 0m 0s The patch appears to include 1 new or modified test files.
        +1 javac 7m 31s There were no new javac warning messages.
        +1 javadoc 9m 36s There were no new javadoc warning messages.
        +1 release audit 0m 22s The applied patch does not increase the total number of release audit warnings.
        +1 checkstyle 3m 22s There were no new checkstyle issues.
        +1 whitespace 0m 0s The patch has no lines that end in whitespace.
        +1 install 1m 33s mvn install still works.
        +1 eclipse:eclipse 0m 32s The patch built with eclipse:eclipse.
        +1 findbugs 4m 43s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
        +1 common tests 23m 3s Tests passed in hadoop-common.
        +1 hdfs tests 162m 58s Tests passed in hadoop-hdfs.
            228m 18s  



        Subsystem Report/Notes
        Patch URL http://issues.apache.org/jira/secure/attachment/12734998/HDFS-5033.001.patch
        Optional Tests javadoc javac unit findbugs checkstyle
        git revision trunk / 446d515
        hadoop-common test log https://builds.apache.org/job/PreCommit-HDFS-Build/11114/artifact/patchprocess/testrun_hadoop-common.txt
        hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/11114/artifact/patchprocess/testrun_hadoop-hdfs.txt
        Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/11114/testReport/
        Java 1.7.0_55
        uname Linux asf909.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/11114/console

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - +1 overall Vote Subsystem Runtime Comment 0 pre-patch 14m 32s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 1 new or modified test files. +1 javac 7m 31s There were no new javac warning messages. +1 javadoc 9m 36s There were no new javadoc warning messages. +1 release audit 0m 22s The applied patch does not increase the total number of release audit warnings. +1 checkstyle 3m 22s There were no new checkstyle issues. +1 whitespace 0m 0s The patch has no lines that end in whitespace. +1 install 1m 33s mvn install still works. +1 eclipse:eclipse 0m 32s The patch built with eclipse:eclipse. +1 findbugs 4m 43s The patch does not introduce any new Findbugs (version 3.0.0) warnings. +1 common tests 23m 3s Tests passed in hadoop-common. +1 hdfs tests 162m 58s Tests passed in hadoop-hdfs.     228m 18s   Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12734998/HDFS-5033.001.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / 446d515 hadoop-common test log https://builds.apache.org/job/PreCommit-HDFS-Build/11114/artifact/patchprocess/testrun_hadoop-common.txt hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/11114/artifact/patchprocess/testrun_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/11114/testReport/ Java 1.7.0_55 uname Linux asf909.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-HDFS-Build/11114/console This message was automatically generated.
        Hide
        ajisakaa Akira Ajisaka added a comment -

        Thanks Darrell Taylor for taking this issue.
        First comment: Would you remove whitespace changes as follows from the patch?

        --- hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FileUtil.java
        +++ hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FileUtil.java
        @@ -65,7 +65,7 @@
         
           /**
            * convert an array of FileStatus to an array of Path
        -   * 
        +   *
            * @param stats
            *          an array of FileStatus objects
            * @return an array of paths corresponding to the input
        
        Show
        ajisakaa Akira Ajisaka added a comment - Thanks Darrell Taylor for taking this issue. First comment: Would you remove whitespace changes as follows from the patch? --- hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FileUtil.java +++ hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FileUtil.java @@ -65,7 +65,7 @@ /** * convert an array of FileStatus to an array of Path - * + * * @param stats * an array of FileStatus objects * @ return an array of paths corresponding to the input
        Hide
        aw Allen Wittenauer added a comment -

        I, for one, welcome the space stripping. But this patch reminds me that I need to add tab checking to whitespace. It looks like the TestDFSShell.java file is full of them.

        +1, going to commit to trunk.

        Show
        aw Allen Wittenauer added a comment - I, for one, welcome the space stripping. But this patch reminds me that I need to add tab checking to whitespace. It looks like the TestDFSShell.java file is full of them. +1, going to commit to trunk.
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-trunk-Commit #7911 (See https://builds.apache.org/job/Hadoop-trunk-Commit/7911/)
        HDFS-5033. Bad error message for fs -put/copyFromLocal if user doesn't have permissions to read the source (Darrell Taylor via aw) (aw: rev bf500d979858b084f0fe5c34a85c271a728e416b)

        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSShell.java
        • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FileUtil.java
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-trunk-Commit #7911 (See https://builds.apache.org/job/Hadoop-trunk-Commit/7911/ ) HDFS-5033 . Bad error message for fs -put/copyFromLocal if user doesn't have permissions to read the source (Darrell Taylor via aw) (aw: rev bf500d979858b084f0fe5c34a85c271a728e416b) hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSShell.java hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FileUtil.java
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #211 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/211/)
        HDFS-5033. Bad error message for fs -put/copyFromLocal if user doesn't have permissions to read the source (Darrell Taylor via aw) (aw: rev bf500d979858b084f0fe5c34a85c271a728e416b)

        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FileUtil.java
        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSShell.java
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #211 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/211/ ) HDFS-5033 . Bad error message for fs -put/copyFromLocal if user doesn't have permissions to read the source (Darrell Taylor via aw) (aw: rev bf500d979858b084f0fe5c34a85c271a728e416b) hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FileUtil.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSShell.java
        Hide
        hudson Hudson added a comment -

        SUCCESS: Integrated in Hadoop-Yarn-trunk #941 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/941/)
        HDFS-5033. Bad error message for fs -put/copyFromLocal if user doesn't have permissions to read the source (Darrell Taylor via aw) (aw: rev bf500d979858b084f0fe5c34a85c271a728e416b)

        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSShell.java
        • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FileUtil.java
        Show
        hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-Yarn-trunk #941 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/941/ ) HDFS-5033 . Bad error message for fs -put/copyFromLocal if user doesn't have permissions to read the source (Darrell Taylor via aw) (aw: rev bf500d979858b084f0fe5c34a85c271a728e416b) hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSShell.java hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FileUtil.java
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Hdfs-trunk #2139 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2139/)
        HDFS-5033. Bad error message for fs -put/copyFromLocal if user doesn't have permissions to read the source (Darrell Taylor via aw) (aw: rev bf500d979858b084f0fe5c34a85c271a728e416b)

        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSShell.java
        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FileUtil.java
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk #2139 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2139/ ) HDFS-5033 . Bad error message for fs -put/copyFromLocal if user doesn't have permissions to read the source (Darrell Taylor via aw) (aw: rev bf500d979858b084f0fe5c34a85c271a728e416b) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSShell.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FileUtil.java
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #199 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/199/)
        HDFS-5033. Bad error message for fs -put/copyFromLocal if user doesn't have permissions to read the source (Darrell Taylor via aw) (aw: rev bf500d979858b084f0fe5c34a85c271a728e416b)

        • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FileUtil.java
        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSShell.java
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #199 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/199/ ) HDFS-5033 . Bad error message for fs -put/copyFromLocal if user doesn't have permissions to read the source (Darrell Taylor via aw) (aw: rev bf500d979858b084f0fe5c34a85c271a728e416b) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FileUtil.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSShell.java
        Hide
        hudson Hudson added a comment -

        SUCCESS: Integrated in Hadoop-Mapreduce-trunk-Java8 #209 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/209/)
        HDFS-5033. Bad error message for fs -put/copyFromLocal if user doesn't have permissions to read the source (Darrell Taylor via aw) (aw: rev bf500d979858b084f0fe5c34a85c271a728e416b)

        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSShell.java
        • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FileUtil.java
        Show
        hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-Mapreduce-trunk-Java8 #209 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/209/ ) HDFS-5033 . Bad error message for fs -put/copyFromLocal if user doesn't have permissions to read the source (Darrell Taylor via aw) (aw: rev bf500d979858b084f0fe5c34a85c271a728e416b) hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSShell.java hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FileUtil.java
        Hide
        hudson Hudson added a comment -

        SUCCESS: Integrated in Hadoop-Mapreduce-trunk #2157 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2157/)
        HDFS-5033. Bad error message for fs -put/copyFromLocal if user doesn't have permissions to read the source (Darrell Taylor via aw) (aw: rev bf500d979858b084f0fe5c34a85c271a728e416b)

        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSShell.java
        • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FileUtil.java
        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        Show
        hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-Mapreduce-trunk #2157 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2157/ ) HDFS-5033 . Bad error message for fs -put/copyFromLocal if user doesn't have permissions to read the source (Darrell Taylor via aw) (aw: rev bf500d979858b084f0fe5c34a85c271a728e416b) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSShell.java hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FileUtil.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt

          People

          • Assignee:
            d4rr3ll Darrell Taylor
            Reporter:
            kasha Karthik Kambatla
          • Votes:
            0 Vote for this issue
            Watchers:
            10 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development