Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-11391

Numeric usernames do no work with WebHDFS FS (write access)

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.7.3
    • Fix Version/s: 2.9.0, 3.0.0-alpha4, 2.8.2
    • Component/s: webhdfs
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      In HDFS-4983, a property has been introduced to configure the pattern validating name of users interacting with WebHDFS because default pattern was excluding names starting with numbers.

      Problem is that this fix works only for read access. In case of write access against data node, the default pattern is still applied whatever the configuration is.

        Issue Links

          Activity

          Hide
          vinodkv Vinod Kumar Vavilapalli added a comment -

          2.8.1 became a security release. Moving fix-version to 2.8.2 after the fact.

          Show
          vinodkv Vinod Kumar Vavilapalli added a comment - 2.8.1 became a security release. Moving fix-version to 2.8.2 after the fact.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user pvillard31 closed the pull request at:

          https://github.com/apache/hadoop/pull/186

          Show
          githubbot ASF GitHub Bot added a comment - Github user pvillard31 closed the pull request at: https://github.com/apache/hadoop/pull/186
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user pvillard31 commented on the issue:

          https://github.com/apache/hadoop/pull/186

          Committed as part of 8e53f2b9b08560bf4f8e81e697063277dbdc68f9. Closing.

          Show
          githubbot ASF GitHub Bot added a comment - Github user pvillard31 commented on the issue: https://github.com/apache/hadoop/pull/186 Committed as part of 8e53f2b9b08560bf4f8e81e697063277dbdc68f9. Closing.
          Hide
          yzhangal Yongjun Zhang added a comment - - edited

          Committed to trunk, branch-2 and branch-2.8.

          Thanks Pierre Villard much for the contribution!

          Show
          yzhangal Yongjun Zhang added a comment - - edited Committed to trunk, branch-2 and branch-2.8. Thanks Pierre Villard much for the contribution!
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Jenkins build Hadoop-trunk-Commit #11248 (See https://builds.apache.org/job/Hadoop-trunk-Commit/11248/)
          HDFS-11391. Numeric usernames do no work with WebHDFS FS write access. (yzhang: rev 8e53f2b9b08560bf4f8e81e697063277dbdc68f9)

          • (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/web/webhdfs/WebHdfsHandler.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Jenkins build Hadoop-trunk-Commit #11248 (See https://builds.apache.org/job/Hadoop-trunk-Commit/11248/ ) HDFS-11391 . Numeric usernames do no work with WebHDFS FS write access. (yzhang: rev 8e53f2b9b08560bf4f8e81e697063277dbdc68f9) (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/web/webhdfs/WebHdfsHandler.java
          Hide
          yzhangal Yongjun Zhang added a comment -

          Thanks Pierre Villard. I'm +1 on the patch, will commit soon.

          Show
          yzhangal Yongjun Zhang added a comment - Thanks Pierre Villard . I'm +1 on the patch, will commit soon.
          Hide
          pvillard Pierre Villard added a comment -

          Yongjun Zhang, anything else required on my side to get the PR reviewed/merged?

          Show
          pvillard Pierre Villard added a comment - Yongjun Zhang , anything else required on my side to get the PR reviewed/merged?
          Hide
          pvillard Pierre Villard added a comment -

          Hi Yongjun Zhang,

          No problem, I updated the PR.

          Regarding the tests:

          • Before patch (with property modified to allow numerical user names)
          $ curl -i -X PUT "http://mynode:50070/webhdfs/v1/tmp/test.txt?op=CREATE&user.name=123"
          HTTP/1.1 307 TEMPORARY_REDIRECT
          Cache-Control: no-cache
          Expires: Sat, 04 Feb 2017 10:19:38 GMT
          Date: Sat, 04 Feb 2017 10:19:38 GMT
          Pragma: no-cache
          Expires: Sat, 04 Feb 2017 10:19:38 GMT
          Date: Sat, 04 Feb 2017 10:19:38 GMT
          Pragma: no-cache
          X-FRAME-OPTIONS: SAMEORIGIN
          Set-Cookie: hadoop.auth="u=123&p=123&t=simple&e=1486239578624&s=UrzCjP0SPpPKDJnSYB5BsKuQVKc="; Path=/; HttpOnly
          Location: http://mynode:50075/webhdfs/v1/tmp/test.txt?op=CREATE&user.name=123&namenoderpcaddress=mynode:8020&createflag=&createparent=true&overwrite=false
          Content-Type: application/octet-stream
          Content-Length: 0
          
          $ curl -i -X PUT -T test.txt "http://mynode:50075/webhdfs/v1/tmp/test.txt?op=CREATE&user.name=123&namenoderpcaddress=mynode:8020&createflag=&createparent=true&overwrite=false"
          HTTP/1.1 400 Bad Request
          Content-Type: application/json; charset=utf-8
          Content-Length: 209
          Connection: close
          
          {"RemoteException":{"exception":"IllegalArgumentException","javaClassName":"java.lang.IllegalArgumentException","message":"Invalid value: \"123\" does not belong to the domain ^[A-Za-z_][A-Za-z0-9._-]*[$]?$"}}
          
          • After patch (with property modified to allow numerical user names)
          $ curl -i -X PUT "http://mynode:50070/webhdfs/v1/tmp/test.txt?op=CREATE&user.name=123"
          HTTP/1.1 307 TEMPORARY_REDIRECT
          Cache-Control: no-cache
          Expires: Sat, 04 Feb 2017 20:25:15 GMT
          Date: Sat, 04 Feb 2017 20:25:15 GMT
          Pragma: no-cache
          Expires: Sat, 04 Feb 2017 20:25:15 GMT
          Date: Sat, 04 Feb 2017 20:25:15 GMT
          Pragma: no-cache
          X-FRAME-OPTIONS: SAMEORIGIN
          Set-Cookie: hadoop.auth="u=123&p=123&t=simple&e=1486275915563&s=te9ylMEmTuFswBr2sK9kH6qj8eE="; Path=/; HttpOnly
          Location: http://mynode:50075/webhdfs/v1/tmp/test.txt?op=CREATE&user.name=123&namenoderpcaddress=mynode:8020&createflag=&createparent=true&overwrite=false
          Content-Type: application/octet-stream
          Content-Length: 0
          
          $ curl -i -X PUT -T test.txt "http://mynode:50075/webhdfs/v1/tmp/test.txt?op=CREATE&user.name=123&namenoderpcaddress=mynode:8020&createflag=&createparent=true&overwrite=false"
          HTTP/1.1 100 Continue
          
          HTTP/1.1 201 Created
          Location: hdfs://mynode:8020/tmp/test.txt
          Content-Length: 0
          Connection: close
          

          Let me know if you need something else.

          Show
          pvillard Pierre Villard added a comment - Hi Yongjun Zhang , No problem, I updated the PR. Regarding the tests: Before patch (with property modified to allow numerical user names) $ curl -i -X PUT "http://mynode:50070/webhdfs/v1/tmp/test.txt?op=CREATE&user.name=123" HTTP/1.1 307 TEMPORARY_REDIRECT Cache-Control: no-cache Expires: Sat, 04 Feb 2017 10:19:38 GMT Date: Sat, 04 Feb 2017 10:19:38 GMT Pragma: no-cache Expires: Sat, 04 Feb 2017 10:19:38 GMT Date: Sat, 04 Feb 2017 10:19:38 GMT Pragma: no-cache X-FRAME-OPTIONS: SAMEORIGIN Set-Cookie: hadoop.auth="u=123&p=123&t=simple&e=1486239578624&s=UrzCjP0SPpPKDJnSYB5BsKuQVKc="; Path=/; HttpOnly Location: http://mynode:50075/webhdfs/v1/tmp/test.txt?op=CREATE&user.name=123&namenoderpcaddress=mynode:8020&createflag=&createparent=true&overwrite=false Content-Type: application/octet-stream Content-Length: 0 $ curl -i -X PUT -T test.txt "http://mynode:50075/webhdfs/v1/tmp/test.txt?op=CREATE&user.name=123&namenoderpcaddress=mynode:8020&createflag=&createparent=true&overwrite=false" HTTP/1.1 400 Bad Request Content-Type: application/json; charset=utf-8 Content-Length: 209 Connection: close {"RemoteException":{"exception":"IllegalArgumentException","javaClassName":"java.lang.IllegalArgumentException","message":"Invalid value: \"123\" does not belong to the domain ^[A-Za-z_][A-Za-z0-9._-]*[$]?$"}} After patch (with property modified to allow numerical user names) $ curl -i -X PUT "http://mynode:50070/webhdfs/v1/tmp/test.txt?op=CREATE&user.name=123" HTTP/1.1 307 TEMPORARY_REDIRECT Cache-Control: no-cache Expires: Sat, 04 Feb 2017 20:25:15 GMT Date: Sat, 04 Feb 2017 20:25:15 GMT Pragma: no-cache Expires: Sat, 04 Feb 2017 20:25:15 GMT Date: Sat, 04 Feb 2017 20:25:15 GMT Pragma: no-cache X-FRAME-OPTIONS: SAMEORIGIN Set-Cookie: hadoop.auth="u=123&p=123&t=simple&e=1486275915563&s=te9ylMEmTuFswBr2sK9kH6qj8eE="; Path=/; HttpOnly Location: http://mynode:50075/webhdfs/v1/tmp/test.txt?op=CREATE&user.name=123&namenoderpcaddress=mynode:8020&createflag=&createparent=true&overwrite=false Content-Type: application/octet-stream Content-Length: 0 $ curl -i -X PUT -T test.txt "http://mynode:50075/webhdfs/v1/tmp/test.txt?op=CREATE&user.name=123&namenoderpcaddress=mynode:8020&createflag=&createparent=true&overwrite=false" HTTP/1.1 100 Continue HTTP/1.1 201 Created Location: hdfs://mynode:8020/tmp/test.txt Content-Length: 0 Connection: close Let me know if you need something else.
          Hide
          yzhangal Yongjun Zhang added a comment -

          Hi Pierre Villard,

          Thanks for reporting the issue and patch. The patch looks good except line 114 is too long (expecte to be <= 80 chars).

              UserParam.setUserPattern(
                  conf.get(DFSConfigKeys.DFS_WEBHDFS_USER_PATTERN_KEY,
                      DFSConfigKeys.DFS_WEBHDFS_USER_PATTERN_DEFAULT));
          

          BTW, would you please share what test you have done?

          Thanks.

          Show
          yzhangal Yongjun Zhang added a comment - Hi Pierre Villard , Thanks for reporting the issue and patch. The patch looks good except line 114 is too long (expecte to be <= 80 chars). UserParam.setUserPattern( conf.get(DFSConfigKeys.DFS_WEBHDFS_USER_PATTERN_KEY, DFSConfigKeys.DFS_WEBHDFS_USER_PATTERN_DEFAULT)); BTW, would you please share what test you have done? Thanks.
          Hide
          githubbot ASF GitHub Bot added a comment -

          GitHub user pvillard31 opened a pull request:

          https://github.com/apache/hadoop/pull/186

          HDFS-11391 Numeric usernames do no work with WebHDFS FS

          In HDFS-4983, a property has been introduced to configure the pattern
          validating name of users interacting with WebHDFS because default
          pattern was excluding names starting with numbers.

          Problem is that this fix works only for read access. In case of write
          access against data node, the default pattern is still applied whatever
          the configuration is.

          This PR fixes the web handler running in data nodes.

          You can merge this pull request into a Git repository by running:

          $ git pull https://github.com/pvillard31/hadoop webhdfs-datanode-username-pattern

          Alternatively you can review and apply these changes as the patch at:

          https://github.com/apache/hadoop/pull/186.patch

          To close this pull request, make a commit to your master/trunk branch
          with (at least) the following in the commit message:

          This closes #186


          commit e09338f3c181da8afa2ad009f6925260d0a778b6
          Author: Pierre Villard <pierre.villard.fr@gmail.com>
          Date: 2017-02-04T21:57:26Z

          HDFS-11391 Numeric usernames do no work with WebHDFS FS

          In HDFS-4983, a property has been introduced to configure the pattern
          validating name of users interacting with WebHDFS because default
          pattern was excluding names starting with numbers.

          Problem is that this fix works only for read access. In case of write
          access against data node, the default pattern is still applied whatever
          the configuration is.

          This PR fixes the web handler running in data nodes.


          Show
          githubbot ASF GitHub Bot added a comment - GitHub user pvillard31 opened a pull request: https://github.com/apache/hadoop/pull/186 HDFS-11391 Numeric usernames do no work with WebHDFS FS In HDFS-4983 , a property has been introduced to configure the pattern validating name of users interacting with WebHDFS because default pattern was excluding names starting with numbers. Problem is that this fix works only for read access. In case of write access against data node, the default pattern is still applied whatever the configuration is. This PR fixes the web handler running in data nodes. You can merge this pull request into a Git repository by running: $ git pull https://github.com/pvillard31/hadoop webhdfs-datanode-username-pattern Alternatively you can review and apply these changes as the patch at: https://github.com/apache/hadoop/pull/186.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #186 commit e09338f3c181da8afa2ad009f6925260d0a778b6 Author: Pierre Villard <pierre.villard.fr@gmail.com> Date: 2017-02-04T21:57:26Z HDFS-11391 Numeric usernames do no work with WebHDFS FS In HDFS-4983 , a property has been introduced to configure the pattern validating name of users interacting with WebHDFS because default pattern was excluding names starting with numbers. Problem is that this fix works only for read access. In case of write access against data node, the default pattern is still applied whatever the configuration is. This PR fixes the web handler running in data nodes.

            People

            • Assignee:
              pvillard Pierre Villard
              Reporter:
              pvillard Pierre Villard
            • Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development