Uploaded image for project: 'Apache Knox'
  1. Apache Knox
  2. KNOX-797

Default Topology Feature Not Working

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: Server
    • Labels:
      None

      Description

      From Mohammad Islam:

      I set "default.app.topology.name" in gateway-site.xml to "uber" (my default topology name).
      It worked fine if I gave the full URL. The command looks like this "curl http://<KNOX_GW_URL>/gateway/uber/webhdfs/v1/?op=GETHOMEDIRECTORY'".

      However, when I tried with command "curl http://<KNOX_GW_URL>/webhdfs/v1/?op=GETHOMEDIRECTORY'". I got the HTTP error code 500. I looked into gateway.log file and found quite a few error related to rewrite. The exact error messages are shown below:

      Error message

      2016-11-30 00:39:51,565 ERROR hadoop.gateway (UrlRewriteProcessor.java:rewrite(169)) - Failed to rewrite URL: http://<KNOX_GW_URL>/webhdfs/v1/?op=GETHOMEDIRECTORY, direction: IN via rule: WEBHDFS/webhdfs/inbound/namenode/root, status: FAILURE
      2016-11-30 00:39:51,565 ERROR hadoop.gateway (UrlRewriteProcessor.java:rewrite(169)) - Failed to rewrite URL: http://<KNOX_GW_URL>/webhdfs/v1/?op=GETHOMEDIRECTORY, direction: IN via rule: WEBHDFS/webhdfs/inbound/namenode/root, status: FAILURE
      

      After that, I modified the webhdfs/2.4.0/rewrite.xml by rewriting the following pattern and it worked for short URL but long URL faces the same issue.

      Original:

      <rule dir="IN" name="WEBHDFS/webhdfs/inbound/namenode/root" pattern="*://*:*/**/webhdfs/{version}/?{**}">
              <rewrite template="{$serviceUrl[WEBHDFS]}/{version}/?{**}"/>
          </rule>
      

      Modified :

          <rule dir="IN" name="WEBHDFS/webhdfs/inbound/namenode/root" pattern="*://*:*/webhdfs/{version}/?{**}">
              <rewrite template="{$serviceUrl[WEBHDFS]}/{version}/?{**}"/>
          </rule>
      

      Overall, the rewrite pattern may be the issue. We will need to support for both short and long URL. May be, we can add multiple rewrite rules for each route in service.xml.
      Is there any other cleaner way which may work for all cases such as webhdfs, yarn, hive, UIs etc?

        Attachments

          Activity

            People

            • Assignee:
              lmccay Larry McCay
              Reporter:
              lmccay Larry McCay
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: