Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-1623 High Availability Framework for HDFS NN
  3. HDFS-2683

Authority-based lookup of proxy provider fails if path becomes canonicalized

    XMLWordPrintableJSON

    Details

    • Type: Sub-task
    • Status: Resolved
    • Priority: Critical
    • Resolution: Fixed
    • Affects Version/s: HA branch (HDFS-1623)
    • Fix Version/s: HA branch (HDFS-1623)
    • Component/s: ha, hdfs-client
    • Labels:
      None

      Description

      When testing MapReduce on top of an HA cluster we ran into the following bug: some uses of HDFS paths go through a canonicalization step which ensures that the authority component in the URI includes a port number. So our hdfs://logical-nn-uri/foo path turned into hdfs://logical-nn-uri:8020/foo. The code which looks up the failover proxy provider then failed to find the associated config. We should only compare the hostname portion of the URI when looking up proxy providers.

        Attachments

        1. hdfs-2683.txt
          6 kB
          Todd Lipcon
        2. hdfs-2683.txt
          7 kB
          Todd Lipcon

          Activity

            People

            • Assignee:
              tlipcon Todd Lipcon
              Reporter:
              tlipcon Todd Lipcon
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: