Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-1623 High Availability Framework for HDFS NN
  3. HDFS-2683

Authority-based lookup of proxy provider fails if path becomes canonicalized

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Resolved
    • Critical
    • Resolution: Fixed
    • HA branch (HDFS-1623)
    • HA branch (HDFS-1623)
    • ha, hdfs-client
    • None

    Description

      When testing MapReduce on top of an HA cluster we ran into the following bug: some uses of HDFS paths go through a canonicalization step which ensures that the authority component in the URI includes a port number. So our hdfs://logical-nn-uri/foo path turned into hdfs://logical-nn-uri:8020/foo. The code which looks up the failover proxy provider then failed to find the associated config. We should only compare the hostname portion of the URI when looking up proxy providers.

      Attachments

        1. hdfs-2683.txt
          6 kB
          Todd Lipcon
        2. hdfs-2683.txt
          7 kB
          Todd Lipcon

        Activity

          People

            tlipcon Todd Lipcon
            tlipcon Todd Lipcon
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: