Hadoop HDFS
  1. Hadoop HDFS
  2. HDFS-1623 High Availability Framework for HDFS NN
  3. HDFS-2683

Authority-based lookup of proxy provider fails if path becomes canonicalized

    Details

    • Type: Sub-task Sub-task
    • Status: Resolved
    • Priority: Critical Critical
    • Resolution: Fixed
    • Affects Version/s: HA branch (HDFS-1623)
    • Fix Version/s: HA branch (HDFS-1623)
    • Component/s: ha, hdfs-client
    • Labels:
      None

      Description

      When testing MapReduce on top of an HA cluster we ran into the following bug: some uses of HDFS paths go through a canonicalization step which ensures that the authority component in the URI includes a port number. So our hdfs://logical-nn-uri/foo path turned into hdfs://logical-nn-uri:8020/foo. The code which looks up the failover proxy provider then failed to find the associated config. We should only compare the hostname portion of the URI when looking up proxy providers.

      1. hdfs-2683.txt
        6 kB
        Todd Lipcon
      2. hdfs-2683.txt
        7 kB
        Todd Lipcon

        Activity

        Todd Lipcon made changes -
        Status Open [ 1 ] Resolved [ 5 ]
        Hadoop Flags Reviewed [ 10343 ]
        Fix Version/s HA branch (HDFS-1623) [ 12317568 ]
        Resolution Fixed [ 1 ]
        Todd Lipcon made changes -
        Attachment hdfs-2683.txt [ 12507461 ]
        Todd Lipcon made changes -
        Field Original Value New Value
        Attachment hdfs-2683.txt [ 12507459 ]
        Todd Lipcon created issue -

          People

          • Assignee:
            Todd Lipcon
            Reporter:
            Todd Lipcon
          • Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development