Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-13388

RequestHedgingProxyProvider calls multiple configured NNs all the time

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 3.2.0, 3.1.1, 3.0.3
    • Component/s: hdfs-client
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      In HDFS-7858 RequestHedgingProxyProvider was designed to "first simultaneously call multiple configured NNs to decide which is the active Namenode and then for subsequent calls it will invoke the previously successful NN ." But the current code call multiple configured NNs every time even when we already got the successful NN.
      That's because in RetryInvocationHandler.java, ProxyDescriptor's member proxyInfo is assigned only when it is constructed or when failover occurs. RequestHedgingProxyProvider.currentUsedProxy is null in both cases, so the only proxy we can get is always a dynamic proxy handled by RequestHedgingInvocationHandler.class. RequestHedgingInvocationHandler.class handles invoked method by calling multiple configured NNs.

        Attachments

        1. HADOOP-13388.0014.patch
          5 kB
          Jinglun
        2. HADOOP-13388.0013.patch
          5 kB
          Jinglun
        3. HADOOP-13388.0012.patch
          5 kB
          Jinglun
        4. HADOOP-13388.0011.patch
          5 kB
          Jinglun
        5. HADOOP-13388.0010.patch
          5 kB
          Íñigo Goiri
        6. HADOOP-13388.0009.patch
          5 kB
          Jinglun
        7. HADOOP-13388.0008.patch
          4 kB
          Jinglun
        8. HADOOP-13388.0007.patch
          1 kB
          Jinglun
        9. HADOOP-13388.0006.patch
          3 kB
          Jinglun
        10. HADOOP-13388.0005.patch
          3 kB
          Jinglun
        11. HADOOP-13388.0004.patch
          3 kB
          Jinglun
        12. HADOOP-13388.0003.patch
          3 kB
          Jinglun
        13. HADOOP-13388.0002.patch
          27 kB
          Jinglun
        14. HADOOP-13388.0001.patch
          3 kB
          Jinglun

          Issue Links

            Activity

              People

              • Assignee:
                LiJinglun Jinglun
                Reporter:
                LiJinglun Jinglun
              • Votes:
                0 Vote for this issue
                Watchers:
                9 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: