Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Won't Fix
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None

      Description

      Replacing the topology script requires a namenode bounce because the NN caches the information permanently. It should really either expire it periodically or expire on -refreshNodes.

        Activity

        Hide
        dhruba borthakur added a comment -

        Does your network topology change so frequently?

        Show
        dhruba borthakur added a comment - Does your network topology change so frequently?
        Hide
        Allen Wittenauer added a comment -

        No, but there should be a way to flush it when it does other than restart (== grid down time) when it does, esp. considering we can dynamically add/remove nodes (to some degree).

        Show
        Allen Wittenauer added a comment - No, but there should be a way to flush it when it does other than restart (== grid down time) when it does, esp. considering we can dynamically add/remove nodes (to some degree).
        Hide
        Koji Noguchi added a comment -

        If we're going to add this, please make it optional.
        Without HADOOP:5059, Namenode would hang 2-3 seconds for each expired/new datanode when it calls topology script in our environment.
        (or you can use java topology code)

        It's ok at start up time since heap is relatively small and namenode is usually not out of the safemode.

        Show
        Koji Noguchi added a comment - If we're going to add this, please make it optional. Without HADOOP:5059, Namenode would hang 2-3 seconds for each expired/new datanode when it calls topology script in our environment. (or you can use java topology code) It's ok at start up time since heap is relatively small and namenode is usually not out of the safemode.
        Hide
        dhruba borthakur added a comment -

        Hi Koji, we do not use a external script to map Ip address to network topology. Instead we have java code to map IP address to rack location. Don't you folks have jave code too?

        Show
        dhruba borthakur added a comment - Hi Koji, we do not use a external script to map Ip address to network topology. Instead we have java code to map IP address to rack location. Don't you folks have jave code too?
        Hide
        dhruba borthakur added a comment -

        Just for the records, we use topology script written in Java to avoid the problem listed in HADOOP-5059

        Show
        dhruba borthakur added a comment - Just for the records, we use topology script written in Java to avoid the problem listed in HADOOP-5059
        Hide
        Allen Wittenauer added a comment -

        Sites should implement their own Java class that duplicates the current script method but implements caching. Closing as won't fix.

        Show
        Allen Wittenauer added a comment - Sites should implement their own Java class that duplicates the current script method but implements caching. Closing as won't fix.

          People

          • Assignee:
            Unassigned
            Reporter:
            Allen Wittenauer
          • Votes:
            0 Vote for this issue
            Watchers:
            7 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development