Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-237

Better handling of dfsadmin command when namenode is slow

    Details

    • Type: New Feature
    • Status: Resolved
    • Priority: Major
    • Resolution: Later
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None

      Description

      Probably when hitting HADOOP-3810, Namenode became unresponsive. Large time spent in GC.
      All dfs/dfsadmin command were timing out.

      WebUI was coming up after waiting for a long time.

      Maybe setting a long timeout would have made the dfsadmin command go through.
      But it would be nice to have a separate queue/handler which doesn't compete with regular rpc calls.

      All I wanted to do was dfsadmin -safemode enter, dfsadmin -finalizeUpgrade ...

        Issue Links

          Activity

          Hide
          qwertymaniac Harsh J added a comment -

          This older JIRA is a bit stale given the multiple changes that went into the RPC side. Follow HADOOP-9640 and related JIRAs instead for more recent work.

          a separate rpc queue

          This is supported today via the servicerpc-address configs (typically set to 8022, and strongly recommended for HA modes).

          Show
          qwertymaniac Harsh J added a comment - This older JIRA is a bit stale given the multiple changes that went into the RPC side. Follow HADOOP-9640 and related JIRAs instead for more recent work. a separate rpc queue This is supported today via the servicerpc-address configs (typically set to 8022, and strongly recommended for HA modes).
          Hide
          aw Allen Wittenauer added a comment -

          QoS might help here.

          Show
          aw Allen Wittenauer added a comment - QoS might help here.
          Hide
          rangadi Raghu Angadi added a comment -

          This is mainly for emergencies. Simpler thing might be for NN to provide webui for these commands so that it bypasses RPC traffic.

          Show
          rangadi Raghu Angadi added a comment - This is mainly for emergencies. Simpler thing might be for NN to provide webui for these commands so that it bypasses RPC traffic.

            People

            • Assignee:
              Unassigned
              Reporter:
              knoguchi Koji Noguchi
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development