Uploaded image for project: 'Apache Knox'
  1. Apache Knox
  2. KNOX-1440

WEBHDFS: retry should not be attempted for datanode write failures

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Closed
    • Major
    • Resolution: Fixed
    • 1.1.0
    • 1.2.0
    • Server
    • None

    Description

      For WEBHDFS invocations through Knox, if a datanode write fails with an IOException, the currently configured HA dispatch (DefaultHaDispatch) attempts the invocation the configured number of times.

      HDFS dev says that it would be appropriate to more simply respond with the error, deferring the decision to retry to the client.

      The best resolution for this is to have the datanode interactions use the DefaultDispatch (rather than the DefaultHaDispatch) even when HA is enabled, such that no retry/failover will be attempted for those interactions.

      (FYI: the datanode dispatch is configured in the WEBHDFS service definition)

      Attachments

        Activity

          People

            pzampino Philip Zampino
            pzampino Philip Zampino
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: