Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-642

Explicit timeout for ipc.Client

VotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Major
    • Resolution: Fixed
    • 0.7.2
    • 0.8.0
    • None
    • None

    Description

      This bug contributed to the crash discussed in HADOOP-572.
      ipc.Client is trying to establish connection with its server with an infinite timeout.
      For an unknown to me reason infinity equals 3 minutes in this case.
      I guess it is configured somewhere in the native socket implementation.
      With this timeout data-nodes had only 3 chances to send heartbeats during the 10
      minute expiration interval. And with a very busy name-node this makes their
      chances to be accepted close to 0.

      I included an explicit call of Socket.connect() with a timeout set to 1 min, which is
      our default for all connections.
      Modified a log message to include information that turned out to be useful for debugging.
      Removed unnecessary imports.

      Attachments

        1. IpcClientTimeout.patch
          2 kB
          Konstantin Shvachko

        Issue Links

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            shv Konstantin Shvachko
            shv Konstantin Shvachko
            Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment