Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-6156

Fetcher - connect() doesn't handle connection refused correctly

VotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Blocker
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.6.0
    • Component/s: None
    • Labels:
      None
    • Target Version/s:
    • Hadoop Flags:
      Reviewed

      Description

      The connect() function in the fetcher assumes that whenever an IOException is thrown, the amount of time passed equals "connectionTimeout" ( see code snippet below ). This is incorrect. For example, in case the NM is down, an ConnectException is thrown immediately - and the catch block assumes a minute has passed when it is not the case.

        if (connectionTimeout < 0) {
            throw new IOException("Invalid timeout "
                                  + "[timeout = " + connectionTimeout + " ms]");
          } else if (connectionTimeout > 0) {
            unit = Math.min(UNIT_CONNECT_TIMEOUT, connectionTimeout);
          }
          // set the connect timeout to the unit-connect-timeout
          connection.setConnectTimeout(unit);
          while (true) {
            try {
              connection.connect();
              break;
            } catch (IOException ioe) {
              // update the total remaining connect-timeout
              connectionTimeout -= unit;
      
              // throw an exception if we have waited for timeout amount of time
              // note that the updated value if timeout is used here
              if (connectionTimeout == 0) {
                throw ioe;
              }
      
              // reset the connect timeout for the last try
              if (connectionTimeout < unit) {
                unit = connectionTimeout;
                // reset the connect time out for the final connect
                connection.setConnectTimeout(unit);
              }
            }
          }
      

        Attachments

        1. MAPREDUCE-6156-v3.patch
          3 kB
          Junping Du
        2. MAPREDUCE-6156-v2.patch
          3 kB
          Junping Du
        3. MAPREDUCE-6156.patch
          2 kB
          Junping Du

        Issue Links

          Activity

            People

            • Assignee:
              junping_du Junping Du
              Reporter:
              sidharta-s Sidharta Seethana

              Dates

              • Created:
                Updated:
                Resolved:

                Issue deployment