Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-6156

Fetcher - connect() doesn't handle connection refused correctly

VotersStop watchingWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Blocker
    • Resolution: Fixed
    • None
    • 2.6.0
    • None
    • None
    • Reviewed

    Description

      The connect() function in the fetcher assumes that whenever an IOException is thrown, the amount of time passed equals "connectionTimeout" ( see code snippet below ). This is incorrect. For example, in case the NM is down, an ConnectException is thrown immediately - and the catch block assumes a minute has passed when it is not the case.

        if (connectionTimeout < 0) {
            throw new IOException("Invalid timeout "
                                  + "[timeout = " + connectionTimeout + " ms]");
          } else if (connectionTimeout > 0) {
            unit = Math.min(UNIT_CONNECT_TIMEOUT, connectionTimeout);
          }
          // set the connect timeout to the unit-connect-timeout
          connection.setConnectTimeout(unit);
          while (true) {
            try {
              connection.connect();
              break;
            } catch (IOException ioe) {
              // update the total remaining connect-timeout
              connectionTimeout -= unit;
      
              // throw an exception if we have waited for timeout amount of time
              // note that the updated value if timeout is used here
              if (connectionTimeout == 0) {
                throw ioe;
              }
      
              // reset the connect timeout for the last try
              if (connectionTimeout < unit) {
                unit = connectionTimeout;
                // reset the connect time out for the final connect
                connection.setConnectTimeout(unit);
              }
            }
          }
      

      Attachments

        1. MAPREDUCE-6156-v3.patch
          3 kB
          Junping Du
        2. MAPREDUCE-6156-v2.patch
          3 kB
          Junping Du
        3. MAPREDUCE-6156.patch
          2 kB
          Junping Du

        Issue Links

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            junping_du Junping Du
            sidharta-s Sidharta Seethana
            Votes:
            0 Vote for this issue
            Watchers:
            11 Stop watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment