Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-6156

Fetcher - connect() doesn't handle connection refused correctly

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Blocker
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.6.0
    • Component/s: None
    • Labels:
      None
    • Target Version/s:
    • Hadoop Flags:
      Reviewed

      Description

      The connect() function in the fetcher assumes that whenever an IOException is thrown, the amount of time passed equals "connectionTimeout" ( see code snippet below ). This is incorrect. For example, in case the NM is down, an ConnectException is thrown immediately - and the catch block assumes a minute has passed when it is not the case.

        if (connectionTimeout < 0) {
            throw new IOException("Invalid timeout "
                                  + "[timeout = " + connectionTimeout + " ms]");
          } else if (connectionTimeout > 0) {
            unit = Math.min(UNIT_CONNECT_TIMEOUT, connectionTimeout);
          }
          // set the connect timeout to the unit-connect-timeout
          connection.setConnectTimeout(unit);
          while (true) {
            try {
              connection.connect();
              break;
            } catch (IOException ioe) {
              // update the total remaining connect-timeout
              connectionTimeout -= unit;
      
              // throw an exception if we have waited for timeout amount of time
              // note that the updated value if timeout is used here
              if (connectionTimeout == 0) {
                throw ioe;
              }
      
              // reset the connect timeout for the last try
              if (connectionTimeout < unit) {
                unit = connectionTimeout;
                // reset the connect time out for the final connect
                connection.setConnectTimeout(unit);
              }
            }
          }
      

        Attachments

        1. MAPREDUCE-6156.patch
          2 kB
          Junping Du
        2. MAPREDUCE-6156-v2.patch
          3 kB
          Junping Du
        3. MAPREDUCE-6156-v3.patch
          3 kB
          Junping Du

          Issue Links

            Activity

              People

              • Assignee:
                djp Junping Du
                Reporter:
                sidharta-s Sidharta Seethana
              • Votes:
                0 Vote for this issue
                Watchers:
                11 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: