Uploaded image for project: 'Sqoop'
  1. Sqoop
  2. SQOOP-2343

AsyncSqlRecordWriter stucks if any exception is thrown out in its close method

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 1.4.5
    • Fix Version/s: 1.4.7
    • Component/s: connectors
    • Labels:
      None

      Description

      In class AsyncSqlRecordWriter, if any exception is thrown in its close method, the Hadoop MapTask will call this close method once more in case it hasn't been closed. Please see below code snippet (in method runNewMapper):

          try {
            input.initialize(split, mapperContext);
            mapper.run(mapperContext);
            mapPhase.complete();
            setPhase(TaskStatus.Phase.SORT);
            statusUpdate(umbilical);
            input.close();
            input = null;
            output.close(mapperContext);
            output = null;
          } finally {
            closeQuietly(input);
            closeQuietly(output, mapperContext);
          }
      

      The second time the close method is called, the main thread will stuck in executeUpdate when trying to put a new dbOp into the synchronous queue, because at this moment the worker thread has ended and thus not receiver will take that object, which makes the putter (main thread) stuck.

        Attachments

          Activity

            People

            • Assignee:
              Yibing Yibing Shi
              Reporter:
              Yibing Yibing Shi
            • Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: