Uploaded image for project: 'Sqoop'
  1. Sqoop
  2. SQOOP-1300

ToolRunner is not thread-safe

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Critical
    • Resolution: Done
    • Affects Version/s: None
    • Fix Version/s: 2.0.0
    • Component/s: tools
    • Labels:

      Description

      ToolRunner class is not thread-safe because it uses GenericOptionsParser. The constructor of GenericOptionsParser uses 'OptionBuilder' which is a singleton class that uses instance variables. In other words, OptionBuilder is NOT thread safe. As a result, when multiple Hadoop jobs are triggered simultaneously using ToolRunner they end up stepping on each other.

      The easiest way to fix it is by making 'buildGeneralOptions' synchronized in GenericOptionsParser.

      private static synchronized Options buildGeneralOptions(Options opts) {

      If this seems like a correct way of fixing this, either we can provide a patch or someone can quickly fix it. Thanks.

      • Ajay Chitre
        achitre@cisco.com

      Virendra Singh
      virsingh@cisco.com

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              achitre Ajay Chitre
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: