Uploaded image for project: 'Pig'
  1. Pig
  2. PIG-4845

Parallel instantiation of classes in Tez cause tasks to fail

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.16.0
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      OptionBuilder is not thread-safe

      https://commons.apache.org/proper/commons-cli/javadocs/api-release/org/apache/commons/cli/OptionBuilder.html

      This class is NOT thread safe. See CLI-209
      

      HBaseStorage, elephantbird SequenceFileConfig, etc use it in their constructor. This leads to NoSuchMethodException, UnrecognizedOptionException etc when processor, inputs and outputs are initialized in parallel in Tez making the task fail. Retry attempts mostly go through and job might succeed. Need to synchronize the initialization. synchronize would make it similar to mapreduce though at a slight performance cost. But that should be lot better than cost of relaunching containers after hitting failures.

      Following two kinds of exception where seen when HBaseStorage and SequenceFileStorage of elephantbird were used together.

      org.apache.commons.cli.UnrecognizedOptionException: Unrecognized option: -loadKey

      Caused by: java.lang.RuntimeException: Failed to create WritableConverter instance
              at com.twitter.elephantbird.pig.util.SequenceFileConfig.getWritableConverter(SequenceFileConfig.java:225)
              at com.twitter.elephantbird.pig.util.SequenceFileConfig.<init>(SequenceFileConfig.java:101)
              at com.twitter.elephantbird.pig.util.SequenceFileConfig.<init>(SequenceFileConfig.java:115)
              ... 11 more
      Caused by: java.lang.NoSuchMethodException: com.twitter.elephantbird.pig.util.TextConverter.<init>(java.lang.String)
              at java.lang.Class.getConstructor0(Class.java:3082)
              at java.lang.Class.getConstructor(Class.java:1825)
              at com.twitter.elephantbird.pig.util.SequenceFileConfig.getWritableConverter(SequenceFileConfig.java:213)
              ... 13 more
      

        Attachments

        1. PIG-4845-1.patch
          4 kB
          Rohini Palaniswamy

          Activity

            People

            • Assignee:
              rohini Rohini Palaniswamy
              Reporter:
              rohini Rohini Palaniswamy
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: