Uploaded image for project: 'Pig'
  1. Pig
  2. PIG-5160

SchemaTupleFrontend.java is not thread safe, cause PigServer thrown NPE in multithread env

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Patch Available
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 0.17.0
    • Fix Version/s: None
    • Component/s: None
    • Labels:
    • Patch Info:
      Patch Available
    • Flags:
      Patch

      Description

      Calling PigServer in multi-thread environment will randomly throw NPE at

      Caused by: org.apache.pig.backend.executionengine.ExecException: ERROR 0: java.lang.NullPointerException
          at org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.launchPig(HExecutionEngine.java:310)
          at org.apache.pig.PigServer.launchPlan(PigServer.java:1390)
          at org.apache.pig.PigServer.executeCompiledLogicalPlan(PigServer.java:1375)
          at org.apache.pig.PigServer.storeEx(PigServer.java:1034)
          ... 6 more
      Caused by: java.lang.NullPointerException
          at org.apache.pig.data.SchemaTupleFrontend.registerToGenerateIfPossible(SchemaTupleFrontend.java:255)
          at org.apache.pig.newplan.logical.relational.LogToPhyTranslationVisitor.visit(LogToPhyTranslationVisitor.java:876)
          at org.apache.pig.newplan.logical.relational.LOForEach.accept(LOForEach.java:87)
          at org.apache.pig.newplan.DependencyOrderWalker.walk(DependencyOrderWalker.java:75)
          at org.apache.pig.newplan.PlanVisitor.visit(PlanVisitor.java:52)
          at org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.compile(HExecutionEngine.java:260)
          at org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.launchPig(HExecutionEngine.java:295)
          ... 9 more
      

      because SchemaTupleFrontend.java has lots of static methods which refer to static variables. Need a patch to create threadlocal variables to make it thread safe

        Attachments

        1. PIG_5160.patch
          5 kB
          Anyi Li
        2. PIG_5160-1.patch
          5 kB
          Anyi Li

          Activity

            People

            • Assignee:
              anyili Anyi Li
              Reporter:
              anyili Anyi Li
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated: