Pig
  1. Pig
  2. PIG-4139

pig query throws error java.lang.NoSuchFieldException: jobsInProgress on MRv1

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.12.0, 0.11.1
    • Fix Version/s: 0.14.0, 0.13.1
    • Component/s: None
    • Labels:
      None
    • Environment:

      Pig jar running on yarn configured to run MRv1

    • Patch Info:
      Patch Available

      Description

      The following exception is caught and printed on stdout with logging level WARN under the environment mentioned above. Its a harmless msg and the pig job falls back to using HadoopShoms 0.20 API and completes successfully. But the exception can be confusing to the user. Increasing logging level to debug in the patch.


      2014-08-20 11:12:00,879 [main] WARN
      org.apache.pig.backend.hadoop23.PigJobControl - falling back to default
      JobControl (not using hadoop 0.23 ?)
      java.lang.NoSuchFieldException: jobsInProgress
      at java.lang.Class.getDeclaredField(Class.java:1899)
      at
      org.apache.pig.backend.hadoop23.PigJobControl.<clinit>(PigJobControl.java:58)
      at
      org.apache.pig.backend.hadoop.executionengine.shims.HadoopShims.newJobControl(HadoopShims.java:104)
      at
      org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler.compile(JobControlCompiler.java:287)
      at
      org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher.launchPig(MapReduceLauncher.java:192)
      at org.apache.pig.PigServer.launchPlan(PigServer.java:1322)
      at org.apache.pig.PigServer.executeCompiledLogicalPlan(PigServer.java:1307)
      at org.apache.pig.PigServer.execute(PigServer.java:1297)
      at org.apache.pig.PigServer.executeBatch(PigServer.java:375)
      at org.apache.pig.PigServer.executeBatch(PigServer.java:353)
      at
      org.apache.pig.tools.grunt.GruntParser.executeBatch(GruntParser.java:140)
      at
      org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:202)
      at
      org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:173)
      at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:84)
      at org.apache.pig.Main.run(Main.java:607)
      at org.apache.pig.Main.main(Main.java:156)
      at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      at
      sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
      at
      sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      at java.lang.reflect.Method.invoke(Method.java:601)
      at org.apache.hadoop.util.RunJar.main(RunJar.java:212)

      Also reported at -
      http://mail-archives.apache.org/mod_mbox/pig-user/201402.mbox/%3C2014022712405096933026@gmail.com%3E

      and

      http://stackoverflow.com/questions/17533154/pig-java-lang-nosuchfieldexception-jobsinprogress-exception

      1. PIG-4139.patch
        0.7 kB
        Suhas Satish

        Activity

        Suhas Satish created issue -
        Suhas Satish made changes -
        Field Original Value New Value
        Status Open [ 1 ] Patch Available [ 10002 ]
        Suhas Satish made changes -
        Attachment PIG-4139.patch [ 12663490 ]
        Suhas Satish made changes -
        Description The following exception is caught and printed on stdout with logging level WARN under the environment mentined above. Its a harmless msg and the pig job falls back to using HadoopShoms 0.20 API and completes successfully. But the exception can be confusing to the user. Increasing logging level to debug in the patch.

        -----
        2014-08-20 11:12:00,879 [main] WARN
        org.apache.pig.backend.hadoop23.PigJobControl - falling back to default
        JobControl (not using hadoop 0.23 ?)
        java.lang.NoSuchFieldException: jobsInProgress
            at java.lang.Class.getDeclaredField(Class.java:1899)
            at
        org.apache.pig.backend.hadoop23.PigJobControl.<clinit>(PigJobControl.java:58)
            at
        org.apache.pig.backend.hadoop.executionengine.shims.HadoopShims.newJobControl(HadoopShims.java:104)
            at
        org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler.compile(JobControlCompiler.java:287)
            at
        org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher.launchPig(MapReduceLauncher.java:192)
            at org.apache.pig.PigServer.launchPlan(PigServer.java:1322)
            at org.apache.pig.PigServer.executeCompiledLogicalPlan(PigServer.java:1307)
            at org.apache.pig.PigServer.execute(PigServer.java:1297)
            at org.apache.pig.PigServer.executeBatch(PigServer.java:375)
            at org.apache.pig.PigServer.executeBatch(PigServer.java:353)
            at
        org.apache.pig.tools.grunt.GruntParser.executeBatch(GruntParser.java:140)
            at
        org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:202)
            at
        org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:173)
            at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:84)
            at org.apache.pig.Main.run(Main.java:607)
            at org.apache.pig.Main.main(Main.java:156)
            at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
            at
        sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
            at
        sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
            at java.lang.reflect.Method.invoke(Method.java:601)
            at org.apache.hadoop.util.RunJar.main(RunJar.java:212)

        Also reported at -
        http://mail-archives.apache.org/mod_mbox/pig-user/201402.mbox/%3C2014022712405096933026@gmail.com%3E

        and

        http://stackoverflow.com/questions/17533154/pig-java-lang-nosuchfieldexception-jobsinprogress-exception
        The following exception is caught and printed on stdout with logging level WARN under the environment mentioned above. Its a harmless msg and the pig job falls back to using HadoopShoms 0.20 API and completes successfully. But the exception can be confusing to the user. Increasing logging level to debug in the patch.

        -----
        2014-08-20 11:12:00,879 [main] WARN
        org.apache.pig.backend.hadoop23.PigJobControl - falling back to default
        JobControl (not using hadoop 0.23 ?)
        java.lang.NoSuchFieldException: jobsInProgress
            at java.lang.Class.getDeclaredField(Class.java:1899)
            at
        org.apache.pig.backend.hadoop23.PigJobControl.<clinit>(PigJobControl.java:58)
            at
        org.apache.pig.backend.hadoop.executionengine.shims.HadoopShims.newJobControl(HadoopShims.java:104)
            at
        org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler.compile(JobControlCompiler.java:287)
            at
        org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher.launchPig(MapReduceLauncher.java:192)
            at org.apache.pig.PigServer.launchPlan(PigServer.java:1322)
            at org.apache.pig.PigServer.executeCompiledLogicalPlan(PigServer.java:1307)
            at org.apache.pig.PigServer.execute(PigServer.java:1297)
            at org.apache.pig.PigServer.executeBatch(PigServer.java:375)
            at org.apache.pig.PigServer.executeBatch(PigServer.java:353)
            at
        org.apache.pig.tools.grunt.GruntParser.executeBatch(GruntParser.java:140)
            at
        org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:202)
            at
        org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:173)
            at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:84)
            at org.apache.pig.Main.run(Main.java:607)
            at org.apache.pig.Main.main(Main.java:156)
            at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
            at
        sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
            at
        sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
            at java.lang.reflect.Method.invoke(Method.java:601)
            at org.apache.hadoop.util.RunJar.main(RunJar.java:212)

        Also reported at -
        http://mail-archives.apache.org/mod_mbox/pig-user/201402.mbox/%3C2014022712405096933026@gmail.com%3E

        and

        http://stackoverflow.com/questions/17533154/pig-java-lang-nosuchfieldexception-jobsinprogress-exception
        Hide
        Cheolsoo Park added a comment -

        +1

        Show
        Cheolsoo Park added a comment - +1
        Cheolsoo Park made changes -
        Assignee Suhas Satish [ suhassatish ]
        Hide
        Cheolsoo Park added a comment -

        Committed to trunk and branch 0.13.

        Show
        Cheolsoo Park added a comment - Committed to trunk and branch 0.13.
        Cheolsoo Park made changes -
        Status Patch Available [ 10002 ] Resolved [ 5 ]
        Fix Version/s 0.14.0 [ 12326954 ]
        Fix Version/s 0.13.1 [ 12327254 ]
        Resolution Fixed [ 1 ]
        Hide
        Suhas Satish added a comment -
        Show
        Suhas Satish added a comment - thanks Cheolsoo Park
        Daniel Dai made changes -
        Status Resolved [ 5 ] Closed [ 6 ]
        Transition Time In Source Status Execution Times Last Executer Last Execution Date
        Open Open Patch Available Patch Available
        1m 12s 1 Suhas Satish 21/Aug/14 20:31
        Patch Available Patch Available Resolved Resolved
        2d 8h 14m 1 Cheolsoo Park 24/Aug/14 04:46
        Resolved Resolved Closed Closed
        89d 1h 12m 1 Daniel Dai 21/Nov/14 05:58

          People

          • Assignee:
            Suhas Satish
            Reporter:
            Suhas Satish
          • Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development