Pig
  1. Pig
  2. PIG-1283

COUNT on null bag causes failure

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.11
    • Component/s: impl
    • Labels:

      Description

      grunt> l = load '/tmp/e.bag' as (b : bag

      {t: (i : int)}

      , a : int);

      1. b is null for the only row
        grunt> c = foreach l generate COUNT(b);
        grunt> dump c

      It results in following exception-

      org.apache.pig.backend.executionengine.ExecException: ERROR 2106: Error while computing count in COUNT
      at org.apache.pig.builtin.COUNT.exec(COUNT.java:59)
      at org.apache.pig.builtin.COUNT.exec(COUNT.java:39)
      at org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POUserFunc.getNext(POUserFunc.java:212)
      at org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POUserFunc.getNext(POUserFunc.java:293)
      at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.processPlan(POForEach.java:358)
      at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.getNext(POForEach.java:288)
      at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.runPipeline(PigMapBase.java:232)
      at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.map(PigMapBase.java:227)
      at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.map(PigMapBase.java:52)
      at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
      at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:583)
      at org.apache.hadoop.mapred.MapTask.run(MapTask.java:305)
      at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:176)
      Caused by: java.lang.NullPointerException
      at org.apache.pig.builtin.COUNT.exec(COUNT.java:46)
      ... 12 more

      1. pig_1283-3.patch
        2 kB
        Anandha L Ranganathan
      2. PIG-1283-2.patch
        1 kB
        Anandha L Ranganathan
      3. PIG-1283-1.patch
        0.5 kB
        Anandha L Ranganathan

        Activity

        Hide
        Jonathan Coveney added a comment -

        Anand,

        Thanks for the contribution! Committed.

        Show
        Jonathan Coveney added a comment - Anand, Thanks for the contribution! Committed.
        Hide
        Anandha L Ranganathan added a comment -

        Please find pig_1283-3.patch for this bug.
        I added the changes suggested by Jonathan.

        Show
        Anandha L Ranganathan added a comment - Please find pig_1283-3.patch for this bug. I added the changes suggested by Jonathan.
        Hide
        Anandha L Ranganathan added a comment -

        I will modify test case and make it as
        assertNull(count.exec(tup));

        Also, I will clean up the code (remove tab and add space) and update the patch.

        Show
        Anandha L Ranganathan added a comment - I will modify test case and make it as assertNull(count.exec(tup)); Also, I will clean up the code (remove tab and add space) and update the patch.
        Hide
        Jonathan Coveney added a comment -

        Anand,

        Thanks for working on this.

        Some feedback:

        1. make sure the spacing is good. No tabs, four space indents, etc. I realize a lot of existing Pig code is janky, but we try to make sure new stuff doesn't introduce that.
        2. I don't know if your test case is actually testing the change? I mean, the change you made has COUNT return null in the case when you give it a null bag, right? So why does it return 0?
        3. use assertEquals(expected, actual) instead of assertTrue

        Show
        Jonathan Coveney added a comment - Anand, Thanks for working on this. Some feedback: 1. make sure the spacing is good. No tabs, four space indents, etc. I realize a lot of existing Pig code is janky, but we try to make sure new stuff doesn't introduce that. 2. I don't know if your test case is actually testing the change? I mean, the change you made has COUNT return null in the case when you give it a null bag, right? So why does it return 0? 3. use assertEquals(expected, actual) instead of assertTrue
        Hide
        Dmitriy V. Ryaboy added a comment -

        marking as patch available

        Show
        Dmitriy V. Ryaboy added a comment - marking as patch available
        Hide
        Anandha L Ranganathan added a comment -

        Please find the patch for this bug

        Show
        Anandha L Ranganathan added a comment - Please find the patch for this bug
        Hide
        Anandha L Ranganathan added a comment -

        Working.

        Show
        Anandha L Ranganathan added a comment - Working.
        Hide
        Daniel Dai added a comment -

        Hi, Anand, can you add a test case? Usually we add a test case for bug fix if possible. Thanks.

        Show
        Daniel Dai added a comment - Hi, Anand, can you add a test case? Usually we add a test case for bug fix if possible. Thanks.
        Hide
        Anandha L Ranganathan added a comment -

        Please review this patch.

        Show
        Anandha L Ranganathan added a comment - Please review this patch.
        Hide
        Anandha L Ranganathan added a comment -

        Hi

        Please take a look at the patch and provide your feedback.

        Show
        Anandha L Ranganathan added a comment - Hi Please take a look at the patch and provide your feedback.

          People

          • Assignee:
            Anandha L Ranganathan
            Reporter:
            Thejas M Nair
          • Votes:
            1 Vote for this issue
            Watchers:
            7 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development