Uploaded image for project: 'Pig'
  1. Pig
  2. PIG-3860

Refactor PigStatusReporter and PigLogger for non-MR execution engine

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Major
    • Resolution: Fixed
    • None
    • tez-branch, 0.13.0
    • None
    • None

    Description

      PIG-3829 handles some cases, but NPE can be still thrown for custom counters in Tez mode. For example, the following query fails with NPE-

      SET opt.fetch false;
      SET pig.udf.profile true;
      a = LOAD 'foo' AS (x:chararray);
      b = FOREACH a GENERATE LOWER(x);
      DUMP b;
      

      The problem is that PigStatusReport.getInstance().getCounter() returns null since TezStatusReporter is used in Tez.

      PigStatusReporter.getInstance().getCounter(counterGroup, INVOCATION_COUNTER).increment(TIMING_FREQ);
      

      Attachments

        1. PIG-3860-1-tez.patch
          60 kB
          Cheolsoo Park
        2. PIG-3860-1-trunk.patch
          76 kB
          Cheolsoo Park
        3. PIG-3860-2-tez.patch
          66 kB
          Cheolsoo Park
        4. PIG-3860-2-trunk.patch
          76 kB
          Cheolsoo Park

        Issue Links

          Activity

            People

              cheolsoo Cheolsoo Park
              cheolsoo Cheolsoo Park
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: