Pig
  1. Pig
  2. PIG-2924

PigStats should not be assuming all Storage classes to be file-based storage

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.9.2, 0.10.0
    • Fix Version/s: 0.12.0
    • Component/s: tools
    • Labels:
      None

      Description

      Using PigStatsUtil (like Oozie does) to collect JobStats for jobs that use a HBaseStorage blows up when the stats are asked to be accumulated.

      This is because JobStats (which adds stuff up) is assuming all storages are file based and that it can do listStatus/etc. operations on their filespec-provided filename. For HBaseStorage, this is set to the tablename and there's no such file, leading to an exception (FileNotFound or Invalid URI - depending on using 'tablename' or 'hbase://tablename').

      1. PIG-2924.patch
        18 kB
        Cheolsoo Park
      2. PIG-2924-2.patch
        19 kB
        Cheolsoo Park
      3. PIG-2924-3.patch
        18 kB
        Cheolsoo Park
      4. PIG-2924-4.patch
        19 kB
        Cheolsoo Park
      5. PIG-2924-5.patch
        20 kB
        Cheolsoo Park

        Issue Links

          Activity

          Mona Chitnis made changes -
          Link This issue relates to PIG-3891 [ PIG-3891 ]
          Daniel Dai made changes -
          Status Resolved [ 5 ] Closed [ 6 ]
          Cheolsoo Park made changes -
          Link This issue breaks PIG-3066 [ PIG-3066 ]
          Cheolsoo Park made changes -
          Status Patch Available [ 10002 ] Resolved [ 5 ]
          Fix Version/s 0.12 [ 12323380 ]
          Resolution Fixed [ 1 ]
          Cheolsoo Park made changes -
          Attachment PIG-2924-5.patch [ 12552726 ]
          Cheolsoo Park made changes -
          Attachment PIG-2924-4.patch [ 12551472 ]
          Cheolsoo Park made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Cheolsoo Park made changes -
          Attachment PIG-2924-3.patch [ 12551305 ]
          Bill Graham made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Cheolsoo Park made changes -
          Attachment PIG-2924-2.patch [ 12551152 ]
          Cheolsoo Park made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Affects Version/s 0.10.0 [ 12316246 ]
          Cheolsoo Park made changes -
          Attachment PIG-2924.patch [ 12551048 ]
          Cheolsoo Park made changes -
          Field Original Value New Value
          Assignee Cheolsoo Park [ cheolsoo ]
          Harsh J created issue -

            People

            • Assignee:
              Cheolsoo Park
              Reporter:
              Harsh J
            • Votes:
              1 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development