Uploaded image for project: 'Pig'
  1. Pig
  2. PIG-2924

PigStats should not be assuming all Storage classes to be file-based storage

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 0.9.2, 0.10.0
    • Fix Version/s: 0.12.0
    • Component/s: tools
    • Labels:
      None

      Description

      Using PigStatsUtil (like Oozie does) to collect JobStats for jobs that use a HBaseStorage blows up when the stats are asked to be accumulated.

      This is because JobStats (which adds stuff up) is assuming all storages are file based and that it can do listStatus/etc. operations on their filespec-provided filename. For HBaseStorage, this is set to the tablename and there's no such file, leading to an exception (FileNotFound or Invalid URI - depending on using 'tablename' or 'hbase://tablename').

        Attachments

        1. PIG-2924-5.patch
          20 kB
          Cheolsoo Park
        2. PIG-2924-4.patch
          19 kB
          Cheolsoo Park
        3. PIG-2924-3.patch
          18 kB
          Cheolsoo Park
        4. PIG-2924-2.patch
          19 kB
          Cheolsoo Park
        5. PIG-2924.patch
          18 kB
          Cheolsoo Park

          Issue Links

            Activity

              People

              • Assignee:
                cheolsoo Cheolsoo Park
                Reporter:
                qwertymaniac Harsh J
              • Votes:
                1 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: