Uploaded image for project: 'Pig'
  1. Pig
  2. PIG-2924

PigStats should not be assuming all Storage classes to be file-based storage

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Major
    • Resolution: Fixed
    • 0.9.2, 0.10.0
    • 0.12.0
    • tools
    • None

    Description

      Using PigStatsUtil (like Oozie does) to collect JobStats for jobs that use a HBaseStorage blows up when the stats are asked to be accumulated.

      This is because JobStats (which adds stuff up) is assuming all storages are file based and that it can do listStatus/etc. operations on their filespec-provided filename. For HBaseStorage, this is set to the tablename and there's no such file, leading to an exception (FileNotFound or Invalid URI - depending on using 'tablename' or 'hbase://tablename').

      Attachments

        1. PIG-2924-5.patch
          20 kB
          Cheolsoo Park
        2. PIG-2924-4.patch
          19 kB
          Cheolsoo Park
        3. PIG-2924-3.patch
          18 kB
          Cheolsoo Park
        4. PIG-2924-2.patch
          19 kB
          Cheolsoo Park
        5. PIG-2924.patch
          18 kB
          Cheolsoo Park

        Issue Links

          Activity

            People

              cheolsoo Cheolsoo Park
              qwertymaniac Harsh J
              Votes:
              1 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: