Pig
  1. Pig
  2. PIG-3617

problem with temp file deletion in MAPREDUCE operator

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.12.0
    • Fix Version/s: 0.13.0
    • Component/s: None
    • Labels:
      None
    • Patch Info:
      Patch Available

      Description

      Hi all,
      When I run a native MR job with the MAPREDUCE keyword and store the intermediate data in HBase with:
      stored = MAPREDUCE 'my.jar'
      STORE x INTO 'hbase://temp_table'
      USING org.apache.pig.backend.hadoop.hbase.HBaseStorage('hbase_schema')
      .... and the rest ....;

      Pig tries to delete the temp files, which in this case has an HBase path, and fails with the exception:

      Caused by: java.lang.IllegalArgumentException: java.net.URISyntaxException: Relative path in absolute URI: file:hbase:/temp_table
      at org.apache.hadoop.fs.Path.initialize(Path.java:148)
      at org.apache.hadoop.fs.Path.<init>(Path.java:126)
      at org.apache.pig.backend.hadoop.datastorage.HDataStorage.isContainer(HDataStorage.java:197)
      at org.apache.pig.backend.hadoop.datastorage.HDataStorage.asElement(HDataStorage.java:128)
      at org.apache.pig.impl.io.FileLocalizer.delete(FileLocalizer.java:415)
      at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher.launchPig(MapReduceLauncher.java:419)
      at org.apache.pig.PigServer.launchPlan(PigServer.java:1322)

      1. PIG-3617.patch
        1 kB
        Nezih Yigitbasi
      2. PIG-3617-2.patch
        1.0 kB
        Cheolsoo Park

        Activity

        No work has yet been logged on this issue.

          People

          • Assignee:
            Nezih Yigitbasi
            Reporter:
            Nezih Yigitbasi
          • Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development