Pig
  1. Pig
  2. PIG-3617

problem with temp file deletion in MAPREDUCE operator

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.12.0
    • Fix Version/s: 0.13.0
    • Component/s: None
    • Labels:
      None
    • Patch Info:
      Patch Available

      Description

      Hi all,
      When I run a native MR job with the MAPREDUCE keyword and store the intermediate data in HBase with:
      stored = MAPREDUCE 'my.jar'
      STORE x INTO 'hbase://temp_table'
      USING org.apache.pig.backend.hadoop.hbase.HBaseStorage('hbase_schema')
      .... and the rest ....;

      Pig tries to delete the temp files, which in this case has an HBase path, and fails with the exception:

      Caused by: java.lang.IllegalArgumentException: java.net.URISyntaxException: Relative path in absolute URI: file:hbase:/temp_table
      at org.apache.hadoop.fs.Path.initialize(Path.java:148)
      at org.apache.hadoop.fs.Path.<init>(Path.java:126)
      at org.apache.pig.backend.hadoop.datastorage.HDataStorage.isContainer(HDataStorage.java:197)
      at org.apache.pig.backend.hadoop.datastorage.HDataStorage.asElement(HDataStorage.java:128)
      at org.apache.pig.impl.io.FileLocalizer.delete(FileLocalizer.java:415)
      at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher.launchPig(MapReduceLauncher.java:419)
      at org.apache.pig.PigServer.launchPlan(PigServer.java:1322)

      1. PIG-3617-2.patch
        1.0 kB
        Cheolsoo Park
      2. PIG-3617.patch
        1 kB
        Nezih Yigitbasi

        Activity

        Nezih Yigitbasi created issue -
        Nezih Yigitbasi made changes -
        Field Original Value New Value
        Patch Info Patch Available [ 10042 ]
        Nezih Yigitbasi made changes -
        Patch Info Patch Available [ 10042 ]
        Nezih Yigitbasi made changes -
        Patch Info Patch Available [ 10042 ]
        Nezih Yigitbasi made changes -
        Attachment PIG-3617.patch [ 12621630 ]
        Cheolsoo Park made changes -
        Attachment PIG-3617-2.patch [ 12622061 ]
        Cheolsoo Park made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Assignee Nezih Yigitbasi [ nezihyigitbasi ]
        Fix Version/s 0.13.0 [ 12324971 ]
        Cheolsoo Park made changes -
        Status Patch Available [ 10002 ] Resolved [ 5 ]
        Resolution Fixed [ 1 ]
        Daniel Dai made changes -
        Status Resolved [ 5 ] Closed [ 6 ]

          People

          • Assignee:
            Nezih Yigitbasi
            Reporter:
            Nezih Yigitbasi
          • Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development