Hive
  1. Hive
  2. HIVE-6811

LOAD command does not work with relative paths on Windows

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.14.0
    • Component/s: None
    • Labels:
      None

      Description

      qfile tests on Windows fail when trying to load data, with URISyntaxException: Relative path in absolute URI

        Activity

        Transition Time In Source Status Execution Times Last Executer Last Execution Date
        Open Open Patch Available Patch Available
        6m 28s 1 Jason Dere 02/Apr/14 07:35
        Patch Available Patch Available Resolved Resolved
        2d 17h 29m 1 Jason Dere 05/Apr/14 01:04
        Resolved Resolved Closed Closed
        222d 19h 39m 1 Thejas M Nair 13/Nov/14 19:44
        Thejas M Nair made changes -
        Status Resolved [ 5 ] Closed [ 6 ]
        Hide
        Thejas M Nair added a comment -

        This has been fixed in 0.14 release. Please open new jira if you see any issues.

        Show
        Thejas M Nair added a comment - This has been fixed in 0.14 release. Please open new jira if you see any issues.
        Jason Dere made changes -
        Status Patch Available [ 10002 ] Resolved [ 5 ]
        Fix Version/s 0.14.0 [ 12326450 ]
        Resolution Fixed [ 1 ]
        Hide
        Jason Dere added a comment -

        Committed to trunk.

        Show
        Jason Dere added a comment - Committed to trunk.
        Hide
        Jason Dere added a comment -

        TestNegativeMinimrCliDriver.testNegativeCliDriver_minimr_broken_pipe passes for me locally on Linux VM. Will go ahead and commit this.

        Show
        Jason Dere added a comment - TestNegativeMinimrCliDriver.testNegativeCliDriver_minimr_broken_pipe passes for me locally on Linux VM. Will go ahead and commit this.
        Hide
        Xuefu Zhang added a comment -

        +1.
        I guess URIUtil.decode() is okay to be used here because later the load work is constructed using Path instead of previous String as the source location.

        Show
        Xuefu Zhang added a comment - +1. I guess URIUtil.decode() is okay to be used here because later the load work is constructed using Path instead of previous String as the source location.
        Hide
        Hive QA added a comment -

        Overall: -1 at least one tests failed

        Here are the results of testing the latest attachment:
        https://issues.apache.org/jira/secure/attachment/12638204/HIVE-6811.1.patch

        ERROR: -1 due to 1 failed/errored test(s), 5539 tests executed
        Failed tests:

        org.apache.hadoop.hive.cli.TestNegativeMinimrCliDriver.testNegativeCliDriver_minimr_broken_pipe
        

        Test results: http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/2082/testReport
        Console output: http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/2082/console

        Messages:

        Executing org.apache.hive.ptest.execution.PrepPhase
        Executing org.apache.hive.ptest.execution.ExecutionPhase
        Executing org.apache.hive.ptest.execution.ReportingPhase
        Tests exited with: TestsFailedException: 1 tests failed
        

        This message is automatically generated.

        ATTACHMENT ID: 12638204

        Show
        Hive QA added a comment - Overall : -1 at least one tests failed Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12638204/HIVE-6811.1.patch ERROR: -1 due to 1 failed/errored test(s), 5539 tests executed Failed tests: org.apache.hadoop.hive.cli.TestNegativeMinimrCliDriver.testNegativeCliDriver_minimr_broken_pipe Test results: http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/2082/testReport Console output: http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/2082/console Messages: Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 1 tests failed This message is automatically generated. ATTACHMENT ID: 12638204
        Hide
        Xuefu Zhang added a comment -

        I think I'd be surprised, but I'm fine with the current patch if the tests do pass.

        Show
        Xuefu Zhang added a comment - I think I'd be surprised, but I'm fine with the current patch if the tests do pass.
        Hide
        Jason Dere added a comment -

        Thanks for the input Xuefu. Actually it looks like the 2 test cases with space/+ in the name pass (load_file_with_space_in_the_name.q, load_hdfs_file_with_space_in_the_name.q), is it ok for this change to be done? Or would you prefer that we do something Windows-specific here and prepend a "/" to the path string if on Windows?

        Show
        Jason Dere added a comment - Thanks for the input Xuefu. Actually it looks like the 2 test cases with space/+ in the name pass (load_file_with_space_in_the_name.q, load_hdfs_file_with_space_in_the_name.q), is it ok for this change to be done? Or would you prefer that we do something Windows-specific here and prepend a "/" to the path string if on Windows?
        Hide
        Xuefu Zhang added a comment -

        URIUtil.decode() is the evil that HIVE-6048 and others were trying to avoid. Thus, the patch is likely to break things. One way to try out is to have a file with '+' in its name.

        Show
        Xuefu Zhang added a comment - URIUtil.decode() is the evil that HIVE-6048 and others were trying to avoid. Thus, the patch is likely to break things. One way to try out is to have a file with '+' in its name.
        Jason Dere made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Jason Dere made changes -
        Field Original Value New Value
        Attachment HIVE-6811.1.patch [ 12638204 ]
        Hide
        Jason Dere added a comment -

        Patch to use toUri() again when generating the path string. Xuefu Zhang does this break any of the other changes in HIVE-6048, or will this work ok?

        Show
        Jason Dere added a comment - Patch to use toUri() again when generating the path string. Xuefu Zhang does this break any of the other changes in HIVE-6048 , or will this work ok?
        Hide
        Jason Dere added a comment -

        Looks like LoadSemanticAnalyzer changes in HIVE-6048 may have caused this, when path string in windows is generated with Path.toUri(), "C:/path" becomes "/C:/path" which seems to have allowed things to work before. Without the toUri() call, the path remains "C:/path" and the we see the error.

        Show
        Jason Dere added a comment - Looks like LoadSemanticAnalyzer changes in HIVE-6048 may have caused this, when path string in windows is generated with Path.toUri(), "C:/path" becomes "/C:/path" which seems to have allowed things to work before. Without the toUri() call, the path remains "C:/path" and the we see the error.
        Jason Dere created issue -

          People

          • Assignee:
            Jason Dere
            Reporter:
            Jason Dere
          • Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development