Hive
  1. Hive
  2. HIVE-2054

Exception on windows when using the jdbc driver. "IOException: The system cannot find the path specified"

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Minor Minor
    • Resolution: Fixed
    • Affects Version/s: 0.8.0
    • Fix Version/s: 0.7.1, 0.8.0
    • Component/s: JDBC
    • Labels:
      None
    • Hadoop Flags:
      Reviewed
    • Release Note:
      Fix for IOException on the jdbc driver on windows.

      Description

      It seems something recently changed on the jdbc driver which causes this IOException on windows.

      java.lang.RuntimeException: java.io.IOException: The system cannot find the path specified
      at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:237)
      at org.apache.hadoop.hive.jdbc.HiveConnection.<init>(HiveConnection.java:73)
      at org.apache.hadoop.hive.jdbc.HiveDriver.connect(HiveDriver.java:110)

      1. HIVE-2054.3.patch.txt
        5 kB
        Bennie Schut
      2. HIVE-2054.2.patch.txt
        5 kB
        Bennie Schut
      3. HIVE-2054.1.patch.txt
        5 kB
        Bennie Schut

        Activity

        Hide
        Bennie Schut added a comment -

        This seems to happen because we use the same SessionState class the cli is using which is now including some temporary output files and history file references.
        It's rather trivial to remove the SessionState from the jdbc driver to make it work again (just tried this a few minutes ago). We currently have a JdbcSessionState which extends the SessonState but I don't see a need for the JdbcSessionState either. It seems to be there as a placeholder but is not actually used.

        Show
        Bennie Schut added a comment - This seems to happen because we use the same SessionState class the cli is using which is now including some temporary output files and history file references. It's rather trivial to remove the SessionState from the jdbc driver to make it work again (just tried this a few minutes ago). We currently have a JdbcSessionState which extends the SessonState but I don't see a need for the JdbcSessionState either. It seems to be there as a placeholder but is not actually used.
        Hide
        Bennie Schut added a comment -

        Removing SessionState so the jdbc works correctly on windows.

        Show
        Bennie Schut added a comment - Removing SessionState so the jdbc works correctly on windows.
        Show
        Bennie Schut added a comment - https://reviews.apache.org/r/513/
        Hide
        Ning Zhang added a comment -

        Bennie, do you know what changes in SessionState causes JDBC failed on Windows?

        We recently committed HIVE-818 which changes HiveServer and some behavior on how to pass End-of-file from the server side to the client side. Previously server just send an empty string and now server throws an HiveServerException with ErrorCode = 0. I think that may be the reason. Can you try your test case with and without HIVE-818 to verify?

        Show
        Ning Zhang added a comment - Bennie, do you know what changes in SessionState causes JDBC failed on Windows? We recently committed HIVE-818 which changes HiveServer and some behavior on how to pass End-of-file from the server side to the client side. Previously server just send an empty string and now server throws an HiveServerException with ErrorCode = 0. I think that may be the reason. Can you try your test case with and without HIVE-818 to verify?
        Hide
        Bennie Schut added a comment -

        Yes it was this code block:

        try

        { File tmpFile = File.createTempFile(sessionID, ".pipeout", tmpDir); tmpFile.deleteOnExit(); startSs.setTmpOutputFile(tmpFile); }

        catch (IOException e)

        { throw new RuntimeException(e); }

        So you are correct it's related to changes from HIVE-818.

        Show
        Bennie Schut added a comment - Yes it was this code block: try { File tmpFile = File.createTempFile(sessionID, ".pipeout", tmpDir); tmpFile.deleteOnExit(); startSs.setTmpOutputFile(tmpFile); } catch (IOException e) { throw new RuntimeException(e); } So you are correct it's related to changes from HIVE-818 .
        Hide
        Ning Zhang added a comment -

        I think the problem is that the tmpDir by default is /tmp/<user.name>/, specified by hive.querylog.location. Bennie, I think if you set hive.querylog.location to be something like 'c:\' on Windows, it should work. Can you try that?

        The JdbcSessionState was introduced in the very first version of the JDBC driver (HIVE-48). It is not used but I guess the reason for it to be able to set session level states for the JDBC connection. I'm OK removing it, but not sure about others' opinion.

        Show
        Ning Zhang added a comment - I think the problem is that the tmpDir by default is /tmp/<user.name>/, specified by hive.querylog.location. Bennie, I think if you set hive.querylog.location to be something like 'c:\' on Windows, it should work. Can you try that? The JdbcSessionState was introduced in the very first version of the JDBC driver ( HIVE-48 ). It is not used but I guess the reason for it to be able to set session level states for the JDBC connection. I'm OK removing it, but not sure about others' opinion.
        Hide
        Bennie Schut added a comment -

        Yes setting hive.querylog.location makes it work.

        At the very least we should remove the "extends SessionState" since it introduces a link to the hive server code which makes no sense at this point in time. However I have a preference for removing it all together since it currently adds no value. On the jdbc side I would expect the HiveConnection to hold the state which it is actually doing right now.

        Show
        Bennie Schut added a comment - Yes setting hive.querylog.location makes it work. At the very least we should remove the "extends SessionState" since it introduces a link to the hive server code which makes no sense at this point in time. However I have a preference for removing it all together since it currently adds no value. On the jdbc side I would expect the HiveConnection to hold the state which it is actually doing right now.
        Hide
        Ning Zhang added a comment -

        Sounds reasonable to me.

        Raghu, what do you think?

        Show
        Ning Zhang added a comment - Sounds reasonable to me. Raghu, what do you think?
        Hide
        Bennie Schut added a comment -

        New patch because of merge from HIVE-1815.

        Show
        Bennie Schut added a comment - New patch because of merge from HIVE-1815 .
        Hide
        Bennie Schut added a comment -

        New patch because of changes from HIVE-2062

        Show
        Bennie Schut added a comment - New patch because of changes from HIVE-2062
        Hide
        Bennie Schut added a comment -

        Any chance in getting this committed This is rather annoying for windows users.

        Show
        Bennie Schut added a comment - Any chance in getting this committed This is rather annoying for windows users.
        Hide
        Carl Steinbach added a comment -

        +1. Will commit if tests pass.

        Show
        Carl Steinbach added a comment - +1. Will commit if tests pass.
        Hide
        Carl Steinbach added a comment -

        Committed to trunk and 0.7.1. Thanks Bennie!

        Show
        Carl Steinbach added a comment - Committed to trunk and 0.7.1. Thanks Bennie!

          People

          • Assignee:
            Bennie Schut
            Reporter:
            Bennie Schut
          • Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development