Hive
  1. Hive
  2. HIVE-6570

Hive variable substitution does not work with the "source" command

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.14.0
    • Component/s: None
    • Labels:
    • Release Note:
      Hide
      This patch adds Hive variable substitution support to the "source" command. For example, you will now be able to use a statement such as:
      source ${hivevar:test-dir}/test.q;
      Show
      This patch adds Hive variable substitution support to the "source" command. For example, you will now be able to use a statement such as: source ${hivevar:test-dir}/test.q;

      Description

      The following does not work:

      source ${hivevar:test-dir}/test.q;
      

        Issue Links

          Activity

          Hide
          Thejas M Nair added a comment -

          This has been fixed in 0.14 release. Please open new jira if you see any issues.

          Show
          Thejas M Nair added a comment - This has been fixed in 0.14 release. Please open new jira if you see any issues.
          Hide
          Anthony Hsu added a comment -

          Lefty Leverenz - Thanks for the instructions!
          Xuefu Zhang - Thanks for committing this!

          Show
          Anthony Hsu added a comment - Lefty Leverenz - Thanks for the instructions! Xuefu Zhang - Thanks for committing this!
          Hide
          Xuefu Zhang added a comment -

          Patch committed to trunk. Thanks to Anthony for the contribution.

          Show
          Xuefu Zhang added a comment - Patch committed to trunk. Thanks to Anthony for the contribution.
          Hide
          Lefty Leverenz added a comment -

          "AboutThisWiki" gives these instructions:

          How to get permission to edit

          • Create a Confluence account
          • Sign up for the user mailing list by sending a message to user-subscribe@hive.apache.org
          • Send a message to user@hive.apache.org requesting write access

          AboutThisWiki

          Show
          Lefty Leverenz added a comment - "AboutThisWiki" gives these instructions: How to get permission to edit Create a Confluence account Sign up for the user mailing list by sending a message to user-subscribe@hive.apache.org Send a message to user@hive.apache.org requesting write access AboutThisWiki
          Hide
          Anthony Hsu added a comment -

          It would probably be nice to add an example to the Variable Substitution page that uses variable substitution with the source command.

          On a side note, how does one get edit privileges for the wiki?

          Show
          Anthony Hsu added a comment - It would probably be nice to add an example to the Variable Substitution page that uses variable substitution with the source command. On a side note, how does one get edit privileges for the wiki?
          Hide
          Lefty Leverenz added a comment -

          Do the bug & fix need to be mentioned in the wiki? A version note could be added in the CLI doc and/or Variable Substitution:

          Show
          Lefty Leverenz added a comment - Do the bug & fix need to be mentioned in the wiki? A version note could be added in the CLI doc and/or Variable Substitution: CLI: Hive Interactive Shell Commands Variable Substitution
          Hide
          Anthony Hsu added a comment -

          Thanks. Could one of you guys commit the patch for me please?

          Show
          Anthony Hsu added a comment - Thanks. Could one of you guys commit the patch for me please?
          Hide
          Edward Capriolo added a comment -

          No major concern the release note is enough information. Sorry I was not paying attention to this thread. Please proceed.

          Show
          Edward Capriolo added a comment - No major concern the release note is enough information. Sorry I was not paying attention to this thread. Please proceed.
          Hide
          Anthony Hsu added a comment -

          What concerns does Edward Capriolo have?

          Show
          Anthony Hsu added a comment - What concerns does Edward Capriolo have?
          Hide
          Ashutosh Chauhan added a comment -

          I am waiting to hear from Edward Capriolo seems like he had some concerns.

          Show
          Ashutosh Chauhan added a comment - I am waiting to hear from Edward Capriolo seems like he had some concerns.
          Hide
          Anthony Hsu added a comment -

          Ping

          Show
          Anthony Hsu added a comment - Ping
          Hide
          Ashutosh Chauhan added a comment -

          Edward Capriolo Let us know if you have any more feedback.

          Show
          Ashutosh Chauhan added a comment - Edward Capriolo Let us know if you have any more feedback.
          Hide
          Anthony Hsu added a comment -

          Added a Release Note explaining the changes in this patch.

          Show
          Anthony Hsu added a comment - Added a Release Note explaining the changes in this patch.
          Hide
          Ashutosh Chauhan added a comment -

          If I am reading the patch correctly, it does that interpretation only for source statement, not for anything else.

          Show
          Ashutosh Chauhan added a comment - If I am reading the patch correctly, it does that interpretation only for source statement, not for anything else.
          Hide
          Edward Capriolo added a comment -

          WE should make a release note if someone has $ in there file hive might now try to interpret it.

          Show
          Edward Capriolo added a comment - WE should make a release note if someone has $ in there file hive might now try to interpret it.
          Hide
          Ashutosh Chauhan added a comment -

          yeah, failure is unrelated. I will commit this shortly.

          Show
          Ashutosh Chauhan added a comment - yeah, failure is unrelated. I will commit this shortly.
          Hide
          Anthony Hsu added a comment -

          The single test failure seems unrelated to my change.

          Ashutosh, can you commit my change for me?

          Show
          Anthony Hsu added a comment - The single test failure seems unrelated to my change. Ashutosh, can you commit my change for me?
          Hide
          Ashutosh Chauhan added a comment -

          +1

          Show
          Ashutosh Chauhan added a comment - +1
          Hide
          Hive QA added a comment -

          Overall: -1 at least one tests failed

          Here are the results of testing the latest attachment:
          https://issues.apache.org/jira/secure/attachment/12633453/HIVE-6570.1.patch

          ERROR: -1 due to 1 failed/errored test(s), 5374 tests executed
          Failed tests:

          org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_root_dir_external_table
          

          Test results: http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/1678/testReport
          Console output: http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/1678/console

          Messages:

          Executing org.apache.hive.ptest.execution.PrepPhase
          Executing org.apache.hive.ptest.execution.ExecutionPhase
          Executing org.apache.hive.ptest.execution.ReportingPhase
          Tests exited with: TestsFailedException: 1 tests failed
          

          This message is automatically generated.

          ATTACHMENT ID: 12633453

          Show
          Hive QA added a comment - Overall : -1 at least one tests failed Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12633453/HIVE-6570.1.patch ERROR: -1 due to 1 failed/errored test(s), 5374 tests executed Failed tests: org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_root_dir_external_table Test results: http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/1678/testReport Console output: http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/1678/console Messages: Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 1 tests failed This message is automatically generated. ATTACHMENT ID: 12633453
          Hide
          Hive QA added a comment -

          Overall: -1 at least one tests failed

          Here are the results of testing the latest attachment:
          https://issues.apache.org/jira/secure/attachment/12633248/HIVE-6570.1.patch.txt

          ERROR: -1 due to 3 failed/errored test(s), 5360 tests executed
          Failed tests:

          org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_root_dir_external_table
          org.apache.hive.beeline.TestSchemaTool.testSchemaInit
          org.apache.hive.beeline.TestSchemaTool.testSchemaUpgrade
          

          Test results: http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/1650/testReport
          Console output: http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/1650/console

          Messages:

          Executing org.apache.hive.ptest.execution.PrepPhase
          Executing org.apache.hive.ptest.execution.ExecutionPhase
          Executing org.apache.hive.ptest.execution.ReportingPhase
          Tests exited with: TestsFailedException: 3 tests failed
          

          This message is automatically generated.

          ATTACHMENT ID: 12633248

          Show
          Hive QA added a comment - Overall : -1 at least one tests failed Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12633248/HIVE-6570.1.patch.txt ERROR: -1 due to 3 failed/errored test(s), 5360 tests executed Failed tests: org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_root_dir_external_table org.apache.hive.beeline.TestSchemaTool.testSchemaInit org.apache.hive.beeline.TestSchemaTool.testSchemaUpgrade Test results: http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/1650/testReport Console output: http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/1650/console Messages: Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 3 tests failed This message is automatically generated. ATTACHMENT ID: 12633248
          Hide
          Anthony Hsu added a comment -

          Reuploading identical patch without the ".txt" suffix so that the precommit tests will run.

          Show
          Anthony Hsu added a comment - Reuploading identical patch without the ".txt" suffix so that the precommit tests will run.
          Hide
          Szehon Ho added a comment -

          Looks like this will work, +1 (non-binding)

          Show
          Szehon Ho added a comment - Looks like this will work, +1 (non-binding)
          Hide
          Anthony Hsu added a comment -

          Review Board code review: https://reviews.apache.org/r/18874/

          Show
          Anthony Hsu added a comment - Review Board code review: https://reviews.apache.org/r/18874/
          Hide
          Anthony Hsu added a comment -

          Added support for Hive variable substitution with the "source" command, and added a test for this in source.q.

          Show
          Anthony Hsu added a comment - Added support for Hive variable substitution with the "source" command, and added a test for this in source.q.
          Hide
          Anthony Hsu added a comment -

          I have a fix for this issue and will upload a patch shortly.

          Show
          Anthony Hsu added a comment - I have a fix for this issue and will upload a patch shortly.

            People

            • Assignee:
              Anthony Hsu
              Reporter:
              Anthony Hsu
            • Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development