Hive
  1. Hive
  2. HIVE-6561

Beeline should accept -i option to Initializing a SQL file

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.10.0, 0.11.0, 0.12.0
    • Fix Version/s: 0.14.0
    • Component/s: None
    • Labels:
      None
    • Release Note:
      Document the new feature.
    • Tags:
      TODOC14

      Description

      Hive CLI has -i option. From Hive CLI help:

      ...
       -i <filename>                    Initialization SQL file
      ...
      

      However, Beeline has no such option:

      xzhang@xzlt:~/apa/hive3$ ./packaging/target/apache-hive-0.14.0-SNAPSHOT-bin/apache-hive-0.14.0-SNAPSHOT-bin/bin/beeline -u jdbc:hive2:// -i hive.rc
      ...
      Connected to: Apache Hive (version 0.14.0-SNAPSHOT)
      Driver: Hive JDBC (version 0.14.0-SNAPSHOT)
      Transaction isolation: TRANSACTION_REPEATABLE_READ
      -i (No such file or directory)
      Property "url" is required
      Beeline version 0.14.0-SNAPSHOT by Apache Hive
      ...
      
      1. HIVE-6561.4.patch.txt
        13 kB
        Navis
      2. HIVE-6561.3.patch.txt
        11 kB
        Navis
      3. HIVE-6561.2.patch.txt
        11 kB
        Navis
      4. HIVE-6561.1.patch.txt
        10 kB
        Navis

        Issue Links

          Activity

          Hide
          Navis added a comment -

          Xuefu Zhang I have a patch on this. Can I attach it?

          Show
          Navis added a comment - Xuefu Zhang I have a patch on this. Can I attach it?
          Hide
          Xuefu Zhang added a comment -

          Navis, by all means please. I haven't started yet. Thanks. I'm going to assign this to you.

          Show
          Xuefu Zhang added a comment - Navis , by all means please. I haven't started yet. Thanks. I'm going to assign this to you.
          Hide
          Xuefu Zhang added a comment -

          Thanks for the patch, Navis. I went thru it briefly and had the following questions:

          1. When Beeline is started, it may not connect to any HS2. In that case, it probably doesn't make sense to execute the init file. This is different from Hive CLI, which will always have DB to work on.

          2. When a Beeline makes a connection to a DB, we probably should execute the init file.

          Thus, I was thinking that executing the init file is tied to a DB connection. No connection, no execution. For each connection, the init file should be execute once.

          What is your thought on this?

          Show
          Xuefu Zhang added a comment - Thanks for the patch, Navis . I went thru it briefly and had the following questions: 1. When Beeline is started, it may not connect to any HS2. In that case, it probably doesn't make sense to execute the init file. This is different from Hive CLI, which will always have DB to work on. 2. When a Beeline makes a connection to a DB, we probably should execute the init file. Thus, I was thinking that executing the init file is tied to a DB connection. No connection, no execution. For each connection, the init file should be execute once. What is your thought on this?
          Hide
          Navis added a comment -

          Yes, I've thought of that. But would it be possible to do something with beeline without connection for init? I remember there is an issue for beeline to run shell commands like CLI. If init file is for initializing connection property or resources, doing it after connection would be meaningless.

          Show
          Navis added a comment - Yes, I've thought of that. But would it be possible to do something with beeline without connection for init? I remember there is an issue for beeline to run shell commands like CLI. If init file is for initializing connection property or resources, doing it after connection would be meaningless.
          Hide
          Xuefu Zhang added a comment -

          While it's possible for Beeline to do some initialization w/o a connection, such as set color or execute a shell command (HIVE-6694), per my experience, init script for Hive is mostly used to control server behavior, such as loading jars or setting conf variables. Without a connection, these become meaningless.

          Hive CLI doesn't have the issue because a connection always exist, either embedded "server" or a remote server. Beeline can be started w/o either an embedded server or remote server, on the other hand.

          When I said "doing it after connection", I really meant "doing immediately WHEN the connection is made", which is the current Beeline behavior for variables that come either from connection string or command line. I'd propose that we execute additional commands specified in the init file whenever a connection is made. This is consistent with both Hive CLI and existing Beeline behavior. Under this proposal, client-side initialization is skipped w/o a connection, but I think we should be Okay: 1. most of time beeline will have a connection when started; 2. user can do "!run script.init" to execute a script file w/o a connection.

          Show
          Xuefu Zhang added a comment - While it's possible for Beeline to do some initialization w/o a connection, such as set color or execute a shell command ( HIVE-6694 ), per my experience, init script for Hive is mostly used to control server behavior, such as loading jars or setting conf variables. Without a connection, these become meaningless. Hive CLI doesn't have the issue because a connection always exist, either embedded "server" or a remote server. Beeline can be started w/o either an embedded server or remote server, on the other hand. When I said "doing it after connection", I really meant "doing immediately WHEN the connection is made", which is the current Beeline behavior for variables that come either from connection string or command line. I'd propose that we execute additional commands specified in the init file whenever a connection is made. This is consistent with both Hive CLI and existing Beeline behavior. Under this proposal, client-side initialization is skipped w/o a connection, but I think we should be Okay: 1. most of time beeline will have a connection when started; 2. user can do "!run script.init" to execute a script file w/o a connection.
          Hide
          Xuefu Zhang added a comment -

          Patch looks good. I had a couple of minor comments on RB.

          Show
          Xuefu Zhang added a comment - Patch looks good. I had a couple of minor comments on RB.
          Hide
          Hive QA added a comment -

          Overall: -1 at least one tests failed

          Here are the results of testing the latest attachment:
          https://issues.apache.org/jira/secure/attachment/12643683/HIVE-6561.3.patch.txt

          ERROR: -1 due to 8 failed/errored test(s), 5495 tests executed
          Failed tests:

          org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_stats_partscan_1_23
          org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_bucket_map_join_tez1
          org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_bucket_map_join_tez2
          org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_insert1
          org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_load_dyn_part1
          org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_tez_dml
          org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_tez_union
          org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_root_dir_external_table
          

          Test results: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-Build/139/testReport
          Console output: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-Build/139/console

          Messages:

          Executing org.apache.hive.ptest.execution.PrepPhase
          Executing org.apache.hive.ptest.execution.ExecutionPhase
          Executing org.apache.hive.ptest.execution.ReportingPhase
          Tests exited with: TestsFailedException: 8 tests failed
          

          This message is automatically generated.

          ATTACHMENT ID: 12643683

          Show
          Hive QA added a comment - Overall : -1 at least one tests failed Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12643683/HIVE-6561.3.patch.txt ERROR: -1 due to 8 failed/errored test(s), 5495 tests executed Failed tests: org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_stats_partscan_1_23 org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_bucket_map_join_tez1 org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_bucket_map_join_tez2 org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_insert1 org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_load_dyn_part1 org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_tez_dml org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_tez_union org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_root_dir_external_table Test results: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-Build/139/testReport Console output: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-Build/139/console Messages: Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 8 tests failed This message is automatically generated. ATTACHMENT ID: 12643683
          Hide
          Xuefu Zhang added a comment -

          Navis Would you have time to make this forward? Let me know if you need someone's help on this.

          Show
          Xuefu Zhang added a comment - Navis Would you have time to make this forward? Let me know if you need someone's help on this.
          Hide
          Xuefu Zhang added a comment -

          +1 pending on test result.

          Show
          Xuefu Zhang added a comment - +1 pending on test result.
          Hide
          Hive QA added a comment -

          Overall: -1 at least one tests failed

          Here are the results of testing the latest attachment:
          https://issues.apache.org/jira/secure/attachment/12650709/HIVE-6561.4.patch.txt

          ERROR: -1 due to 4 failed/errored test(s), 5654 tests executed
          Failed tests:

          org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_parquet_columnar
          org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_root_dir_external_table
          org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_authorization_ctas
          org.apache.hadoop.hive.ql.exec.tez.TestTezTask.testSubmit
          

          Test results: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-Build/496/testReport
          Console output: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-Build/496/console
          Test logs: http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-Build-496/

          Messages:

          Executing org.apache.hive.ptest.execution.PrepPhase
          Executing org.apache.hive.ptest.execution.ExecutionPhase
          Executing org.apache.hive.ptest.execution.ReportingPhase
          Tests exited with: TestsFailedException: 4 tests failed
          

          This message is automatically generated.

          ATTACHMENT ID: 12650709

          Show
          Hive QA added a comment - Overall : -1 at least one tests failed Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12650709/HIVE-6561.4.patch.txt ERROR: -1 due to 4 failed/errored test(s), 5654 tests executed Failed tests: org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_parquet_columnar org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_root_dir_external_table org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_authorization_ctas org.apache.hadoop.hive.ql.exec.tez.TestTezTask.testSubmit Test results: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-Build/496/testReport Console output: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-Build/496/console Test logs: http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-Build-496/ Messages: Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 4 tests failed This message is automatically generated. ATTACHMENT ID: 12650709
          Hide
          Xuefu Zhang added a comment -

          Patch committed to trunk. Thanks to Navis for the contribution.

          Show
          Xuefu Zhang added a comment - Patch committed to trunk. Thanks to Navis for the contribution.
          Hide
          Thejas M Nair added a comment -

          This has been fixed in 0.14 release. Please open new jira if you see any issues.

          Show
          Thejas M Nair added a comment - This has been fixed in 0.14 release. Please open new jira if you see any issues.

            People

            • Assignee:
              Navis
              Reporter:
              Xuefu Zhang
            • Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development