Hive
  1. Hive
  2. HIVE-6963

Beeline logs are printing on the console

    Details

    • Type: Bug Bug
    • Status: Open
    • Priority: Major Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: HiveServer2
    • Labels:
      None

      Description

      beeline logs are not redirected to the log file.
      If log is redirected to log file, only required information will print on the console.
      This way it is more easy to read the output.

      1. HIVE-6963.patch
        1 kB
        Chinna Rao Lalam

        Activity

        Hide
        Bing Li added a comment -

        Hi, Chinna
        Have you uploaded the latest patch?

        I tried the patch attached in this Jira, and found:
        1. In order to launch bin/beeline, I need to add the following jars to HADOOP_CLASSPATH in bin/ext/beeline.sh

        hive/lib/hive-shims-0.23.jar
        hive/lib/hive-shims-common-secure.jar
        hive/lib/hive-shims-common.jar

        2. The log file doesn't contain much info as the one for HiveCLI

        in its log file, it only has the following lines:
        [biadmin@bdvs1100 biadmin]$ cat hive.log
        2015-02-13 06:53:50,145 INFO jdbc.Utils (Utils.java:parseURL(285)) - Supplied authorities: bdvs1100.svl.ibm.com:10000
        2015-02-13 06:53:50,149 INFO jdbc.Utils (Utils.java:parseURL(372)) - Resolved authority: bdvs1100.svl.ibm.com:10000
        2015-02-13 06:53:50,184 INFO jdbc.HiveConnection (HiveConnection.java:openTransport(191)) - Will try to open client transport with JDBC Uri: jdbc:hive2://9.123.2.21:10000

        Are they known issue or worked as design?

        Thank you.

        • Bing
        Show
        Bing Li added a comment - Hi, Chinna Have you uploaded the latest patch? I tried the patch attached in this Jira, and found: 1. In order to launch bin/beeline, I need to add the following jars to HADOOP_CLASSPATH in bin/ext/beeline.sh hive/lib/hive-shims-0.23.jar hive/lib/hive-shims-common-secure.jar hive/lib/hive-shims-common.jar 2. The log file doesn't contain much info as the one for HiveCLI in its log file, it only has the following lines: [biadmin@bdvs1100 biadmin] $ cat hive.log 2015-02-13 06:53:50,145 INFO jdbc.Utils (Utils.java:parseURL(285)) - Supplied authorities: bdvs1100.svl.ibm.com:10000 2015-02-13 06:53:50,149 INFO jdbc.Utils (Utils.java:parseURL(372)) - Resolved authority: bdvs1100.svl.ibm.com:10000 2015-02-13 06:53:50,184 INFO jdbc.HiveConnection (HiveConnection.java:openTransport(191)) - Will try to open client transport with JDBC Uri: jdbc:hive2://9.123.2.21:10000 Are they known issue or worked as design? Thank you. Bing
        Hide
        Chinna Rao Lalam added a comment -

        I will updated the patch with review comments..

        Show
        Chinna Rao Lalam added a comment - I will updated the patch with review comments..
        Hide
        Ashutosh Chauhan added a comment -

        Test failures are probably unrelated. But having different file name, configurable path for log file and different namespace for log4j properties is a good idea. Otherwise, this patch has limited usability.

        Show
        Ashutosh Chauhan added a comment - Test failures are probably unrelated. But having different file name, configurable path for log file and different namespace for log4j properties is a good idea. Otherwise, this patch has limited usability.
        Hide
        Ashutosh Chauhan added a comment -

        Failed tests need to be looked at.

        Show
        Ashutosh Chauhan added a comment - Failed tests need to be looked at.
        Hide
        Hive QA added a comment -

        Overall: -1 at least one tests failed

        Here are the results of testing the latest attachment:
        https://issues.apache.org/jira/secure/attachment/12641442/HIVE-6963.patch

        ERROR: -1 due to 53 failed/errored test(s), 5484 tests executed
        Failed tests:

        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join32
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_filter_numeric
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_groupby2_map_skew
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_groupby_sort_1
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_groupby_sort_skew_1
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_infer_bucket_sort_list_bucket
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_list_bucket_dml_6
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_list_bucket_dml_7
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_list_bucket_dml_8
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_mapjoin_test_outer
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_metadataonly1
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_nullgroup3
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_orc_createas1
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_ppd_join4
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_select_dummy_source
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_stats_list_bucket
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_stats_partscan_1_23
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_symlink_text_input_format
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_truncate_column_list_bucket
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_udf_current_database
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_1
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_10
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_12
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_13
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_14
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_19
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_2
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_20
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_21
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_22
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_23
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_24
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_4
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_5
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_7
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_8
        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_9
        org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_bucket_map_join_tez1
        org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_count
        org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_limit_pushdown
        org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_load_dyn_part1
        org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_mrr
        org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_tez_union
        org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_union2
        org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_union3
        org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_union5
        org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_union7
        org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_union9
        org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_bucketizedhiveinputformat
        org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_root_dir_external_table
        org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_schemeAuthority
        org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_dynamic_partitions_with_whitelist
        org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_stats_partialscan_autogether
        

        Test results: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-Build/26/testReport
        Console output: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-Build/26/console

        Messages:

        Executing org.apache.hive.ptest.execution.PrepPhase
        Executing org.apache.hive.ptest.execution.ExecutionPhase
        Executing org.apache.hive.ptest.execution.ReportingPhase
        Tests exited with: TestsFailedException: 53 tests failed
        

        This message is automatically generated.

        ATTACHMENT ID: 12641442

        Show
        Hive QA added a comment - Overall : -1 at least one tests failed Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12641442/HIVE-6963.patch ERROR: -1 due to 53 failed/errored test(s), 5484 tests executed Failed tests: org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join32 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_filter_numeric org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_groupby2_map_skew org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_groupby_sort_1 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_groupby_sort_skew_1 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_infer_bucket_sort_list_bucket org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_list_bucket_dml_6 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_list_bucket_dml_7 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_list_bucket_dml_8 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_mapjoin_test_outer org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_metadataonly1 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_nullgroup3 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_orc_createas1 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_ppd_join4 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_select_dummy_source org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_stats_list_bucket org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_stats_partscan_1_23 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_symlink_text_input_format org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_truncate_column_list_bucket org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_udf_current_database org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_1 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_10 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_12 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_13 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_14 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_19 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_2 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_20 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_21 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_22 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_23 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_24 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_4 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_5 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_7 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_8 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_9 org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_bucket_map_join_tez1 org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_count org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_limit_pushdown org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_load_dyn_part1 org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_mrr org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_tez_union org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_union2 org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_union3 org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_union5 org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_union7 org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_union9 org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_bucketizedhiveinputformat org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_root_dir_external_table org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_schemeAuthority org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_dynamic_partitions_with_whitelist org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_stats_partialscan_autogether Test results: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-Build/26/testReport Console output: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-Build/26/console Messages: Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 53 tests failed This message is automatically generated. ATTACHMENT ID: 12641442
        Hide
        Vaibhav Gumashta added a comment -

        Chinna Rao Lalam Xuefu Zhang You might want to take a look at HIVE-6923 (Use slf4j For Logging Everywhere). Thanks for looking at this issue.

        Show
        Vaibhav Gumashta added a comment - Chinna Rao Lalam Xuefu Zhang You might want to take a look at HIVE-6923 (Use slf4j For Logging Everywhere). Thanks for looking at this issue.
        Hide
        Chinna Rao Lalam added a comment -

        Having separate log file also good idea. For this i will introduce one properties file like hive-beeling-log4j.properties, one configuration hive.beeline.log4j.file for configuring the log file location and log file name is hive.beeline.log.

        Show
        Chinna Rao Lalam added a comment - Having separate log file also good idea. For this i will introduce one properties file like hive-beeling-log4j.properties , one configuration hive.beeline.log4j.file for configuring the log file location and log file name is hive.beeline.log.
        Hide
        Xuefu Zhang added a comment -

        Thanks for looking into this. The patch looks good. What would be the log file name with this? I'm wondering if we need to have a separate log file, such as beeline.log, so that server log and beeline log are not clustered in a single file.

        Show
        Xuefu Zhang added a comment - Thanks for looking into this. The patch looks good. What would be the log file name with this? I'm wondering if we need to have a separate log file, such as beeline.log, so that server log and beeline log are not clustered in a single file.
        Hide
        Chinna Rao Lalam added a comment -

        Initialized the log4j in beeline.

        Show
        Chinna Rao Lalam added a comment - Initialized the log4j in beeline.

          People

          • Assignee:
            Chinna Rao Lalam
            Reporter:
            Chinna Rao Lalam
          • Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

            • Created:
              Updated:

              Development