Hive
  1. Hive
  2. HIVE-2776

Setting test.silent=false has not effect

    Details

    • Type: Bug Bug
    • Status: Open
    • Priority: Major Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: Testing Infrastructure
    • Labels:
      None

      Activity

      Hide
      Carl Steinbach added a comment -

      QTestUtil and the various CliDriver templates all print out the following help message:

      try "ant test ... -Dtest.silent=false" to get more logs.
      

      However, it turns out that test.silent=false by default, and you change the behavior of the tests only by setting test.silent=true.

      From MapredLocalTask:

            String isSilent = "true".equalsIgnoreCase(System.getProperty("test.silent")) ? "-nolog" : "";
      

      From MapRedTask:

            String isSilent = "true".equalsIgnoreCase(System
                .getProperty("test.silent")) ? "-nolog" : "";
      
      Show
      Carl Steinbach added a comment - QTestUtil and the various CliDriver templates all print out the following help message: try "ant test ... -Dtest.silent=false" to get more logs. However, it turns out that test.silent=false by default, and you change the behavior of the tests only by setting test.silent=true. From MapredLocalTask: String isSilent = "true".equalsIgnoreCase(System.getProperty("test.silent")) ? "-nolog" : ""; From MapRedTask: String isSilent = "true".equalsIgnoreCase(System .getProperty("test.silent")) ? "-nolog" : "";
      Hide
      Zhenxiao Luo added a comment -

      test.silent is set to be true by default, and there is no -nolog parameter passed to the hadoop command that submitted as a MapReduce Job.

      When setting test.silent to be false, -nolog parameter is passed to the hadoop command, and configured the MapReduce Job.

      This -nolog will only make effect when the MapReduce Job is failing. While, in most testcases, the MapReduce job succeeded, but the optimizer/compiler is doing wrong, so the result is different from the expected output.

      My proposed solution:

      1. Remove this "try "ant test ... -Dtest.silent=false" to get more logs." from QTestUtil, since this message does not make sense in the test context.
      Even if you set -Dtest.silent=false, you may get no detailed log(in most cases), since the MapReduce Job does not fail, the optimizer/compiler is doing things wrong.

      2. Print these log info when a MapReduce Job is failing. while only happens in MapRedTask, should not be in MapRedLocalTask.

      3. The property name is misleading, it seems like a test property, but actually, both MapRedTask and MapredLocalTask is using this property. We could either remove this property at all, and always print the log info when a MapReduce Job is failing as stated in #2, or choose another non-test related property name to determine the behavior of MapReduce Job failing logs.

      Show
      Zhenxiao Luo added a comment - test.silent is set to be true by default, and there is no -nolog parameter passed to the hadoop command that submitted as a MapReduce Job. When setting test.silent to be false, -nolog parameter is passed to the hadoop command, and configured the MapReduce Job. This -nolog will only make effect when the MapReduce Job is failing. While, in most testcases, the MapReduce job succeeded, but the optimizer/compiler is doing wrong, so the result is different from the expected output. My proposed solution: 1. Remove this "try "ant test ... -Dtest.silent=false" to get more logs." from QTestUtil, since this message does not make sense in the test context. Even if you set -Dtest.silent=false, you may get no detailed log(in most cases), since the MapReduce Job does not fail, the optimizer/compiler is doing things wrong. 2. Print these log info when a MapReduce Job is failing. while only happens in MapRedTask, should not be in MapRedLocalTask. 3. The property name is misleading, it seems like a test property, but actually, both MapRedTask and MapredLocalTask is using this property. We could either remove this property at all, and always print the log info when a MapReduce Job is failing as stated in #2, or choose another non-test related property name to determine the behavior of MapReduce Job failing logs.

        People

        • Assignee:
          Zhenxiao Luo
          Reporter:
          Carl Steinbach
        • Votes:
          0 Vote for this issue
          Watchers:
          1 Start watching this issue

          Dates

          • Created:
            Updated:

            Development