Hive
  1. Hive
  2. HIVE-4561

Column stats : LOW_VALUE (or HIGH_VALUE) will always be 0.0000 ,if all the column values larger than 0.0 (or if all column values smaller than 0.0)

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.12.0, 0.13.0
    • Fix Version/s: 0.14.0
    • Component/s: Statistics
    • Labels:
      None

      Description

      if all column values larger than 0.0 DOUBLE_LOW_VALUE always will be 0.0
      or if all column values less than 0.0, DOUBLE_HIGH_VALUE will always be

      hive (default)> create table src_test (price double);
      hive (default)> load data local inpath './test.txt' into table src_test;
      hive (default)> select * from src_test;
      OK
      1.0
      2.0
      3.0
      Time taken: 0.313 seconds, Fetched: 3 row(s)
      hive (default)> analyze table src_test compute statistics for columns price;

      mysql> select * from TAB_COL_STATS \G;
      CS_ID: 16
      DB_NAME: default
      TABLE_NAME: src_test
      COLUMN_NAME: price
      COLUMN_TYPE: double
      TBL_ID: 2586
      LONG_LOW_VALUE: 0
      LONG_HIGH_VALUE: 0
      DOUBLE_LOW_VALUE: 0.0000 # Wrong Result ! Expected is 1.0000
      DOUBLE_HIGH_VALUE: 3.0000
      BIG_DECIMAL_LOW_VALUE: NULL
      BIG_DECIMAL_HIGH_VALUE: NULL
      NUM_NULLS: 0
      NUM_DISTINCTS: 1
      AVG_COL_LEN: 0.0000
      MAX_COL_LEN: 0
      NUM_TRUES: 0
      NUM_FALSES: 0
      LAST_ANALYZED: 1368596151
      2 rows in set (0.00 sec)

      1. HIVE-4561.1.patch
        2 kB
        Zhuoluo (Clark) Yang
      2. HIVE-4561.2.patch
        2 kB
        Zhuoluo (Clark) Yang
      3. HIVE-4561.3.patch
        3 kB
        Zhuoluo (Clark) Yang
      4. HIVE-4561.4.patch.txt
        79 kB
        Navis
      5. HIVE-4561.5.patch.txt
        129 kB
        Navis
      6. HIVE-4561.6.patch.txt
        131 kB
        Navis
      7. HIVE-4561.7.patch.txt
        132 kB
        Navis
      8. HIVE-4561.8.patch.txt
        157 kB
        Navis
      9. HIVE-4561.9.patch.txt
        133 kB
        Navis

        Issue Links

          Activity

          caofangkun created issue -
          caofangkun made changes -
          Field Original Value New Value
          Link This issue is related to HIVE-1362 [ HIVE-1362 ]
          caofangkun made changes -
          Description if all column values larger than 0.0 DOUBLE_LOW_VALUE always will be 0.0
          or if all column values less than 0.0, DOUBLE_HIGH_VALUE will always be

          hive (default)> create table src_test (price double);
          hive (default)> load data local inpath './test.txt' into table src_test;
          hive (default)> select * from src_test;
          OK
          1.0
          2.0
          3.0
          Time taken: 0.313 seconds, Fetched: 3 row(s)
          hive (default)> analyze table src_test compute statistics for columns price;

          mysql> select * from TAB_COL_STATS \G;
          *************************** 1. row ***************************
                           CS_ID: 16
                         DB_NAME: default
                      TABLE_NAME: src_test
                     COLUMN_NAME: price
                     COLUMN_TYPE: double
                          TBL_ID: 2586
                  LONG_LOW_VALUE: 0
                 LONG_HIGH_VALUE: 0
                DOUBLE_LOW_VALUE: 0.0000 # Wrong Result ! Expected is 1.0000
               DOUBLE_HIGH_VALUE: 3.0000
           BIG_DECIMAL_LOW_VALUE: NULL
          BIG_DECIMAL_HIGH_VALUE: NULL
                       NUM_NULLS: 0
                   NUM_DISTINCTS: 1
                     AVG_COL_LEN: 0.0000
                     MAX_COL_LEN: 0
                       NUM_TRUES: 0
                      NUM_FALSES: 0
                   LAST_ANALYZED: 1368596151
          2 rows in set (0.00 sec)
          if all column values larger than 0.0 DOUBLE_LOW_VALUE always will be 0.0
          or if all column values less than 0.0, DOUBLE_HIGH_VALUE will always be

          hive (default)> create table src_test (price double);
          hive (default)> load data local inpath './test.txt' into table src_test;
          hive (default)> select * from src_test;
          OK
          1.0
          2.0
          3.0
          Time taken: 0.313 seconds, Fetched: 3 row(s)
          hive (default)> analyze table src_test compute statistics for columns price;

          mysql> select * from TAB_COL_STATS \G;
                           CS_ID: 16
                         DB_NAME: default
                      TABLE_NAME: src_test
                     COLUMN_NAME: price
                     COLUMN_TYPE: double
                          TBL_ID: 2586
                  LONG_LOW_VALUE: 0
                 LONG_HIGH_VALUE: 0
                DOUBLE_LOW_VALUE: 0.0000 # Wrong Result ! Expected is 1.0000
               DOUBLE_HIGH_VALUE: 3.0000
           BIG_DECIMAL_LOW_VALUE: NULL
          BIG_DECIMAL_HIGH_VALUE: NULL
                       NUM_NULLS: 0
                   NUM_DISTINCTS: 1
                     AVG_COL_LEN: 0.0000
                     MAX_COL_LEN: 0
                       NUM_TRUES: 0
                      NUM_FALSES: 0
                   LAST_ANALYZED: 1368596151
          2 rows in set (0.00 sec)
          Zhuoluo (Clark) Yang made changes -
          Priority Minor [ 4 ] Major [ 3 ]
          Hide
          Zhuoluo (Clark) Yang added a comment -

          A quick fix, would any body assign the issue to me?

          Show
          Zhuoluo (Clark) Yang added a comment - A quick fix, would any body assign the issue to me?
          Zhuoluo (Clark) Yang made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Hide
          Zhuoluo (Clark) Yang added a comment -

          A quick fix.

          Show
          Zhuoluo (Clark) Yang added a comment - A quick fix.
          Zhuoluo (Clark) Yang made changes -
          Attachment HIVE-4561.1.patch [ 12583290 ]
          Show
          Zhuoluo (Clark) Yang added a comment - https://reviews.apache.org/r/11172/
          Ashutosh Chauhan made changes -
          Assignee Zhuoluo (Clark) Yang [ clarkyzl ]
          Hide
          Ashutosh Chauhan added a comment -

          Assigned to Zhuoluo (Clark) Yang. Can you also take a look at review comments?

          Show
          Ashutosh Chauhan added a comment - Assigned to Zhuoluo (Clark) Yang . Can you also take a look at review comments?
          Hide
          Zhuoluo (Clark) Yang added a comment -

          Update a new patch.
          In case of all the long values are positive, we can get the right min. In case of all the values are negative, we can get the right max.
          UT "compute_stats_long.q" reads values from data/files/int.txt which values are all above zero. Original ut computes the min value "0", however, the correct min value is "4". This patch fixes the bug.

          Show
          Zhuoluo (Clark) Yang added a comment - Update a new patch. In case of all the long values are positive, we can get the right min. In case of all the values are negative, we can get the right max. UT "compute_stats_long.q" reads values from data/files/int.txt which values are all above zero. Original ut computes the min value "0", however, the correct min value is "4". This patch fixes the bug.
          Zhuoluo (Clark) Yang made changes -
          Attachment HIVE-4561.2.patch [ 12585794 ]
          Hide
          Zhuoluo (Clark) Yang added a comment -

          Hi, Ashutosh Chauhan, Thanks for your comments. I've updated a new patch on the reveiwboard.

          Show
          Zhuoluo (Clark) Yang added a comment - Hi, Ashutosh Chauhan , Thanks for your comments. I've updated a new patch on the reveiwboard.
          Hide
          Ashutosh Chauhan added a comment -

          Shreepadma Venugopalan Since you wrote this originally, would you like to review this as well ?

          Show
          Ashutosh Chauhan added a comment - Shreepadma Venugopalan Since you wrote this originally, would you like to review this as well ?
          Hide
          Shreepadma Venugopalan added a comment -

          Ashutosh Chauhan: Sure, I'll take a look at this today.

          Show
          Shreepadma Venugopalan added a comment - Ashutosh Chauhan : Sure, I'll take a look at this today.
          Hide
          Shreepadma Venugopalan added a comment -

          LGTM! +1 (non-binding).

          Show
          Shreepadma Venugopalan added a comment - LGTM! +1 (non-binding).
          Hide
          Ashutosh Chauhan added a comment -

          Test compute_stats_empty_table.q failed.

          Show
          Ashutosh Chauhan added a comment - Test compute_stats_empty_table.q failed.
          Ashutosh Chauhan made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Hide
          Zhuoluo (Clark) Yang added a comment -

          fix compute_stats_empty_table.q test results.

          Show
          Zhuoluo (Clark) Yang added a comment - fix compute_stats_empty_table.q test results.
          Zhuoluo (Clark) Yang made changes -
          Attachment HIVE-4561.3.patch [ 12586100 ]
          Zhuoluo (Clark) Yang made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Hide
          Ashutosh Chauhan added a comment -

          Is that correct? For empty table "min":9223372036854775807,"max":-9223372036854775808 doesn't sound right.

          Show
          Ashutosh Chauhan added a comment - Is that correct? For empty table "min":9223372036854775807,"max":-9223372036854775808 doesn't sound right.
          Hide
          Zhuoluo (Clark) Yang added a comment -

          Ashutosh Chauhan The values sounds quite strange, I will try to make a new patch.

          Show
          Zhuoluo (Clark) Yang added a comment - Ashutosh Chauhan The values sounds quite strange, I will try to make a new patch.
          Zhuoluo (Clark) Yang made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Hide
          Ashutosh Chauhan added a comment -

          If there are no rows in table, low and high values should really be null (or NaN) and not 0.

          Show
          Ashutosh Chauhan added a comment - If there are no rows in table, low and high values should really be null (or NaN) and not 0.
          Hide
          Zhuoluo (Clark) Yang added a comment -

          Update patch, make HIGH/LOW values of empty tables return null.

          Show
          Zhuoluo (Clark) Yang added a comment - Update patch, make HIGH/LOW values of empty tables return null.
          Zhuoluo (Clark) Yang made changes -
          Attachment HIVE-4561.4.patch [ 12586314 ]
          Zhuoluo (Clark) Yang made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Hide
          Ashutosh Chauhan added a comment -

          +1

          Show
          Ashutosh Chauhan added a comment - +1
          Hide
          Ashutosh Chauhan added a comment -

          Now, columnstats_tbllvl.q failed with following exception:

             [junit] java.lang.NullPointerException
              [junit]     at org.apache.hadoop.hive.serde2.lazy.objectinspector.primitive.LazyLongObjectInspector.get(LazyLongObjectInspector.java:38)
              [junit]     at org.apache.hadoop.hive.ql.exec.ColumnStatsTask.unpackLongStats(ColumnStatsTask.java:126)
              [junit]     at org.apache.hadoop.hive.ql.exec.ColumnStatsTask.unpackPrimitiveObject(ColumnStatsTask.java:196)
              [junit]     at org.apache.hadoop.hive.ql.exec.ColumnStatsTask.unpackStructObject(ColumnStatsTask.java:224)
              [junit]     at org.apache.hadoop.hive.ql.exec.ColumnStatsTask.constructColumnStatsFromPackedRow(ColumnStatsTask.java:263)
              [junit]     at org.apache.hadoop.hive.ql.exec.ColumnStatsTask.persistTableStats(ColumnStatsTask.java:327)
              [junit]     at org.apache.hadoop.hive.ql.exec.ColumnStatsTask.execute(ColumnStatsTask.java:343)
              [junit]     at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:145)
              [junit]     at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:65)
              [junit]     at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1355)
              [junit]     at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1139)
              [junit]     at org.apache.hadoop.hive.ql.Driver.run(Driver.java:945)
              [junit]     at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:259)
              [junit]     at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:216)
              [junit]     at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:413)
              [junit]     at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:348)
              [junit]     at org.apache.hadoop.hive.ql.QTestUtil.executeClient(QTestUtil.java:790)
              [junit]     at org.apache.hadoop.hive.cli.TestCliDriver.runTest(TestCliDriver.java:6279)
              [junit]     at org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_columnstats_tbllvl(TestCliDriver.java:1971)
          
          Show
          Ashutosh Chauhan added a comment - Now, columnstats_tbllvl.q failed with following exception: [junit] java.lang.NullPointerException [junit] at org.apache.hadoop.hive.serde2.lazy.objectinspector.primitive.LazyLongObjectInspector.get(LazyLongObjectInspector.java:38) [junit] at org.apache.hadoop.hive.ql.exec.ColumnStatsTask.unpackLongStats(ColumnStatsTask.java:126) [junit] at org.apache.hadoop.hive.ql.exec.ColumnStatsTask.unpackPrimitiveObject(ColumnStatsTask.java:196) [junit] at org.apache.hadoop.hive.ql.exec.ColumnStatsTask.unpackStructObject(ColumnStatsTask.java:224) [junit] at org.apache.hadoop.hive.ql.exec.ColumnStatsTask.constructColumnStatsFromPackedRow(ColumnStatsTask.java:263) [junit] at org.apache.hadoop.hive.ql.exec.ColumnStatsTask.persistTableStats(ColumnStatsTask.java:327) [junit] at org.apache.hadoop.hive.ql.exec.ColumnStatsTask.execute(ColumnStatsTask.java:343) [junit] at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:145) [junit] at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:65) [junit] at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1355) [junit] at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1139) [junit] at org.apache.hadoop.hive.ql.Driver.run(Driver.java:945) [junit] at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:259) [junit] at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:216) [junit] at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:413) [junit] at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:348) [junit] at org.apache.hadoop.hive.ql.QTestUtil.executeClient(QTestUtil.java:790) [junit] at org.apache.hadoop.hive.cli.TestCliDriver.runTest(TestCliDriver.java:6279) [junit] at org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_columnstats_tbllvl(TestCliDriver.java:1971)
          Hide
          Zhuoluo (Clark) Yang added a comment -

          Ashutosh Chauhan I think it happens when we try to persist a null max/min,I think the simplest way is to leave it empty in the ColumnStatsTask. I will try to make a new patch and make a full test.

          Show
          Zhuoluo (Clark) Yang added a comment - Ashutosh Chauhan I think it happens when we try to persist a null max/min,I think the simplest way is to leave it empty in the ColumnStatsTask. I will try to make a new patch and make a full test.
          Hide
          Shreepadma Venugopalan added a comment -

          Zhuoluo (Clark) Yang: I'm not sure I understand the fix here. Can you please elaborate on what it means to leaving it empty in the ColumnStatsTask? Thanks!

          Show
          Shreepadma Venugopalan added a comment - Zhuoluo (Clark) Yang : I'm not sure I understand the fix here. Can you please elaborate on what it means to leaving it empty in the ColumnStatsTask? Thanks!
          Hide
          Zhuoluo (Clark) Yang added a comment -

          Shreepadma Venugopalan I think I am wrong. Originally, I want to "return" like this:

          @@ -189,6 +187,11 @@
                   statsObj.setStatsData(statsData);
                 }
               } else {
          +      // Any null object, such as min/max value of an empty table,
          +      // need not be unpacked.
          +      if (o == null) {
          +        return;
          +      }
                 // invoke the right unpack method depending on data type of the column
                 if (statsObj.getStatsData().isSetBooleanStats()) {
                   unpackBooleanStats(oi, o, fieldName, statsObj);
          

          However, I've found that LongColumnStatsData.highValue is required by thrift. And also modifications of ObjectStore is required and checks LongColumnStatsData.isSetHighValue(). Any suggestions? Thanks!

          Show
          Zhuoluo (Clark) Yang added a comment - Shreepadma Venugopalan I think I am wrong. Originally, I want to "return" like this: @@ -189,6 +187,11 @@ statsObj.setStatsData(statsData); } } else { + // Any null object, such as min/max value of an empty table, + // need not be unpacked. + if (o == null ) { + return ; + } // invoke the right unpack method depending on data type of the column if (statsObj.getStatsData().isSetBooleanStats()) { unpackBooleanStats(oi, o, fieldName, statsObj); However, I've found that LongColumnStatsData.highValue is required by thrift. And also modifications of ObjectStore is required and checks LongColumnStatsData.isSetHighValue(). Any suggestions? Thanks!
          Hide
          Shreepadma Venugopalan added a comment -

          Zhuoluo (Clark) Yang: My suggestion is to use Long.Min/Long.Max value instead of a null value. The code that looks at column stats can use the min/max in conjunction with other stats such as number of rows etc. to infer that the values are initialization values for min/max and not true values that represent the bounds on the column.

          Show
          Shreepadma Venugopalan added a comment - Zhuoluo (Clark) Yang : My suggestion is to use Long.Min/Long.Max value instead of a null value. The code that looks at column stats can use the min/max in conjunction with other stats such as number of rows etc. to infer that the values are initialization values for min/max and not true values that represent the bounds on the column.
          Zhuoluo (Clark) Yang made changes -
          Attachment HIVE-4561.4.patch [ 12586314 ]
          Hide
          Zhuoluo (Clark) Yang added a comment -

          Shreepadma Venugopalan It seems null value will involve a lot of modification, such as ColumnStatsTask or ObjectStore or thrift files.
          Currently, statistics are looked up by code, I think it make sense to keep Long.Min/Long.Max.
          And we can apply HIVE-4561.3.patch instead of HIVE-4561.4.patch.

          Show
          Zhuoluo (Clark) Yang added a comment - Shreepadma Venugopalan It seems null value will involve a lot of modification, such as ColumnStatsTask or ObjectStore or thrift files. Currently, statistics are looked up by code, I think it make sense to keep Long.Min/Long.Max. And we can apply HIVE-4561 .3.patch instead of HIVE-4561 .4.patch.
          Ashutosh Chauhan made changes -
          Assignee Zhuoluo (Clark) Yang [ clarkyzl ] Ashutosh Chauhan [ ashutoshc ]
          Ashutosh Chauhan made changes -
          Assignee Ashutosh Chauhan [ ashutoshc ] caofangkun [ caofangkun ]
          Hide
          Hive QA added a comment -

          Overall: -1 no tests executed

          Here are the results of testing the latest attachment:
          https://issues.apache.org/jira/secure/attachment/12586100/HIVE-4561.3.patch

          Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/951/testReport
          Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/951/console

          Messages:

          Executing org.apache.hive.ptest.execution.PrepPhase
          Tests failed with: NonZeroExitCodeException: Command 'bash /data/hive-ptest/working/scratch/source-prep.sh' failed with exit status 1 and output '+ [[ -n '' ]]
          + export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m -Dhttp.proxyHost=localhost -Dhttp.proxyPort=3128'
          + ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m -Dhttp.proxyHost=localhost -Dhttp.proxyPort=3128'
          + cd /data/hive-ptest/working/
          + tee /data/hive-ptest/logs/PreCommit-HIVE-Build-951/source-prep.txt
          + mkdir -p maven ivy
          + [[ svn = \s\v\n ]]
          + [[ -n '' ]]
          + [[ -d apache-svn-trunk-source ]]
          + [[ ! -d apache-svn-trunk-source/.svn ]]
          + [[ ! -d apache-svn-trunk-source ]]
          + cd apache-svn-trunk-source
          + svn revert -R .
          ++ awk '{print $2}'
          ++ egrep -v '^X|^Performing status on external'
          ++ svn status --no-ignore
          + rm -rf
          + svn update
          
          Fetching external item into 'hcatalog/src/test/e2e/harness'
          External at revision 1527155.
          
          At revision 1527155.
          + patchCommandPath=/data/hive-ptest/working/scratch/smart-apply-patch.sh
          + patchFilePath=/data/hive-ptest/working/scratch/build.patch
          + [[ -f /data/hive-ptest/working/scratch/build.patch ]]
          + chmod +x /data/hive-ptest/working/scratch/smart-apply-patch.sh
          + /data/hive-ptest/working/scratch/smart-apply-patch.sh /data/hive-ptest/working/scratch/build.patch
          The patch does not appear to apply with p0 to p2
          + exit 1
          '
          

          This message is automatically generated.

          Show
          Hive QA added a comment - Overall : -1 no tests executed Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12586100/HIVE-4561.3.patch Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/951/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/951/console Messages: Executing org.apache.hive.ptest.execution.PrepPhase Tests failed with: NonZeroExitCodeException: Command 'bash /data/hive-ptest/working/scratch/source-prep.sh' failed with exit status 1 and output '+ [[ -n '' ]] + export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m -Dhttp.proxyHost=localhost -Dhttp.proxyPort=3128' + ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m -Dhttp.proxyHost=localhost -Dhttp.proxyPort=3128' + cd /data/hive-ptest/working/ + tee /data/hive-ptest/logs/PreCommit-HIVE-Build-951/source-prep.txt + mkdir -p maven ivy + [[ svn = \s\v\n ]] + [[ -n '' ]] + [[ -d apache-svn-trunk-source ]] + [[ ! -d apache-svn-trunk-source/.svn ]] + [[ ! -d apache-svn-trunk-source ]] + cd apache-svn-trunk-source + svn revert -R . ++ awk '{print $2}' ++ egrep -v '^X|^Performing status on external' ++ svn status --no-ignore + rm -rf + svn update Fetching external item into 'hcatalog/src/test/e2e/harness' External at revision 1527155. At revision 1527155. + patchCommandPath=/data/hive-ptest/working/scratch/smart-apply-patch.sh + patchFilePath=/data/hive-ptest/working/scratch/build.patch + [[ -f /data/hive-ptest/working/scratch/build.patch ]] + chmod +x /data/hive-ptest/working/scratch/smart-apply-patch.sh + /data/hive-ptest/working/scratch/smart-apply-patch.sh /data/hive-ptest/working/scratch/build.patch The patch does not appear to apply with p0 to p2 + exit 1 ' This message is automatically generated.
          Ashutosh Chauhan made changes -
          Link This issue relates to HIVE-7060 [ HIVE-7060 ]
          Hide
          Navis added a comment -

          Changed to allow null for min/max value of long/double/decimal stats. Let's see what will happen.

          Show
          Navis added a comment - Changed to allow null for min/max value of long/double/decimal stats. Let's see what will happen.
          Navis made changes -
          Attachment HIVE-4561.4.patch.txt [ 12645973 ]
          Hide
          Hive QA added a comment -

          Overall: -1 at least one tests failed

          Here are the results of testing the latest attachment:
          https://issues.apache.org/jira/secure/attachment/12645973/HIVE-4561.4.patch.txt

          ERROR: -1 due to 13 failed/errored test(s), 5451 tests executed
          Failed tests:

          org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_annotate_stats_groupby
          org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_annotate_stats_table
          org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_metadata_only_queries
          org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_root_dir_external_table
          org.apache.hadoop.hive.conf.TestHiveConf.testConfProperties
          org.apache.hive.hcatalog.pig.TestOrcHCatPigStorer.testWriteDecimal
          org.apache.hive.hcatalog.pig.TestOrcHCatPigStorer.testWriteDecimalX
          org.apache.hive.hcatalog.pig.TestOrcHCatPigStorer.testWriteDecimalXY
          org.apache.hive.hcatalog.templeton.TestWebHCatE2e.getHadoopVersion
          org.apache.hive.hcatalog.templeton.TestWebHCatE2e.getHiveVersion
          org.apache.hive.hcatalog.templeton.TestWebHCatE2e.getPigVersion
          org.apache.hive.hcatalog.templeton.TestWebHCatE2e.getStatus
          org.apache.hive.hcatalog.templeton.TestWebHCatE2e.invalidPath
          

          Test results: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-Build/259/testReport
          Console output: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-Build/259/console
          Test logs: http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-Build-259/

          Messages:

          Executing org.apache.hive.ptest.execution.PrepPhase
          Executing org.apache.hive.ptest.execution.ExecutionPhase
          Executing org.apache.hive.ptest.execution.ReportingPhase
          Tests exited with: TestsFailedException: 13 tests failed
          

          This message is automatically generated.

          ATTACHMENT ID: 12645973

          Show
          Hive QA added a comment - Overall : -1 at least one tests failed Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12645973/HIVE-4561.4.patch.txt ERROR: -1 due to 13 failed/errored test(s), 5451 tests executed Failed tests: org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_annotate_stats_groupby org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_annotate_stats_table org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_metadata_only_queries org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_root_dir_external_table org.apache.hadoop.hive.conf.TestHiveConf.testConfProperties org.apache.hive.hcatalog.pig.TestOrcHCatPigStorer.testWriteDecimal org.apache.hive.hcatalog.pig.TestOrcHCatPigStorer.testWriteDecimalX org.apache.hive.hcatalog.pig.TestOrcHCatPigStorer.testWriteDecimalXY org.apache.hive.hcatalog.templeton.TestWebHCatE2e.getHadoopVersion org.apache.hive.hcatalog.templeton.TestWebHCatE2e.getHiveVersion org.apache.hive.hcatalog.templeton.TestWebHCatE2e.getPigVersion org.apache.hive.hcatalog.templeton.TestWebHCatE2e.getStatus org.apache.hive.hcatalog.templeton.TestWebHCatE2e.invalidPath Test results: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-Build/259/testReport Console output: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-Build/259/console Test logs: http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-Build-259/ Messages: Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 13 tests failed This message is automatically generated. ATTACHMENT ID: 12645973
          Hide
          Ashutosh Chauhan added a comment -

          metadata_only_queries.q update seems to be correct now. But, annotate_stats_groupby.q & annotate_stats_table.q seems like failures with NPE that needs to be looked at.

          Show
          Ashutosh Chauhan added a comment - metadata_only_queries.q update seems to be correct now. But, annotate_stats_groupby.q & annotate_stats_table.q seems like failures with NPE that needs to be looked at.
          Ashutosh Chauhan made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Assignee caofangkun [ caofangkun ] Navis [ navis ]
          Navis made changes -
          Attachment HIVE-4561.5.patch.txt [ 12646662 ]
          Navis made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Navis made changes -
          Remote Link This issue links to "review board (Web Link)" [ 15214 ]
          Hide
          Ashutosh Chauhan added a comment -

          +1

          Show
          Ashutosh Chauhan added a comment - +1
          Hide
          Hive QA added a comment -

          Overall: -1 at least one tests failed

          Here are the results of testing the latest attachment:
          https://issues.apache.org/jira/secure/attachment/12646662/HIVE-4561.5.patch.txt

          ERROR: -1 due to 11 failed/errored test(s), 5462 tests executed
          Failed tests:

          org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_columnstats_tbllvl
          org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_compute_stats_empty_table
          org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_metadata_only_queries_with_filters
          org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_parquet_decimal1
          org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_root_dir_external_table
          org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_schemeAuthority2
          org.apache.hive.hcatalog.pig.TestHCatLoader.testReadDataPrimitiveTypes
          org.apache.hive.hcatalog.pig.TestOrcHCatPigStorer.testWriteDecimal
          org.apache.hive.hcatalog.pig.TestOrcHCatPigStorer.testWriteDecimalX
          org.apache.hive.hcatalog.pig.TestOrcHCatPigStorer.testWriteDecimalXY
          org.apache.hive.jdbc.miniHS2.TestHiveServer2.testConnection
          

          Test results: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-Build/291/testReport
          Console output: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-Build/291/console
          Test logs: http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-Build-291/

          Messages:

          Executing org.apache.hive.ptest.execution.PrepPhase
          Executing org.apache.hive.ptest.execution.ExecutionPhase
          Executing org.apache.hive.ptest.execution.ReportingPhase
          Tests exited with: TestsFailedException: 11 tests failed
          

          This message is automatically generated.

          ATTACHMENT ID: 12646662

          Show
          Hive QA added a comment - Overall : -1 at least one tests failed Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12646662/HIVE-4561.5.patch.txt ERROR: -1 due to 11 failed/errored test(s), 5462 tests executed Failed tests: org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_columnstats_tbllvl org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_compute_stats_empty_table org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_metadata_only_queries_with_filters org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_parquet_decimal1 org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_root_dir_external_table org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_schemeAuthority2 org.apache.hive.hcatalog.pig.TestHCatLoader.testReadDataPrimitiveTypes org.apache.hive.hcatalog.pig.TestOrcHCatPigStorer.testWriteDecimal org.apache.hive.hcatalog.pig.TestOrcHCatPigStorer.testWriteDecimalX org.apache.hive.hcatalog.pig.TestOrcHCatPigStorer.testWriteDecimalXY org.apache.hive.jdbc.miniHS2.TestHiveServer2.testConnection Test results: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-Build/291/testReport Console output: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-Build/291/console Test logs: http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-Build-291/ Messages: Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 11 tests failed This message is automatically generated. ATTACHMENT ID: 12646662
          Hide
          Ashutosh Chauhan added a comment -

          Failures need to be looked at

          Show
          Ashutosh Chauhan added a comment - Failures need to be looked at
          Ashutosh Chauhan made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Affects Version/s 0.13.0 [ 12324986 ]
          Navis made changes -
          Attachment HIVE-4561.6.patch.txt [ 12646727 ]
          Hide
          Navis added a comment -

          Fixed result of metadata_only_queries_with_filters.q, but cannot reproduce other failures. Rerunning test.

          Show
          Navis added a comment - Fixed result of metadata_only_queries_with_filters.q, but cannot reproduce other failures. Rerunning test.
          Navis made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Hide
          Ashutosh Chauhan added a comment -

          Both testCliDriver_columnstats_tbllvl.q & testCliDriver_compute_stats_empty_table.q failed for me on latest patch with following exception. which is same as Hive QA run reported on http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-Build/291/testReport/org.apache.hadoop.hive.cli/TestCliDriver/testCliDriver_columnstats_tbllvl/

          java.lang.Exception: java.lang.RuntimeException: Hive Runtime Error while closing operators
                  at org.apache.hadoop.mapred.LocalJobRunner$Job.runTasks(LocalJobRunner.java:462)
                  at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:522)
          Caused by: java.lang.RuntimeException: Hive Runtime Error while closing operators
                  at org.apache.hadoop.hive.ql.exec.mr.ExecMapper.close(ExecMapper.java:260)
                  at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:61)
                  at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:430)
                  at org.apache.hadoop.mapred.MapTask.run(MapTask.java:342)
                  at org.apache.hadoop.mapred.LocalJobRunner$Job$MapTaskRunnable.run(LocalJobRunner.java:243)
                  at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
                  at java.util.concurrent.FutureTask.run(FutureTask.java:262)
                  at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
                  at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
                  at java.lang.Thread.run(Thread.java:745)
          Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.NullPointerException
                  at org.apache.hadoop.hive.ql.exec.GroupByOperator.closeOp(GroupByOperator.java:1141)
                  at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:579)
                  at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:591)
                  at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:591)
                  at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:591)
                  at org.apache.hadoop.hive.ql.exec.mr.ExecMapper.close(ExecMapper.java:227)
                  ... 9 more
          Caused by: java.lang.NullPointerException
                  at org.apache.hadoop.hive.serde2.objectinspector.primitive.WritableIntObjectInspector.get(WritableIntObjectInspector.java:36)
                  at org.apache.hadoop.hive.serde2.objectinspector.primitive.PrimitiveObjectInspectorUtils.getInt(PrimitiveObjectInspectorUtils.java:598)
                  at org.apache.hadoop.hive.ql.udf.generic.GenericUDAFComputeStats$GenericUDAFLongStatsEvaluator.iterate(GenericUDAFComputeStats.java:468)
                  at org.apache.hadoop.hive.ql.udf.generic.GenericUDAFEvaluator.aggregate(GenericUDAFEvaluator.java:183)
                  at org.apache.hadoop.hive.ql.exec.GroupByOperator.closeOp(GroupByOperator.java:1132)
          
          Show
          Ashutosh Chauhan added a comment - Both testCliDriver_columnstats_tbllvl.q & testCliDriver_compute_stats_empty_table.q failed for me on latest patch with following exception. which is same as Hive QA run reported on http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-Build/291/testReport/org.apache.hadoop.hive.cli/TestCliDriver/testCliDriver_columnstats_tbllvl/ java.lang.Exception: java.lang.RuntimeException: Hive Runtime Error while closing operators at org.apache.hadoop.mapred.LocalJobRunner$Job.runTasks(LocalJobRunner.java:462) at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:522) Caused by: java.lang.RuntimeException: Hive Runtime Error while closing operators at org.apache.hadoop.hive.ql.exec.mr.ExecMapper.close(ExecMapper.java:260) at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:61) at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:430) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:342) at org.apache.hadoop.mapred.LocalJobRunner$Job$MapTaskRunnable.run(LocalJobRunner.java:243) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) at java.util.concurrent.FutureTask.run(FutureTask.java:262) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang. Thread .run( Thread .java:745) Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.NullPointerException at org.apache.hadoop.hive.ql.exec.GroupByOperator.closeOp(GroupByOperator.java:1141) at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:579) at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:591) at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:591) at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:591) at org.apache.hadoop.hive.ql.exec.mr.ExecMapper.close(ExecMapper.java:227) ... 9 more Caused by: java.lang.NullPointerException at org.apache.hadoop.hive.serde2.objectinspector.primitive.WritableIntObjectInspector.get(WritableIntObjectInspector.java:36) at org.apache.hadoop.hive.serde2.objectinspector.primitive.PrimitiveObjectInspectorUtils.getInt(PrimitiveObjectInspectorUtils.java:598) at org.apache.hadoop.hive.ql.udf. generic .GenericUDAFComputeStats$GenericUDAFLongStatsEvaluator.iterate(GenericUDAFComputeStats.java:468) at org.apache.hadoop.hive.ql.udf. generic .GenericUDAFEvaluator.aggregate(GenericUDAFEvaluator.java:183) at org.apache.hadoop.hive.ql.exec.GroupByOperator.closeOp(GroupByOperator.java:1132)
          Hide
          Navis added a comment -

          Ok, I'll take a look.

          Show
          Navis added a comment - Ok, I'll take a look.
          Navis made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Navis made changes -
          Attachment HIVE-4561.7.patch.txt [ 12646836 ]
          Navis made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Hide
          Zhuoluo (Clark) Yang added a comment -

          Thanks, Navis, It seems more situations needs to be consider.

          Show
          Zhuoluo (Clark) Yang added a comment - Thanks, Navis , It seems more situations needs to be consider.
          Hide
          Hive QA added a comment -

          Overall: -1 at least one tests failed

          Here are the results of testing the latest attachment:
          https://issues.apache.org/jira/secure/attachment/12646836/HIVE-4561.7.patch.txt

          ERROR: -1 due to 16 failed/errored test(s), 5466 tests executed
          Failed tests:

          org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_list_bucket_dml_2
          org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_nullgroup
          org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_nullgroup4
          org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_nullgroup4_multi_distinct
          org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_orc_ppd_decimal
          org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_parquet_decimal1
          org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_partition_wise_fileformat6
          org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_udf_count
          org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_root_dir_external_table
          org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_schemeAuthority2
          org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_authorization_ctas
          org.apache.hadoop.hive.ql.exec.tez.TestTezTask.testSubmit
          org.apache.hive.hcatalog.pig.TestOrcHCatLoader.testReadDataPrimitiveTypes
          org.apache.hive.hcatalog.pig.TestOrcHCatPigStorer.testWriteDecimal
          org.apache.hive.hcatalog.pig.TestOrcHCatPigStorer.testWriteDecimalX
          org.apache.hive.hcatalog.pig.TestOrcHCatPigStorer.testWriteDecimalXY
          

          Test results: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-Build/313/testReport
          Console output: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-Build/313/console
          Test logs: http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-Build-313/

          Messages:

          Executing org.apache.hive.ptest.execution.PrepPhase
          Executing org.apache.hive.ptest.execution.ExecutionPhase
          Executing org.apache.hive.ptest.execution.ReportingPhase
          Tests exited with: TestsFailedException: 16 tests failed
          

          This message is automatically generated.

          ATTACHMENT ID: 12646836

          Show
          Hive QA added a comment - Overall : -1 at least one tests failed Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12646836/HIVE-4561.7.patch.txt ERROR: -1 due to 16 failed/errored test(s), 5466 tests executed Failed tests: org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_list_bucket_dml_2 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_nullgroup org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_nullgroup4 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_nullgroup4_multi_distinct org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_orc_ppd_decimal org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_parquet_decimal1 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_partition_wise_fileformat6 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_udf_count org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_root_dir_external_table org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_schemeAuthority2 org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_authorization_ctas org.apache.hadoop.hive.ql.exec.tez.TestTezTask.testSubmit org.apache.hive.hcatalog.pig.TestOrcHCatLoader.testReadDataPrimitiveTypes org.apache.hive.hcatalog.pig.TestOrcHCatPigStorer.testWriteDecimal org.apache.hive.hcatalog.pig.TestOrcHCatPigStorer.testWriteDecimalX org.apache.hive.hcatalog.pig.TestOrcHCatPigStorer.testWriteDecimalXY Test results: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-Build/313/testReport Console output: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-Build/313/console Test logs: http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-Build-313/ Messages: Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 16 tests failed This message is automatically generated. ATTACHMENT ID: 12646836
          Navis made changes -
          Attachment HIVE-4561.8.patch.txt [ 12647045 ]
          Hide
          Ashutosh Chauhan added a comment -

          Can you also update RB with latest patch ?

          Show
          Ashutosh Chauhan added a comment - Can you also update RB with latest patch ?
          Hide
          Hive QA added a comment -

          Overall: -1 no tests executed

          Here are the results of testing the latest attachment:
          https://issues.apache.org/jira/secure/attachment/12647045/HIVE-4561.8.patch.txt

          Test results: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-Build/323/testReport
          Console output: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-Build/323/console
          Test logs: http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-Build-323/

          Messages:

          Executing org.apache.hive.ptest.execution.PrepPhase
          Tests exited with: NonZeroExitCodeException
          Command 'bash /data/hive-ptest/working/scratch/source-prep.sh' failed with exit status 1 and output '+ [[ -n /usr/java/jdk1.7.0_45-cloudera ]]
          + export JAVA_HOME=/usr/java/jdk1.7.0_45-cloudera
          + JAVA_HOME=/usr/java/jdk1.7.0_45-cloudera
          + export PATH=/usr/java/jdk1.7.0_45-cloudera/bin/:/usr/java/jdk1.6.0_34/bin:/usr/local/apache-maven-3.0.5/bin:/usr/local/apache-maven-3.0.5/bin:/usr/java/jdk1.6.0_34/bin:/usr/local/apache-ant-1.9.1/bin:/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin:/home/hiveptest/bin
          + PATH=/usr/java/jdk1.7.0_45-cloudera/bin/:/usr/java/jdk1.6.0_34/bin:/usr/local/apache-maven-3.0.5/bin:/usr/local/apache-maven-3.0.5/bin:/usr/java/jdk1.6.0_34/bin:/usr/local/apache-ant-1.9.1/bin:/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin:/home/hiveptest/bin
          + export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m '
          + ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m '
          + export 'M2_OPTS=-Xmx1g -XX:MaxPermSize=256m -Dhttp.proxyHost=localhost -Dhttp.proxyPort=3128'
          + M2_OPTS='-Xmx1g -XX:MaxPermSize=256m -Dhttp.proxyHost=localhost -Dhttp.proxyPort=3128'
          + cd /data/hive-ptest/working/
          + tee /data/hive-ptest/logs/PreCommit-HIVE-Build-323/source-prep.txt
          + [[ false == \t\r\u\e ]]
          + mkdir -p maven ivy
          + [[ svn = \s\v\n ]]
          + [[ -n '' ]]
          + [[ -d apache-svn-trunk-source ]]
          + [[ ! -d apache-svn-trunk-source/.svn ]]
          + [[ ! -d apache-svn-trunk-source ]]
          + cd apache-svn-trunk-source
          + svn revert -R .
          Reverted 'common/src/test/org/apache/hadoop/hive/common/type/TestHiveDecimal.java'
          Reverted 'ql/src/test/org/apache/hadoop/hive/ql/io/parquet/TestHiveSchemaConverter.java'
          Reverted 'ql/src/java/org/apache/hadoop/hive/ql/io/parquet/convert/HiveSchemaConverter.java'
          Reverted 'ql/src/java/org/apache/hadoop/hive/ql/io/parquet/serde/ParquetHiveSerDe.java'
          ++ awk '{print $2}'
          ++ egrep -v '^X|^Performing status on external'
          ++ svn status --no-ignore
          + rm -rf target datanucleus.log ant/target shims/target shims/0.20/target shims/0.20S/target shims/0.23/target shims/aggregator/target shims/common/target shims/common-secure/target packaging/target hbase-handler/target testutils/target jdbc/target metastore/target data/files/dec_comp.txt itests/target itests/hcatalog-unit/target itests/test-serde/target itests/qtest/target itests/hive-minikdc/target itests/hive-unit/target itests/custom-serde/target itests/util/target hcatalog/target hcatalog/core/target hcatalog/streaming/target hcatalog/server-extensions/target hcatalog/webhcat/svr/target hcatalog/webhcat/java-client/target hcatalog/hcatalog-pig-adapter/target hwi/target common/target common/src/gen contrib/target service/target serde/target beeline/target odbc/target cli/target ql/dependency-reduced-pom.xml ql/target
          + svn update
          
          Fetching external item into 'hcatalog/src/test/e2e/harness'
          External at revision 1598190.
          
          At revision 1598190.
          + patchCommandPath=/data/hive-ptest/working/scratch/smart-apply-patch.sh
          + patchFilePath=/data/hive-ptest/working/scratch/build.patch
          + [[ -f /data/hive-ptest/working/scratch/build.patch ]]
          + chmod +x /data/hive-ptest/working/scratch/smart-apply-patch.sh
          + /data/hive-ptest/working/scratch/smart-apply-patch.sh /data/hive-ptest/working/scratch/build.patch
          The patch does not appear to apply with p0, p1, or p2
          + exit 1
          '
          

          This message is automatically generated.

          ATTACHMENT ID: 12647045

          Show
          Hive QA added a comment - Overall : -1 no tests executed Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12647045/HIVE-4561.8.patch.txt Test results: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-Build/323/testReport Console output: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-Build/323/console Test logs: http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-Build-323/ Messages: Executing org.apache.hive.ptest.execution.PrepPhase Tests exited with: NonZeroExitCodeException Command 'bash /data/hive-ptest/working/scratch/source-prep.sh' failed with exit status 1 and output '+ [[ -n /usr/java/jdk1.7.0_45-cloudera ]] + export JAVA_HOME=/usr/java/jdk1.7.0_45-cloudera + JAVA_HOME=/usr/java/jdk1.7.0_45-cloudera + export PATH=/usr/java/jdk1.7.0_45-cloudera/bin/:/usr/java/jdk1.6.0_34/bin:/usr/local/apache-maven-3.0.5/bin:/usr/local/apache-maven-3.0.5/bin:/usr/java/jdk1.6.0_34/bin:/usr/local/apache-ant-1.9.1/bin:/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin:/home/hiveptest/bin + PATH=/usr/java/jdk1.7.0_45-cloudera/bin/:/usr/java/jdk1.6.0_34/bin:/usr/local/apache-maven-3.0.5/bin:/usr/local/apache-maven-3.0.5/bin:/usr/java/jdk1.6.0_34/bin:/usr/local/apache-ant-1.9.1/bin:/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin:/home/hiveptest/bin + export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m ' + ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m ' + export 'M2_OPTS=-Xmx1g -XX:MaxPermSize=256m -Dhttp.proxyHost=localhost -Dhttp.proxyPort=3128' + M2_OPTS='-Xmx1g -XX:MaxPermSize=256m -Dhttp.proxyHost=localhost -Dhttp.proxyPort=3128' + cd /data/hive-ptest/working/ + tee /data/hive-ptest/logs/PreCommit-HIVE-Build-323/source-prep.txt + [[ false == \t\r\u\e ]] + mkdir -p maven ivy + [[ svn = \s\v\n ]] + [[ -n '' ]] + [[ -d apache-svn-trunk-source ]] + [[ ! -d apache-svn-trunk-source/.svn ]] + [[ ! -d apache-svn-trunk-source ]] + cd apache-svn-trunk-source + svn revert -R . Reverted 'common/src/test/org/apache/hadoop/hive/common/type/TestHiveDecimal.java' Reverted 'ql/src/test/org/apache/hadoop/hive/ql/io/parquet/TestHiveSchemaConverter.java' Reverted 'ql/src/java/org/apache/hadoop/hive/ql/io/parquet/convert/HiveSchemaConverter.java' Reverted 'ql/src/java/org/apache/hadoop/hive/ql/io/parquet/serde/ParquetHiveSerDe.java' ++ awk '{print $2}' ++ egrep -v '^X|^Performing status on external' ++ svn status --no-ignore + rm -rf target datanucleus.log ant/target shims/target shims/0.20/target shims/0.20S/target shims/0.23/target shims/aggregator/target shims/common/target shims/common-secure/target packaging/target hbase-handler/target testutils/target jdbc/target metastore/target data/files/dec_comp.txt itests/target itests/hcatalog-unit/target itests/test-serde/target itests/qtest/target itests/hive-minikdc/target itests/hive-unit/target itests/custom-serde/target itests/util/target hcatalog/target hcatalog/core/target hcatalog/streaming/target hcatalog/server-extensions/target hcatalog/webhcat/svr/target hcatalog/webhcat/java-client/target hcatalog/hcatalog-pig-adapter/target hwi/target common/target common/src/gen contrib/target service/target serde/target beeline/target odbc/target cli/target ql/dependency-reduced-pom.xml ql/target + svn update Fetching external item into 'hcatalog/src/test/e2e/harness' External at revision 1598190. At revision 1598190. + patchCommandPath=/data/hive-ptest/working/scratch/smart-apply-patch.sh + patchFilePath=/data/hive-ptest/working/scratch/build.patch + [[ -f /data/hive-ptest/working/scratch/build.patch ]] + chmod +x /data/hive-ptest/working/scratch/smart-apply-patch.sh + /data/hive-ptest/working/scratch/smart-apply-patch.sh /data/hive-ptest/working/scratch/build.patch The patch does not appear to apply with p0, p1, or p2 + exit 1 ' This message is automatically generated. ATTACHMENT ID: 12647045
          Navis made changes -
          Attachment HIVE-4561.9.patch.txt [ 12647283 ]
          Hide
          Ashutosh Chauhan added a comment -

          +1

          Show
          Ashutosh Chauhan added a comment - +1
          Hide
          Hive QA added a comment -

          Overall: -1 at least one tests failed

          Here are the results of testing the latest attachment:
          https://issues.apache.org/jira/secure/attachment/12647283/HIVE-4561.9.patch.txt

          ERROR: -1 due to 8 failed/errored test(s), 5467 tests executed
          Failed tests:

          org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_bucketmapjoin6
          org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_root_dir_external_table
          org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_authorization_ctas
          org.apache.hadoop.hive.common.metrics.TestMetrics.testScopeConcurrency
          org.apache.hadoop.hive.ql.exec.tez.TestTezTask.testSubmit
          org.apache.hive.hcatalog.pig.TestOrcHCatPigStorer.testWriteDecimal
          org.apache.hive.hcatalog.pig.TestOrcHCatPigStorer.testWriteDecimalX
          org.apache.hive.hcatalog.pig.TestOrcHCatPigStorer.testWriteDecimalXY
          

          Test results: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-Build/340/testReport
          Console output: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-Build/340/console
          Test logs: http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-Build-340/

          Messages:

          Executing org.apache.hive.ptest.execution.PrepPhase
          Executing org.apache.hive.ptest.execution.ExecutionPhase
          Executing org.apache.hive.ptest.execution.ReportingPhase
          Tests exited with: TestsFailedException: 8 tests failed
          

          This message is automatically generated.

          ATTACHMENT ID: 12647283

          Show
          Hive QA added a comment - Overall : -1 at least one tests failed Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12647283/HIVE-4561.9.patch.txt ERROR: -1 due to 8 failed/errored test(s), 5467 tests executed Failed tests: org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_bucketmapjoin6 org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_root_dir_external_table org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_authorization_ctas org.apache.hadoop.hive.common.metrics.TestMetrics.testScopeConcurrency org.apache.hadoop.hive.ql.exec.tez.TestTezTask.testSubmit org.apache.hive.hcatalog.pig.TestOrcHCatPigStorer.testWriteDecimal org.apache.hive.hcatalog.pig.TestOrcHCatPigStorer.testWriteDecimalX org.apache.hive.hcatalog.pig.TestOrcHCatPigStorer.testWriteDecimalXY Test results: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-Build/340/testReport Console output: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-Build/340/console Test logs: http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-Build-340/ Messages: Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 8 tests failed This message is automatically generated. ATTACHMENT ID: 12647283
          Hide
          Ashutosh Chauhan added a comment -

          Committed to trunk. Thanks, Navis!

          Show
          Ashutosh Chauhan added a comment - Committed to trunk. Thanks, Navis!
          Ashutosh Chauhan made changes -
          Status Patch Available [ 10002 ] Resolved [ 5 ]
          Fix Version/s 0.14.0 [ 12326450 ]
          Resolution Fixed [ 1 ]
          Navis made changes -
          Link This issue is duplicated by HIVE-7060 [ HIVE-7060 ]
          Hide
          Thejas M Nair added a comment -

          This has been fixed in 0.14 release. Please open new jira if you see any issues.

          Show
          Thejas M Nair added a comment - This has been fixed in 0.14 release. Please open new jira if you see any issues.
          Thejas M Nair made changes -
          Status Resolved [ 5 ] Closed [ 6 ]
          Transition Time In Source Status Execution Times Last Executer Last Execution Date
          Patch Available Patch Available Open Open
          373d 8h 51m 5 Navis 26/May/14 04:02
          Open Open Patch Available Patch Available
          3d 14h 8m 6 Navis 27/May/14 05:57
          Patch Available Patch Available Resolved Resolved
          3d 19h 4m 1 Ashutosh Chauhan 31/May/14 01:01
          Resolved Resolved Closed Closed
          166d 19h 38m 1 Thejas M Nair 13/Nov/14 19:39

            People

            • Assignee:
              Navis
              Reporter:
              caofangkun
            • Votes:
              1 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development