Hive
  1. Hive
  2. HIVE-2856

Fix TestCliDriver escape1.q failure on MR2

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.8.0
    • Fix Version/s: 0.9.0
    • Component/s: Query Processor, Tests
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      Additional '^' in escape test:

      [junit] Begin query: escape1.q
      [junit] Copying file: file:/home/cloudera/Code/hive/data/files/escapetest.txt
      [junit] 12/01/23 15:22:15 WARN conf.Configuration: mapred.system.dir is deprecated. Instead, use mapreduce.jobtracker.system.dir
      [junit] 12/01/23 15:22:15 WARN conf.Configuration: mapred.local.dir is deprecated. Instead, use mapreduce.cluster.local.dir
      [junit] diff -a -I file: -I pfile: -I hdfs: -I /tmp/ -I invalidscheme: -I lastUpdateTime -I lastAccessTime -I [Oo]wner -I CreateTime -I LastAccessTime -I Location -I LOCATION ' -I transient_lastDdlTime -I last_modified_ -I java.lang.RuntimeException -I at org -I at sun -I at java -I at junit -I Caused by: -I LOCK_QUERYID: -I LOCK_TIME: -I grantTime -I [.][.][.] [0-9]* more -I job_[0-9]_[0-9] -I USING 'java -cp /home/cloudera/Code/hive/build/ql/test/logs/clientpositive/escape1.q.out /home/cloudera/Code/hive/ql/src/test/results/clientpositive/escape1.q.out
      [junit] 893d892
      [junit] < 1 1 ^
      [junit] junit.framework.AssertionFailedError: Client execution results failed with error code = 1
      [junit] See build/ql/tmp/hive.log, or try "ant test ... -Dtest.silent=false" to get more logs.
      [junit] at junit.framework.Assert.fail(Assert.java:50)
      [junit] at org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_escape1(TestCliDriver.java:131)
      [junit] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      [junit] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
      [junit] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      [junit] at java.lang.reflect.Method.invoke(Method.java:616)
      [junit] at junit.framework.TestCase.runTest(TestCase.java:168)
      [junit] at junit.framework.TestCase.runBare(TestCase.java:134)
      [junit] at junit.framework.TestResult$1.protect(TestResult.java:110)
      [junit] at junit.framework.TestResult.runProtected(TestResult.java:128)
      [junit] at junit.framework.TestResult.run(TestResult.java:113)
      [junit] at junit.framework.TestCase.run(TestCase.java:124)
      [junit] at junit.framework.TestSuite.runTest(TestSuite.java:243)
      [junit] at junit.framework.TestSuite.run(TestSuite.java:238)
      [junit] at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:420)
      [junit] at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.launch(JUnitTestRunner.java:911)
      [junit] at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:768)
      [junit] Exception: Client execution results failed with error code = 1
      [junit] See build/ql/tmp/hive.log, or try "ant test ... -Dtest.silent=false" to get more logs.
      [junit] See build/ql/tmp/hive.log, or try "ant test ... -Dtest.silent=false" to get more logs.)

      1. HIVE-2856.2.patch.txt
        2 kB
        Zhenxiao Luo
      2. HIVE-2856.1.patch.txt
        1 kB
        Zhenxiao Luo
      3. HIVE-2794.1.patch.txt
        3 kB
        Zhenxiao Luo
      4. escape2.q.out
        1.01 MB
        Zhenxiao Luo
      5. escape1.q.out
        100 kB
        Zhenxiao Luo
      6. escape1.q.out
        100 kB
        Zhenxiao Luo

        Activity

        Hide
        Hudson added a comment -

        Integrated in Hive-trunk-hadoop2 #54 (See https://builds.apache.org/job/Hive-trunk-hadoop2/54/)
        HIVE-2856. Fix TestCliDriver escape1.q failure on MR2 (Zhenxiao Luo via cws) (Revision 1301162)

        Result = ABORTED
        cws : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1301162
        Files :

        • /hive/trunk/common/src/java/org/apache/hadoop/hive/common/FileUtils.java
        • /hive/trunk/ql/src/test/queries/clientpositive/escape1.q
        • /hive/trunk/ql/src/test/queries/clientpositive/escape2.q
        • /hive/trunk/ql/src/test/results/clientpositive/escape1.q.out
        • /hive/trunk/ql/src/test/results/clientpositive/escape2.q.out
        Show
        Hudson added a comment - Integrated in Hive-trunk-hadoop2 #54 (See https://builds.apache.org/job/Hive-trunk-hadoop2/54/ ) HIVE-2856 . Fix TestCliDriver escape1.q failure on MR2 (Zhenxiao Luo via cws) (Revision 1301162) Result = ABORTED cws : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1301162 Files : /hive/trunk/common/src/java/org/apache/hadoop/hive/common/FileUtils.java /hive/trunk/ql/src/test/queries/clientpositive/escape1.q /hive/trunk/ql/src/test/queries/clientpositive/escape2.q /hive/trunk/ql/src/test/results/clientpositive/escape1.q.out /hive/trunk/ql/src/test/results/clientpositive/escape2.q.out
        Hide
        Ashutosh Chauhan added a comment -

        This issue is closed now. It was released with the fix in 0.9.0. If there is a problem, please open a new jira and link this one with that.

        Show
        Ashutosh Chauhan added a comment - This issue is closed now. It was released with the fix in 0.9.0. If there is a problem, please open a new jira and link this one with that.
        Hide
        Hudson added a comment -

        Integrated in Hive-trunk-h0.21 #1312 (See https://builds.apache.org/job/Hive-trunk-h0.21/1312/)
        HIVE-2856. Fix TestCliDriver escape1.q failure on MR2 (Zhenxiao Luo via cws) (Revision 1301162)

        Result = FAILURE
        cws : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1301162
        Files :

        • /hive/trunk/common/src/java/org/apache/hadoop/hive/common/FileUtils.java
        • /hive/trunk/ql/src/test/queries/clientpositive/escape1.q
        • /hive/trunk/ql/src/test/queries/clientpositive/escape2.q
        • /hive/trunk/ql/src/test/results/clientpositive/escape1.q.out
        • /hive/trunk/ql/src/test/results/clientpositive/escape2.q.out
        Show
        Hudson added a comment - Integrated in Hive-trunk-h0.21 #1312 (See https://builds.apache.org/job/Hive-trunk-h0.21/1312/ ) HIVE-2856 . Fix TestCliDriver escape1.q failure on MR2 (Zhenxiao Luo via cws) (Revision 1301162) Result = FAILURE cws : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1301162 Files : /hive/trunk/common/src/java/org/apache/hadoop/hive/common/FileUtils.java /hive/trunk/ql/src/test/queries/clientpositive/escape1.q /hive/trunk/ql/src/test/queries/clientpositive/escape2.q /hive/trunk/ql/src/test/results/clientpositive/escape1.q.out /hive/trunk/ql/src/test/results/clientpositive/escape2.q.out
        Hide
        Carl Steinbach added a comment -

        Committed to trunk. Thanks Zhenxiao!

        Show
        Carl Steinbach added a comment - Committed to trunk. Thanks Zhenxiao!
        Hide
        Zhenxiao Luo added a comment -

        New patch for review. Both escape1.q.out and escape2.q.out are updated.

        Show
        Zhenxiao Luo added a comment - New patch for review. Both escape1.q.out and escape2.q.out are updated.
        Hide
        Carl Steinbach added a comment -

        Test failure in escape2.q

        Show
        Carl Steinbach added a comment - Test failure in escape2.q
        Hide
        Carl Steinbach added a comment -

        +1. Will commit if tests pass.

        Show
        Carl Steinbach added a comment - +1. Will commit if tests pass.
        Hide
        Zhenxiao Luo added a comment -

        the new expected output

        Show
        Zhenxiao Luo added a comment - the new expected output
        Hide
        Carl Steinbach added a comment -

        @Zhenxiao: Please attach an updated copy of escape1.q.out. You can either attach this separately, or generate a new diff that includes binary changes using the command "git diff --binary HEAD^ > patch.txt". Thanks.

        Show
        Carl Steinbach added a comment - @Zhenxiao: Please attach an updated copy of escape1.q.out. You can either attach this separately, or generate a new diff that includes binary changes using the command "git diff --binary HEAD^ > patch.txt". Thanks.
        Hide
        Zhenxiao Luo added a comment -

        1. '^' should be escaped
        2. the previous escape1.q.out is incorrect, the '^' row is always missing in table escape1. table escape1 has one less row than escape_raw.
        3. #1 and #2 together leads to testcase escape1 always passing
        4. for some reason, escape1.q.out is loaded as binary file, not ascii file. git diff could not get diffs for binary file
        5. This patch only includes code changes and escape1.q change. escape1.q.out not included(could not get via git diff)

        Show
        Zhenxiao Luo added a comment - 1. '^' should be escaped 2. the previous escape1.q.out is incorrect, the '^' row is always missing in table escape1. table escape1 has one less row than escape_raw. 3. #1 and #2 together leads to testcase escape1 always passing 4. for some reason, escape1.q.out is loaded as binary file, not ascii file. git diff could not get diffs for binary file 5. This patch only includes code changes and escape1.q change. escape1.q.out not included(could not get via git diff)
        Hide
        Zhenxiao Luo added a comment -

        1. '^' should be escaped
        2. the previous escape1.q.out is incorrect, the '^' row is always missing in table escape1. table escape1 has one less row than escape_raw.
        3. #1 and #2 together leads to testcase escape1 always passing
        4. for some reason, escape1.q.out is loaded as binary file, not ascii file. git diff could not get diffs for binary file
        5. This patch only includes code changes and escape1.q change. escape1.q.out not included(could not get via git diff)

        Show
        Zhenxiao Luo added a comment - 1. '^' should be escaped 2. the previous escape1.q.out is incorrect, the '^' row is always missing in table escape1. table escape1 has one less row than escape_raw. 3. #1 and #2 together leads to testcase escape1 always passing 4. for some reason, escape1.q.out is loaded as binary file, not ascii file. git diff could not get diffs for binary file 5. This patch only includes code changes and escape1.q change. escape1.q.out not included(could not get via git diff)
        Hide
        Zhenxiao Luo added a comment -

        In testcase escape1.q:

        Seems '^' also needs to be escaped.

        If add:

        SELECT count from escape_raw;
        SELECT count from escape1;

        into escape1.q

        escape1 only get 127 rows, while, escape_raw get 128 rows(note that escapetest.txt has 128 rows).

        one row missing in escape1, which is the '^'.

        Show
        Zhenxiao Luo added a comment - In testcase escape1.q: Seems '^' also needs to be escaped. If add: SELECT count from escape_raw; SELECT count from escape1; into escape1.q escape1 only get 127 rows, while, escape_raw get 128 rows(note that escapetest.txt has 128 rows). one row missing in escape1, which is the '^'.

          People

          • Assignee:
            Zhenxiao Luo
            Reporter:
            Zhenxiao Luo
          • Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development