Hive
  1. Hive
  2. HIVE-3412

Fix TestCliDriver.repair on Hadoop 0.23.3, 3.0.0, and 2.2.0-alpha

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.9.0
    • Fix Version/s: 0.10.0
    • Component/s: Tests
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      TestCliDriver.repair fails on the following Hadoop versions:
      0.23.3, 3.0.0, 2.2.0-alpha

      repair.q fails with "dfs -mkdir":
      [junit] mkdir: `../build/ql/test/data/warehouse/repairtable/p1=a/p2=a': No such file or directory

      The problem is, after fixing HADOOP-8551, which changes the hdfs Shell syntax for mkdir:
      https://issues.apache.org/jira/browse/HADOOP-8551

      all "dfs -mkdir" commands should provide "-p" in order to execute without error.

      This is an intentional change in HDFS. And HADOOP-8551 will be included in 0.23.3, 3.0.0, 2.2.0-alpha versions.

      1. HIVE-3412.2.patch.txt
        7 kB
        Zhenxiao Luo
      2. HIVE-3412.1.patch.txt
        5 kB
        Zhenxiao Luo

        Activity

        Hide
        Zhenxiao Luo added a comment -

        Review Request submitted at:
        https://reviews.facebook.net/D4989

        Show
        Zhenxiao Luo added a comment - Review Request submitted at: https://reviews.facebook.net/D4989
        Hide
        Carl Steinbach added a comment -

        @Zhenxiao: Please see my comments on phabricator. Thanks.

        Show
        Carl Steinbach added a comment - @Zhenxiao: Please see my comments on phabricator. Thanks.
        Hide
        Zhenxiao Luo added a comment -

        @Carl: thanks for the review. I addressed the comments and resubmitted patch at:
        https://reviews.facebook.net/D4989

        Show
        Zhenxiao Luo added a comment - @Carl: thanks for the review. I addressed the comments and resubmitted patch at: https://reviews.facebook.net/D4989
        Hide
        Carl Steinbach added a comment -

        +1. Will commit if tests pass.

        Show
        Carl Steinbach added a comment - +1. Will commit if tests pass.
        Hide
        Carl Steinbach added a comment -

        Committed to trunk. Thanks Zhenxiao!

        Show
        Carl Steinbach added a comment - Committed to trunk. Thanks Zhenxiao!
        Hide
        Hudson added a comment -

        Integrated in Hive-trunk-h0.21 #1646 (See https://builds.apache.org/job/Hive-trunk-h0.21/1646/)
        HIVE-3412. Fix TestCliDriver.repair on Hadoop 0.23.3, 3.0.0, and 2.2.0-alpha (Zhenxiao Luo via cws) (Revision 1380479)

        Result = FAILURE
        cws : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1380479
        Files :

        • /hive/trunk/ql/src/test/queries/clientpositive/repair.q
        • /hive/trunk/ql/src/test/queries/clientpositive/repair_hadoop23.q
        • /hive/trunk/ql/src/test/results/clientpositive/repair.q.out
        • /hive/trunk/ql/src/test/results/clientpositive/repair_hadoop23.q.out
        Show
        Hudson added a comment - Integrated in Hive-trunk-h0.21 #1646 (See https://builds.apache.org/job/Hive-trunk-h0.21/1646/ ) HIVE-3412 . Fix TestCliDriver.repair on Hadoop 0.23.3, 3.0.0, and 2.2.0-alpha (Zhenxiao Luo via cws) (Revision 1380479) Result = FAILURE cws : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1380479 Files : /hive/trunk/ql/src/test/queries/clientpositive/repair.q /hive/trunk/ql/src/test/queries/clientpositive/repair_hadoop23.q /hive/trunk/ql/src/test/results/clientpositive/repair.q.out /hive/trunk/ql/src/test/results/clientpositive/repair_hadoop23.q.out
        Hide
        Hudson added a comment -

        Integrated in Hive-trunk-hadoop2 #54 (See https://builds.apache.org/job/Hive-trunk-hadoop2/54/)
        HIVE-3412. Fix TestCliDriver.repair on Hadoop 0.23.3, 3.0.0, and 2.2.0-alpha (Zhenxiao Luo via cws) (Revision 1380479)

        Result = ABORTED
        cws : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1380479
        Files :

        • /hive/trunk/ql/src/test/queries/clientpositive/repair.q
        • /hive/trunk/ql/src/test/queries/clientpositive/repair_hadoop23.q
        • /hive/trunk/ql/src/test/results/clientpositive/repair.q.out
        • /hive/trunk/ql/src/test/results/clientpositive/repair_hadoop23.q.out
        Show
        Hudson added a comment - Integrated in Hive-trunk-hadoop2 #54 (See https://builds.apache.org/job/Hive-trunk-hadoop2/54/ ) HIVE-3412 . Fix TestCliDriver.repair on Hadoop 0.23.3, 3.0.0, and 2.2.0-alpha (Zhenxiao Luo via cws) (Revision 1380479) Result = ABORTED cws : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1380479 Files : /hive/trunk/ql/src/test/queries/clientpositive/repair.q /hive/trunk/ql/src/test/queries/clientpositive/repair_hadoop23.q /hive/trunk/ql/src/test/results/clientpositive/repair.q.out /hive/trunk/ql/src/test/results/clientpositive/repair_hadoop23.q.out
        Hide
        Ashutosh Chauhan added a comment -

        This issue is fixed and released as part of 0.10.0 release. If you find an issue which seems to be related to this one, please create a new jira and link this one with new jira.

        Show
        Ashutosh Chauhan added a comment - This issue is fixed and released as part of 0.10.0 release. If you find an issue which seems to be related to this one, please create a new jira and link this one with new jira.

          People

          • Assignee:
            Zhenxiao Luo
            Reporter:
            Zhenxiao Luo
          • Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development