Hive
  1. Hive
  2. HIVE-4999

Shim class HiveHarFileSystem does not have a hadoop2 counterpart

    Details

    • Type: Task Task
    • Status: Closed
    • Priority: Minor Minor
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.12.0
    • Component/s: None
    • Labels:
      None

      Description

      HiveHarFileSystem only exists in the 0.20 shim.

      1. HIVE-4999.patch
        6 kB
        Brock Noland

        Activity

        Hide
        Brock Noland added a comment -

        Looks like this would do it:

        svn mv ./shims/src/0.20/java/org/apache/hadoop/hive/shims/HiveHarFileSystem.java shims/src/common/java/org/apache/hadoop/hive/shims/

        Show
        Brock Noland added a comment - Looks like this would do it: svn mv ./shims/src/0.20/java/org/apache/hadoop/hive/shims/HiveHarFileSystem.java shims/src/common/java/org/apache/hadoop/hive/shims/
        Hide
        Brock Noland added a comment -

        If we commit this change we'll use the svn mv command above. However this patch allows us to test the change.

        Show
        Brock Noland added a comment - If we commit this change we'll use the svn mv command above. However this patch allows us to test the change.
        Hide
        Hive QA added a comment -

        Overall: +1 all checks pass

        Here are the results of testing the latest attachment:
        https://issues.apache.org/jira/secure/attachment/12597083/HIVE-4999.patch

        SUCCESS: +1 2773 tests passed

        Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/364/testReport
        Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/364/console

        Messages:

        Executing org.apache.hive.ptest.execution.PrepPhase
        Executing org.apache.hive.ptest.execution.ExecutionPhase
        Executing org.apache.hive.ptest.execution.ReportingPhase
        

        This message is automatically generated.

        Show
        Hive QA added a comment - Overall : +1 all checks pass Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12597083/HIVE-4999.patch SUCCESS: +1 2773 tests passed Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/364/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/364/console Messages: Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase This message is automatically generated.
        Hide
        Ashutosh Chauhan added a comment -

        Is there any test which fails because of this ?

        Show
        Ashutosh Chauhan added a comment - Is there any test which fails because of this ?
        Hide
        Brock Noland added a comment -

        Hey good call, I am not sure why I make this a subtask of the Hadoop2 test failure JIRA. No tests fail because of this...I just think since in practice it's shared so we should move it to the shared directory.

        Show
        Brock Noland added a comment - Hey good call, I am not sure why I make this a subtask of the Hadoop2 test failure JIRA. No tests fail because of this...I just think since in practice it's shared so we should move it to the shared directory.
        Hide
        Edward Capriolo added a comment -

        Ashutosh? Are you reviewing? I am +1 for this. It can not break anything existing.

        Brock Noland There must be some tests for HAR functionality that are excluding 23 we should change those as well?

        Show
        Edward Capriolo added a comment - Ashutosh? Are you reviewing? I am +1 for this. It can not break anything existing. Brock Noland There must be some tests for HAR functionality that are excluding 23 we should change those as well?
        Hide
        Brock Noland added a comment -

        Hey,

        There are test for har for 0.23 (archive_excludeHadoop20.q). I do believe these are failing for 0.23 (HIVE-3025) but not for because of the issue described here. There isn't any binary incompatibility problem here so the class compiled with 0.20 works fine for 0.23. It's just that the file is in the wrong directory.

        Show
        Brock Noland added a comment - Hey, There are test for har for 0.23 (archive_excludeHadoop20.q). I do believe these are failing for 0.23 ( HIVE-3025 ) but not for because of the issue described here. There isn't any binary incompatibility problem here so the class compiled with 0.20 works fine for 0.23. It's just that the file is in the wrong directory.
        Hide
        Edward Capriolo added a comment -

        I am +1. Will run move in 24 hours unless someone stops me.

        Show
        Edward Capriolo added a comment - I am +1. Will run move in 24 hours unless someone stops me.
        Hide
        Edward Capriolo added a comment -

        I think we should go around and change all the q tests that are using archiving but disabling other versions.

        [edward@jackintosh hive-trunk]$ svn diff ql/src/test/queries/clientpositive/archive.q
        Index: ql/src/test/queries/clientpositive/archive.q
        ===================================================================
        — ql/src/test/queries/clientpositive/archive.q (revision 1514126)
        +++ ql/src/test/queries/clientpositive/archive.q (working copy)
        @@ -1,8 +1,6 @@
        set hive.archive.enabled = true;
        set hive.enforce.bucketing = true;

        — INCLUDE_HADOOP_MAJOR_VERSIONS(0.20)
        -
        drop table tstsrc;
        drop table tstsrcpart;

        Show
        Edward Capriolo added a comment - I think we should go around and change all the q tests that are using archiving but disabling other versions. [edward@jackintosh hive-trunk] $ svn diff ql/src/test/queries/clientpositive/archive.q Index: ql/src/test/queries/clientpositive/archive.q =================================================================== — ql/src/test/queries/clientpositive/archive.q (revision 1514126) +++ ql/src/test/queries/clientpositive/archive.q (working copy) @@ -1,8 +1,6 @@ set hive.archive.enabled = true; set hive.enforce.bucketing = true; — INCLUDE_HADOOP_MAJOR_VERSIONS(0.20) - drop table tstsrc; drop table tstsrcpart;
        Hide
        Brock Noland added a comment -

        Hi,

        The testing issue is being taken forward in HIVE-4910.

        Brock

        Show
        Brock Noland added a comment - Hi, The testing issue is being taken forward in HIVE-4910 . Brock
        Hide
        Edward Capriolo added a comment -

        OIC. +1 will commit.

        Show
        Edward Capriolo added a comment - OIC. +1 will commit.
        Hide
        Edward Capriolo added a comment -

        Moveitted... Thanks Brock.

        Show
        Edward Capriolo added a comment - Moveitted... Thanks Brock.
        Hide
        Brock Noland added a comment -

        Thank you much!!

        Show
        Brock Noland added a comment - Thank you much!!
        Hide
        Hudson added a comment -

        FAILURE: Integrated in Hive-trunk-hadoop2 #362 (See https://builds.apache.org/job/Hive-trunk-hadoop2/362/)
        HIVE-4999 Shim class HiveHarFileSystem does not have a hadoop2 counterpart (Brock Noland via egc)

        Submitted by: Brock Noland
        Reviewed by: Edward Capriolo (ecapriolo: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1514277)

        • /hive/trunk/shims/src/0.20/java/org/apache/hadoop/hive/shims/HiveHarFileSystem.java
        • /hive/trunk/shims/src/common/java/org/apache/hadoop/hive/shims/HiveHarFileSystem.java
        Show
        Hudson added a comment - FAILURE: Integrated in Hive-trunk-hadoop2 #362 (See https://builds.apache.org/job/Hive-trunk-hadoop2/362/ ) HIVE-4999 Shim class HiveHarFileSystem does not have a hadoop2 counterpart (Brock Noland via egc) Submitted by: Brock Noland Reviewed by: Edward Capriolo (ecapriolo: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1514277 ) /hive/trunk/shims/src/0.20/java/org/apache/hadoop/hive/shims/HiveHarFileSystem.java /hive/trunk/shims/src/common/java/org/apache/hadoop/hive/shims/HiveHarFileSystem.java
        Hide
        Hudson added a comment -

        FAILURE: Integrated in Hive-trunk-hadoop1-ptest #127 (See https://builds.apache.org/job/Hive-trunk-hadoop1-ptest/127/)
        HIVE-4999 Shim class HiveHarFileSystem does not have a hadoop2 counterpart (Brock Noland via egc)

        Submitted by: Brock Noland
        Reviewed by: Edward Capriolo (ecapriolo: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1514277)

        • /hive/trunk/shims/src/0.20/java/org/apache/hadoop/hive/shims/HiveHarFileSystem.java
        • /hive/trunk/shims/src/common/java/org/apache/hadoop/hive/shims/HiveHarFileSystem.java
        Show
        Hudson added a comment - FAILURE: Integrated in Hive-trunk-hadoop1-ptest #127 (See https://builds.apache.org/job/Hive-trunk-hadoop1-ptest/127/ ) HIVE-4999 Shim class HiveHarFileSystem does not have a hadoop2 counterpart (Brock Noland via egc) Submitted by: Brock Noland Reviewed by: Edward Capriolo (ecapriolo: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1514277 ) /hive/trunk/shims/src/0.20/java/org/apache/hadoop/hive/shims/HiveHarFileSystem.java /hive/trunk/shims/src/common/java/org/apache/hadoop/hive/shims/HiveHarFileSystem.java
        Hide
        Hudson added a comment -

        FAILURE: Integrated in Hive-trunk-hadoop2-ptest #58 (See https://builds.apache.org/job/Hive-trunk-hadoop2-ptest/58/)
        HIVE-4999 Shim class HiveHarFileSystem does not have a hadoop2 counterpart (Brock Noland via egc)

        Submitted by: Brock Noland
        Reviewed by: Edward Capriolo (ecapriolo: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1514277)

        • /hive/trunk/shims/src/0.20/java/org/apache/hadoop/hive/shims/HiveHarFileSystem.java
        • /hive/trunk/shims/src/common/java/org/apache/hadoop/hive/shims/HiveHarFileSystem.java
        Show
        Hudson added a comment - FAILURE: Integrated in Hive-trunk-hadoop2-ptest #58 (See https://builds.apache.org/job/Hive-trunk-hadoop2-ptest/58/ ) HIVE-4999 Shim class HiveHarFileSystem does not have a hadoop2 counterpart (Brock Noland via egc) Submitted by: Brock Noland Reviewed by: Edward Capriolo (ecapriolo: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1514277 ) /hive/trunk/shims/src/0.20/java/org/apache/hadoop/hive/shims/HiveHarFileSystem.java /hive/trunk/shims/src/common/java/org/apache/hadoop/hive/shims/HiveHarFileSystem.java
        Hide
        Hudson added a comment -

        FAILURE: Integrated in Hive-trunk-h0.21 #2271 (See https://builds.apache.org/job/Hive-trunk-h0.21/2271/)
        HIVE-4999 Shim class HiveHarFileSystem does not have a hadoop2 counterpart (Brock Noland via egc)

        Submitted by: Brock Noland
        Reviewed by: Edward Capriolo (ecapriolo: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1514277)

        • /hive/trunk/shims/src/0.20/java/org/apache/hadoop/hive/shims/HiveHarFileSystem.java
        • /hive/trunk/shims/src/common/java/org/apache/hadoop/hive/shims/HiveHarFileSystem.java
        Show
        Hudson added a comment - FAILURE: Integrated in Hive-trunk-h0.21 #2271 (See https://builds.apache.org/job/Hive-trunk-h0.21/2271/ ) HIVE-4999 Shim class HiveHarFileSystem does not have a hadoop2 counterpart (Brock Noland via egc) Submitted by: Brock Noland Reviewed by: Edward Capriolo (ecapriolo: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1514277 ) /hive/trunk/shims/src/0.20/java/org/apache/hadoop/hive/shims/HiveHarFileSystem.java /hive/trunk/shims/src/common/java/org/apache/hadoop/hive/shims/HiveHarFileSystem.java
        Hide
        Ashutosh Chauhan added a comment -

        This issue has been fixed and released as part of 0.12 release. If you find further issues, please create a new jira and link it to this one.

        Show
        Ashutosh Chauhan added a comment - This issue has been fixed and released as part of 0.12 release. If you find further issues, please create a new jira and link it to this one.

          People

          • Assignee:
            Brock Noland
            Reporter:
            Brock Noland
          • Votes:
            1 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development