Hive
  1. Hive
  2. HIVE-1441

Extend ivy offline mode to cover metastore downloads

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.6.0
    • Fix Version/s: 0.7.0
    • Component/s: Build Infrastructure
    • Labels:
      None

      Description

      We recently started downloading datanucleus jars via ivy, and the existing ivy offilne mode doesn't cover this, so we still end up trying to contact the ivy repository even with offline mode enabled.

        Activity

        John Sichi created issue -
        Hide
        John Sichi added a comment -

        I need to test this out some more.

        Show
        John Sichi added a comment - I need to test this out some more.
        John Sichi made changes -
        Field Original Value New Value
        Attachment HIVE-1441.1.patch [ 12448141 ]
        Hide
        John Sichi added a comment -

        Tested this out, should be ready.

        Show
        John Sichi added a comment - Tested this out, should be ready.
        John Sichi made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Hide
        Namit Jain added a comment -

        +1

        Show
        Namit Jain added a comment - +1
        Hide
        Edward Capriolo added a comment -

        Testing.

        test:
        [junit] Running org.apache.hadoop.hive.metastore.TestHiveMetaStore
        [junit] Tests run: 10, Failures: 0, Errors: 0, Time elapsed: 24.539 sec
        [junit] Running org.apache.hadoop.hive.metastore.TestHiveMetaStoreRemote
        [junit] Running metastore!
        [junit] Tests run: 1, Failures: 0, Errors: 1, Time elapsed: 8.238 sec
        [junit] Test org.apache.hadoop.hive.metastore.TestHiveMetaStoreRemote FAILED

        Can this be skipped?

        Show
        Edward Capriolo added a comment - Testing. test: [junit] Running org.apache.hadoop.hive.metastore.TestHiveMetaStore [junit] Tests run: 10, Failures: 0, Errors: 0, Time elapsed: 24.539 sec [junit] Running org.apache.hadoop.hive.metastore.TestHiveMetaStoreRemote [junit] Running metastore! [junit] Tests run: 1, Failures: 0, Errors: 1, Time elapsed: 8.238 sec [junit] Test org.apache.hadoop.hive.metastore.TestHiveMetaStoreRemote FAILED Can this be skipped?
        Hide
        John Sichi added a comment -

        What is the failure? Look in build/metastore/test/TEST-org.apache.hadoop.hive.metastore.TestHiveMetaStoreRemote.xml

        And does it occur only with this patch applied? Offline mode only works after everything has been downloaded for the first time in online mode, so be sure to get a clean test run first in online mode.

        Show
        John Sichi added a comment - What is the failure? Look in build/metastore/test/TEST-org.apache.hadoop.hive.metastore.TestHiveMetaStoreRemote.xml And does it occur only with this patch applied? Offline mode only works after everything has been downloaded for the first time in online mode, so be sure to get a clean test run first in online mode.
        Hide
        Edward Capriolo added a comment -

        Fresh checkout before any after patch. Still looking into it.

         </properties>
          <testcase classname="org.apache.hadoop.hive.metastore.TestHiveMetaStoreRemote" name="testPartition" time="8.242">
            <error message="Could not connect to meta store using any of the URIs provided" type="org.apache.hadoop.hive.metastore.api.MetaException">M
        etaException(message:Could not connect to meta store using any of the URIs provided)
        	at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.open(HiveMetaStoreClient.java:160)
        	at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.&lt;init&gt;(HiveMetaStoreClient.java:128)
        	at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.&lt;init&gt;(HiveMetaStoreClient.java:71)
        	at org.apache.hadoop.hive.metastore.TestHiveMetaStoreRemote.setUp(TestHiveMetaStoreRemote.java:64)
        	at junit.framework.TestCase.runBare(TestCase.java:125)
        	at junit.framework.TestResult$1.protect(TestResult.java:106)
        	at junit.framework.TestResult.runProtected(TestResult.java:124)
        	at junit.framework.TestResult.run(TestResult.java:109)
        	at junit.framework.TestCase.run(TestCase.java:118)
        	at junit.framework.TestSuite.runTest(TestSuite.java:208)
        	at junit.framework.TestSuite.run(TestSuite.java:203)
        	at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:422)
        	at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.launch(JUnitTestRunner.java:931)
        	at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:785)
        </error>
          </testcase>
          <system-out><![CDATA[Running metastore!
        ]]></system-out>
          <system-err><![CDATA[]]></system-err>
        </testsuite>
        
        
        From eclipse:
        Running metastore!
        MetaException(message:hive.metastore.warehouse.dir is not set in the config or blank)
        	at org.apache.hadoop.hive.metastore.Warehouse.<init>(Warehouse.java:58)
        	at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java:155)
        	at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.<init>(HiveMetaStore.java:125)
        	at org.apache.hadoop.hive.metastore.HiveMetaStore.main(HiveMetaStore.java:1965)
        	at org.apache.hadoop.hive.metastore.TestHiveMetaStoreRemote$RunMS.run(TestHiveMetaStoreRemote.java:39)
        	at java.lang.Thread.run(Thread.java:619)
        10/07/30 16:03:22 ERROR metastore.HiveMetaStore: Metastore Thrift Server threw an exception. Exiting...
        10/07/30 16:03:22 ERROR metastore.HiveMetaStore: MetaException(message:hive.metastore.warehouse.dir is not set in the config or blank)
        	at org.apache.hadoop.hive.metastore.Warehouse.<init>(Warehouse.java:58)
        	at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java:155)
        	at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.<init>(HiveMetaStore.java:125)
        	at org.apache.hadoop.hive.metastore.HiveMetaStore.main(HiveMetaStore.java:1965)
        	at org.apache.hadoop.hive.metastore.TestHiveMetaStoreRemote$RunMS.run(TestHiveMetaStoreRemote.java:39)
        	at java.lang.Thread.run(Thread.java:619)
        
        Show
        Edward Capriolo added a comment - Fresh checkout before any after patch. Still looking into it. </properties> <testcase classname="org.apache.hadoop.hive.metastore.TestHiveMetaStoreRemote" name="testPartition" time="8.242"> <error message="Could not connect to meta store using any of the URIs provided" type="org.apache.hadoop.hive.metastore.api.MetaException">M etaException(message:Could not connect to meta store using any of the URIs provided) at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.open(HiveMetaStoreClient.java:160) at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.&lt;init&gt;(HiveMetaStoreClient.java:128) at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.&lt;init&gt;(HiveMetaStoreClient.java:71) at org.apache.hadoop.hive.metastore.TestHiveMetaStoreRemote.setUp(TestHiveMetaStoreRemote.java:64) at junit.framework.TestCase.runBare(TestCase.java:125) at junit.framework.TestResult$1.protect(TestResult.java:106) at junit.framework.TestResult.runProtected(TestResult.java:124) at junit.framework.TestResult.run(TestResult.java:109) at junit.framework.TestCase.run(TestCase.java:118) at junit.framework.TestSuite.runTest(TestSuite.java:208) at junit.framework.TestSuite.run(TestSuite.java:203) at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:422) at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.launch(JUnitTestRunner.java:931) at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:785) </error> </testcase> <system-out><![CDATA[Running metastore! ]]></system-out> <system-err><![CDATA[]]></system-err> </testsuite> From eclipse: Running metastore! MetaException(message:hive.metastore.warehouse.dir is not set in the config or blank) at org.apache.hadoop.hive.metastore.Warehouse.<init>(Warehouse.java:58) at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java:155) at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.<init>(HiveMetaStore.java:125) at org.apache.hadoop.hive.metastore.HiveMetaStore.main(HiveMetaStore.java:1965) at org.apache.hadoop.hive.metastore.TestHiveMetaStoreRemote$RunMS.run(TestHiveMetaStoreRemote.java:39) at java.lang.Thread.run(Thread.java:619) 10/07/30 16:03:22 ERROR metastore.HiveMetaStore: Metastore Thrift Server threw an exception. Exiting... 10/07/30 16:03:22 ERROR metastore.HiveMetaStore: MetaException(message:hive.metastore.warehouse.dir is not set in the config or blank) at org.apache.hadoop.hive.metastore.Warehouse.<init>(Warehouse.java:58) at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java:155) at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.<init>(HiveMetaStore.java:125) at org.apache.hadoop.hive.metastore.HiveMetaStore.main(HiveMetaStore.java:1965) at org.apache.hadoop.hive.metastore.TestHiveMetaStoreRemote$RunMS.run(TestHiveMetaStoreRemote.java:39) at java.lang.Thread.run(Thread.java:619)
        Hide
        Ning Zhang added a comment -

        Committed. Thanks John!

        Show
        Ning Zhang added a comment - Committed. Thanks John!
        Ning Zhang made changes -
        Status Patch Available [ 10002 ] Resolved [ 5 ]
        Resolution Fixed [ 1 ]
        Carl Steinbach made changes -
        Status Resolved [ 5 ] Closed [ 6 ]

          People

          • Assignee:
            John Sichi
            Reporter:
            John Sichi
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development