Tajo
  1. Tajo
  2. TAJO-16

Enable Tajo catalog to access Hive metastore.

    Details

    • Type: Improvement Improvement
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.8.0
    • Component/s: None
    • Labels:
      None

      Description

      Hive has been widely used in this area. Many users have maintained lots of big tables through Hive metastore. It would be great to enable Tajo catalog to access Hive metastore. Hive metastore already provides thrift API. We could employ it.

      1. TAJO-16.pdf
        325 kB
        Jaehwa Jung
      2. TAJO-16.patch
        27 kB
        Jaehwa Jung
      3. TAJO-16_2.pdf
        284 kB
        Jaehwa Jung
      4. TAJO-16_2.patch
        27 kB
        Jaehwa Jung
      5. TAJO-16_3.patch
        27 kB
        Jaehwa Jung

        Activity

        Hide
        Hudson added a comment -

        SUCCESS: Integrated in Tajo-trunk-postcommit #621 (See https://builds.apache.org/job/Tajo-trunk-postcommit/621/)
        TAJO-327: Add testcase to verify TAJO-16. (jaehwa) (jhjung: https://git-wip-us.apache.org/repos/asf?p=incubator-tajo.git&a=commit&h=1d2bd0d97949c6a0475c4e180505aeaa12188b51)

        • tajo-catalog/tajo-catalog-drivers/tajo-hcatalog/src/test/java/org/apache/tajo/catalog/store/DummyListener.java
        • CHANGES.txt
        • tajo-catalog/tajo-catalog-drivers/tajo-hcatalog/src/test/java/org/apache/tajo/catalog/store/TestHCatalogStore.java
        • tajo-catalog/tajo-catalog-drivers/tajo-hcatalog/pom.xml
        • tajo-catalog/tajo-catalog-drivers/tajo-hcatalog/src/main/java/org/apache/tajo/catalog/store/HCatalogStore.java
        Show
        Hudson added a comment - SUCCESS: Integrated in Tajo-trunk-postcommit #621 (See https://builds.apache.org/job/Tajo-trunk-postcommit/621/ ) TAJO-327 : Add testcase to verify TAJO-16 . (jaehwa) (jhjung: https://git-wip-us.apache.org/repos/asf?p=incubator-tajo.git&a=commit&h=1d2bd0d97949c6a0475c4e180505aeaa12188b51 ) tajo-catalog/tajo-catalog-drivers/tajo-hcatalog/src/test/java/org/apache/tajo/catalog/store/DummyListener.java CHANGES.txt tajo-catalog/tajo-catalog-drivers/tajo-hcatalog/src/test/java/org/apache/tajo/catalog/store/TestHCatalogStore.java tajo-catalog/tajo-catalog-drivers/tajo-hcatalog/pom.xml tajo-catalog/tajo-catalog-drivers/tajo-hcatalog/src/main/java/org/apache/tajo/catalog/store/HCatalogStore.java
        Hide
        Hudson added a comment -

        SUCCESS: Integrated in Tajo-trunk-postcommit #553 (See https://builds.apache.org/job/Tajo-trunk-postcommit/553/)
        TAJO-16: Enable Tajo catalog to access Hive metastore. (jaehwa) (jhjung: https://git-wip-us.apache.org/repos/asf?p=incubator-tajo.git&a=commit&h=943899360a29fdab2ee1aefa7f1708f5666939c6)

        • pom.xml
        • tajo-catalog/tajo-catalog-server/src/main/java/org/apache/tajo/catalog/store/HCatalogUtil.java
        • CHANGES.txt
        • tajo-dist/pom.xml
        • tajo-core/tajo-core-pullserver/src/main/java/org/apache/tajo/pullserver/TajoPullServerService.java
        • tajo-catalog/tajo-catalog-server/src/main/java/org/apache/tajo/catalog/store/HCatalogStore.java
        • tajo-catalog/tajo-catalog-server/pom.xml
        Show
        Hudson added a comment - SUCCESS: Integrated in Tajo-trunk-postcommit #553 (See https://builds.apache.org/job/Tajo-trunk-postcommit/553/ ) TAJO-16 : Enable Tajo catalog to access Hive metastore. (jaehwa) (jhjung: https://git-wip-us.apache.org/repos/asf?p=incubator-tajo.git&a=commit&h=943899360a29fdab2ee1aefa7f1708f5666939c6 ) pom.xml tajo-catalog/tajo-catalog-server/src/main/java/org/apache/tajo/catalog/store/HCatalogUtil.java CHANGES.txt tajo-dist/pom.xml tajo-core/tajo-core-pullserver/src/main/java/org/apache/tajo/pullserver/TajoPullServerService.java tajo-catalog/tajo-catalog-server/src/main/java/org/apache/tajo/catalog/store/HCatalogStore.java tajo-catalog/tajo-catalog-server/pom.xml
        Hide
        Jaehwa Jung added a comment -

        I committed patch and I'll implement new issues as soon as possible.

        Show
        Jaehwa Jung added a comment - I committed patch and I'll implement new issues as soon as possible.
        Hide
        Jaehwa Jung added a comment -

        Thanks Hyunsik.

        I missed your comment.
        I agree with you and I'll create new jira issues.

        Show
        Jaehwa Jung added a comment - Thanks Hyunsik. I missed your comment. I agree with you and I'll create new jira issues.
        Hide
        Jihoon Son added a comment -

        Thanks.
        As you commented, write a document for this issue, please.

        Show
        Jihoon Son added a comment - Thanks. As you commented, write a document for this issue, please.
        Hide
        Hyunsik Choi added a comment - - edited

        Even though the patch already was committed to the master, I leave some questions. If necessary, they would be reflected as new jira issues.

        1) Is one modification of TajoPullServer necessary? The change forces the pull server port to set the default port (0).

        2) How about adding some unit tests like TestDBStore?

        3) The patch uses hcatalog-0.5-incubating. However, hcatalog-0.11 is available. Is there any reason to use 0.5-incubating?

        Show
        Hyunsik Choi added a comment - - edited Even though the patch already was committed to the master, I leave some questions. If necessary, they would be reflected as new jira issues. 1) Is one modification of TajoPullServer necessary? The change forces the pull server port to set the default port (0). 2) How about adding some unit tests like TestDBStore? 3) The patch uses hcatalog-0.5-incubating. However, hcatalog-0.11 is available. Is there any reason to use 0.5-incubating?
        Hide
        Jaehwa Jung added a comment -

        Thanks Jihoon.
        I just committed.

        Show
        Jaehwa Jung added a comment - Thanks Jihoon. I just committed.
        Hide
        Jihoon Son added a comment -

        +1.

        When I tested the third patch with hadoop-core-2.0.0-mr1-cdh4.4.0.jar, it works well.

        Show
        Jihoon Son added a comment - +1. When I tested the third patch with hadoop-core-2.0.0-mr1-cdh4.4.0.jar, it works well.
        Hide
        Jaehwa Jung added a comment -

        I removed cdh mr1 jar file on '$

        {TAJO_HOME}

        /lib' because users may want to use another mr1 jar. After then, I'll write a document for this issue on Tajo wiki and I'll comment about cdh mr1 dependency. For reference, the cdh dependency on tajo-catalog-sever configuration is set up by 'provided'.

        Show
        Jaehwa Jung added a comment - I removed cdh mr1 jar file on '$ {TAJO_HOME} /lib' because users may want to use another mr1 jar. After then, I'll write a document for this issue on Tajo wiki and I'll comment about cdh mr1 dependency. For reference, the cdh dependency on tajo-catalog-sever configuration is set up by 'provided'.
        Hide
        Jihoon Son added a comment -

        I also think that adding the cdh dependency is a good temporal solution until Hive is updated to use the recent Hadoop APIs.
        If others have an idea, please give any advice.

        Show
        Jihoon Son added a comment - I also think that adding the cdh dependency is a good temporal solution until Hive is updated to use the recent Hadoop APIs. If others have an idea, please give any advice.
        Hide
        Jaehwa Jung added a comment -

        Thanks, Jihoon.

        I moved cloudera repository configuration to pom.xml file which located at $

        {TAJO_ROOT}

        . And I think that cdh dependency needs to improve. But at this present, Tajo has no choice but to depend on cdh. Because Tajo already includes Hadoop-2.x.x, Tajo cannot reference Hadoop-1.x.x. If Tajo references Hadoop-1.x.x, TajoMaster makes a error as follows:

        ERROR master.TajoMaster (TajoMaster.java:init(172)) - Server IPC version 8 cannot communicate with client version 4
        org.apache.hadoop.ipc.RemoteException: Server IPC version 8 cannot communicate with client version 4
        	at org.apache.hadoop.ipc.Client.call(Client.java:1113)
        	at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:229)
        	at com.sun.proxy.$Proxy5.getProtocolVersion(Unknown Source)
        	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
        	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        	at java.lang.reflect.Method.invoke(Method.java:597)
        	at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:85)
        	at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:62)
        	at com.sun.proxy.$Proxy5.getProtocolVersion(Unknown Source)
        	at org.apache.hadoop.ipc.RPC.checkVersion(RPC.java:422)
        	at org.apache.hadoop.hdfs.DFSClient.createNamenode(DFSClient.java:183)
        	at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:281)
        	at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:245)
        	at org.apache.hadoop.hdfs.DistributedFileSystem.initialize(DistributedFileSystem.java:100)
        	at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:1446)
        	at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:67)
        	at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:1464)
        	at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:263)
        	at org.apache.hadoop.fs.Path.getFileSystem(Path.java:187)
        	at org.apache.tajo.master.TajoMaster.checkAndInitializeSystemDirectories(TajoMaster.java:204)
        	at org.apache.tajo.master.TajoMaster.init(TajoMaster.java:152)
        	at org.apache.tajo.master.TajoMaster.main(TajoMaster.java:603)
        
        Show
        Jaehwa Jung added a comment - Thanks, Jihoon. I moved cloudera repository configuration to pom.xml file which located at $ {TAJO_ROOT} . And I think that cdh dependency needs to improve. But at this present, Tajo has no choice but to depend on cdh. Because Tajo already includes Hadoop-2.x.x, Tajo cannot reference Hadoop-1.x.x. If Tajo references Hadoop-1.x.x, TajoMaster makes a error as follows: ERROR master.TajoMaster (TajoMaster.java:init(172)) - Server IPC version 8 cannot communicate with client version 4 org.apache.hadoop.ipc.RemoteException: Server IPC version 8 cannot communicate with client version 4 at org.apache.hadoop.ipc.Client.call(Client.java:1113) at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:229) at com.sun.proxy.$Proxy5.getProtocolVersion(Unknown Source) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:85) at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:62) at com.sun.proxy.$Proxy5.getProtocolVersion(Unknown Source) at org.apache.hadoop.ipc.RPC.checkVersion(RPC.java:422) at org.apache.hadoop.hdfs.DFSClient.createNamenode(DFSClient.java:183) at org.apache.hadoop.hdfs.DFSClient. <init> (DFSClient.java:281) at org.apache.hadoop.hdfs.DFSClient. <init> (DFSClient.java:245) at org.apache.hadoop.hdfs.DistributedFileSystem.initialize(DistributedFileSystem.java:100) at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:1446) at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:67) at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:1464) at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:263) at org.apache.hadoop.fs.Path.getFileSystem(Path.java:187) at org.apache.tajo.master.TajoMaster.checkAndInitializeSystemDirectories(TajoMaster.java:204) at org.apache.tajo.master.TajoMaster.init(TajoMaster.java:152) at org.apache.tajo.master.TajoMaster.main(TajoMaster.java:603)
        Hide
        Jihoon Son added a comment -

        Thanks, JaeHwa.
        When I tested in my local machine, it works well.
        However, it will be great if we remove the cdh dependency.
        I'll try to replace the dependency to cdh with that to the recent Apache Hadoop.

        Also, because the repository is already configured in

        {TAJO_ROOT}/pom.xml, it would be better to move the added repository configuration to {TAJO_ROOT}

        /pom.xml.

        Show
        Jihoon Son added a comment - Thanks, JaeHwa. When I tested in my local machine, it works well. However, it will be great if we remove the cdh dependency. I'll try to replace the dependency to cdh with that to the recent Apache Hadoop. Also, because the repository is already configured in {TAJO_ROOT}/pom.xml, it would be better to move the added repository configuration to {TAJO_ROOT} /pom.xml.
        Hide
        Jaehwa Jung added a comment - - edited

        Thanks Jihoon.

        It caused by "tajo.catalog.uri" property.First, Hive uses 9083 as default HiveMetaStore port. Second, If you want to use another port as HiveMetaStore, you must run HiveMetaStore as follows

        ${HIVE_HOME}/bin/hive --service metastore -p [PORT]
        

        For examples, If you want to use 10001 port, you have specify as follows

        ${HIVE_HOME}/bin/hive --service metastore -p 10001
        

        Please check your Hive and Tajo configuration again.

        Show
        Jaehwa Jung added a comment - - edited Thanks Jihoon. It caused by "tajo.catalog.uri" property.First, Hive uses 9083 as default HiveMetaStore port. Second, If you want to use another port as HiveMetaStore, you must run HiveMetaStore as follows ${HIVE_HOME}/bin/hive --service metastore -p [PORT] For examples, If you want to use 10001 port, you have specify as follows ${HIVE_HOME}/bin/hive --service metastore -p 10001 Please check your Hive and Tajo configuration again.
        Hide
        Jihoon Son added a comment -

        Thanks for your patch.
        After applying your patch and adding the above catalog configuration, I met the following errors.

        tajo> \d
        2013-11-11 14:56:32,710 INFO  rpc.RpcConnectionPool (RpcConnectionPool.java:closeConnection(78)) - CloseConnection [class org.apache.tajo.ipc.TajoMasterClientProtocol,localhost/127.0.0.1:26002,false]
        2013-11-11 14:56:32,713 ERROR rpc.RpcProtos (BlockingRpcClient.java:exceptionCaught(205)) - RPC Exception:Exception [org.apache.tajo.ipc.TajoMasterClientProtocol(127.0.0.1:26002)]: org.apache.tajo.rpc.RetriesExhaustedException: Failed after attempts=3, exceptions:
        java.lang.NullPointerException
        java.lang.NullPointerException
        java.lang.NullPointerException
        
        	at org.apache.tajo.rpc.ServerCallable.withRetries(ServerCallable.java:91)
        	at org.apache.tajo.catalog.AbstractCatalogClient.getAllTableNames(AbstractCatalogClient.java:73)
        	at org.apache.tajo.master.TajoMasterClientService$TajoMasterClientProtocolServiceHandler.getTableList(TajoMasterClientService.java:273)
        	at org.apache.tajo.ipc.TajoMasterClientProtocol$TajoMasterClientProtocolService$2.callBlockingMethod(TajoMasterClientProtocol.java:244)
        	at org.apache.tajo.rpc.BlockingRpcServer$ServerHandler.messageReceived(BlockingRpcServer.java:91)
        	at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:296)
        	at org.jboss.netty.handler.codec.oneone.OneToOneDecoder.handleUpstream(OneToOneDecoder.java:70)
        	at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:296)
        	at org.jboss.netty.handler.codec.frame.FrameDecoder.unfoldAndFireMessageReceived(FrameDecoder.java:462)
        	at org.jboss.netty.handler.codec.frame.FrameDecoder.callDecode(FrameDecoder.java:443)
        	at org.jboss.netty.handler.codec.frame.FrameDecoder.messageReceived(FrameDecoder.java:303)
        	at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:268)
        	at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:255)
        	at org.jboss.netty.channel.socket.nio.NioWorker.read(NioWorker.java:88)
        	at org.jboss.netty.channel.socket.nio.AbstractNioWorker.process(AbstractNioWorker.java:109)
        	at org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:312)
        	at org.jboss.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:90)
        	at org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:178)
        	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1146)
        	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        	at java.lang.Thread.run(Thread.java:679)
        

        The master log is as follows.

        2013-11-11 14:55:36,538 INFO  rm.TajoWorkerResourceManager (TajoWorkerResourceManager.java:run(209)) - WorkerMonitor start
        2013-11-11 14:55:36,540 INFO  rm.TajoWorkerResourceManager (TajoWorkerResourceManager.java:run(278)) - WorkerResourceAllocationThread start
        SLF4J: Class path contains multiple SLF4J bindings.
        SLF4J: Found binding in [jar:file:{TAJO_HOME}/lib/slf4j-log4j12-1.6.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]
        SLF4J: Found binding in [jar:file:{HADOOP_HOME}/share/hadoop/common/lib/slf4j-log4j12-1.6.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]
        SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
        2013-11-11 14:55:41,584 INFO  mortbay.log (NativeMethodAccessorImpl.java:invoke0(?)) - Logging to org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog
        2013-11-11 14:55:41,626 INFO  webapp.HttpServer (HttpServer.java:start(333)) - Jetty bound to port 26080
        2013-11-11 14:55:41,626 INFO  mortbay.log (NativeMethodAccessorImpl.java:invoke0(?)) - jetty-6.1.14
        2013-11-11 14:55:45,426 INFO  mortbay.log (NativeMethodAccessorImpl.java:invoke0(?)) - Started SelectChannelConnector@0.0.0.0:26080
        2013-11-11 14:55:45,436 INFO  master.TajoMaster (TajoMaster.java:checkAndInitializeSystemDirectories(201)) - Tajo Root Directory: hdfs://localhost:7020/tajo
        2013-11-11 14:55:46,031 INFO  master.TajoMaster (TajoMaster.java:checkAndInitializeSystemDirectories(206)) - FileSystem (hdfs://localhost:7020) is initialized.
        2013-11-11 14:55:46,093 INFO  master.TajoMaster (TajoMaster.java:checkAndInitializeSystemDirectories(226)) - Tajo Warehouse dir: hdfs://localhost:7020/tajo/warehouse
        2013-11-11 14:55:46,095 INFO  master.TajoMaster (TajoMaster.java:checkAndInitializeSystemDirectories(235)) - Staging dir: hdfs://localhost:7020/tajo/warehouse
        2013-11-11 14:55:46,102 WARN  storage.AbstractStorageManager (AbstractStorageManager.java:<init>(85)) - does not support block metadata. ('dfs.datanode.hdfs-blocks-metadata.enabled')
        2013-11-11 14:55:46,135 INFO  service.AbstractService (AbstractService.java:init(81)) - Service:Dispatcher is inited.
        2013-11-11 14:55:46,138 INFO  catalog.CatalogServer (CatalogServer.java:init(108)) - Catalog Store Class: org.apache.tajo.catalog.store.HCatalogStore
        2013-11-11 14:55:46,141 INFO  service.AbstractService (AbstractService.java:init(81)) - Service:org.apache.tajo.catalog.CatalogServer is inited.
        2013-11-11 14:55:46,141 INFO  service.AbstractService (AbstractService.java:init(81)) - Service:org.apache.tajo.master.GlobalEngine is inited.
        2013-11-11 14:55:46,142 INFO  event.AsyncDispatcher (AsyncDispatcher.java:register(153)) - Registering class org.apache.tajo.master.querymaster.QueryJobEvent$Type for class org.apache.tajo.master.querymaster.QueryJobManager$QueryJobManagerEventHandler
        2013-11-11 14:55:46,143 INFO  service.AbstractService (AbstractService.java:init(81)) - Service:Dispatcher is inited.
        2013-11-11 14:55:46,143 INFO  service.AbstractService (AbstractService.java:init(81)) - Service:org.apache.tajo.master.querymaster.QueryJobManager is inited.
        2013-11-11 14:55:46,143 INFO  service.AbstractService (AbstractService.java:init(81)) - Service:org.apache.tajo.master.TajoMasterClientService is inited.
        2013-11-11 14:55:46,143 INFO  service.AbstractService (AbstractService.java:init(81)) - Service:org.apache.tajo.master.TajoMasterService is inited.
        2013-11-11 14:55:46,143 INFO  service.AbstractService (AbstractService.java:init(81)) - Service:org.apache.tajo.master.TajoMaster is inited.
        2013-11-11 14:55:46,143 INFO  master.TajoMaster (TajoMaster.java:init(177)) - Tajo Master is initialized.
        2013-11-11 14:55:46,143 INFO  master.TajoMaster (TajoMaster.java:start(412)) - TajoMaster startup
        2013-11-11 14:55:46,143 INFO  service.AbstractService (AbstractService.java:start(94)) - Service:Dispatcher is started.
        2013-11-11 14:55:46,201 INFO  rpc.NettyServerBase (NettyServerBase.java:start(106)) - Rpc (CatalogProtocol) listens on /127.0.0.1:26005
        2013-11-11 14:55:46,201 INFO  catalog.CatalogServer (CatalogServer.java:start(156)) - Catalog Server startup (127.0.0.1:26005)
        2013-11-11 14:55:46,201 INFO  service.AbstractService (AbstractService.java:start(94)) - Service:org.apache.tajo.catalog.CatalogServer is started.
        2013-11-11 14:55:46,247 INFO  service.AbstractService (AbstractService.java:start(94)) - Service:org.apache.tajo.master.GlobalEngine is started.
        2013-11-11 14:55:46,247 INFO  service.AbstractService (AbstractService.java:start(94)) - Service:Dispatcher is started.
        2013-11-11 14:55:46,247 INFO  service.AbstractService (AbstractService.java:start(94)) - Service:org.apache.tajo.master.querymaster.QueryJobManager is started.
        2013-11-11 14:55:46,256 INFO  rpc.NettyServerBase (NettyServerBase.java:start(106)) - Rpc (TajoMasterClientProtocol) listens on /127.0.0.1:26002
        2013-11-11 14:55:46,256 INFO  master.TajoMasterClientService (TajoMasterClientService.java:start(90)) - Instantiated TajoMasterClientService at /127.0.0.1:26002
        2013-11-11 14:55:46,257 INFO  service.AbstractService (AbstractService.java:start(94)) - Service:org.apache.tajo.master.TajoMasterClientService is started.
        2013-11-11 14:55:46,265 INFO  rpc.NettyServerBase (NettyServerBase.java:start(106)) - Rpc (TajoMasterProtocol) listens on /127.0.0.1:26001
        2013-11-11 14:55:46,265 INFO  master.TajoMasterService (TajoMasterService.java:start(72)) - Instantiated TajoMasterService at /127.0.0.1:26001
        2013-11-11 14:55:46,266 INFO  service.AbstractService (AbstractService.java:start(94)) - Service:org.apache.tajo.master.TajoMasterService is started.
        2013-11-11 14:55:46,266 INFO  service.AbstractService (AbstractService.java:start(94)) - Service:org.apache.tajo.master.TajoMaster is started.
        2013-11-11 14:55:46,318 WARN  conf.Configuration (Configuration.java:warnOnceIfDeprecated(817)) - fs.default.name is deprecated. Instead, use fs.defaultFS
        2013-11-11 14:55:49,754 INFO  rm.TajoWorkerResourceManager (TajoWorkerResourceManager.java:workerHeartbeat(487)) - TajoWorker:host:localhost, port=28093,28091,28092,50461, slots=1024:1:1, used=0:0:0 added in live TajoWorker list
        2013-11-11 14:56:22,035 INFO  hive.metastore (HiveMetaStoreClient.java:open(244)) - Trying to connect to metastore with URI thrift://localhost:10001
        2013-11-11 14:56:22,049 WARN  hive.metastore (HiveMetaStoreClient.java:open(291)) - Failed to connect to the MetaStore Server...
        2013-11-11 14:56:22,049 INFO  hive.metastore (HiveMetaStoreClient.java:open(322)) - Waiting 1 seconds before next connection attempt.
        2013-11-11 14:56:23,049 INFO  hive.metastore (HiveMetaStoreClient.java:open(244)) - Trying to connect to metastore with URI thrift://localhost:10001
        2013-11-11 14:56:23,050 WARN  hive.metastore (HiveMetaStoreClient.java:open(291)) - Failed to connect to the MetaStore Server...
        2013-11-11 14:56:23,050 INFO  hive.metastore (HiveMetaStoreClient.java:open(322)) - Waiting 1 seconds before next connection attempt.
        2013-11-11 14:56:24,051 INFO  hive.metastore (HiveMetaStoreClient.java:open(244)) - Trying to connect to metastore with URI thrift://localhost:10001
        2013-11-11 14:56:24,052 WARN  hive.metastore (HiveMetaStoreClient.java:open(291)) - Failed to connect to the MetaStore Server...
        2013-11-11 14:56:24,052 INFO  hive.metastore (HiveMetaStoreClient.java:open(322)) - Waiting 1 seconds before next connection attempt.
        2013-11-11 14:56:25,631 INFO  hive.metastore (HiveMetaStoreClient.java:open(244)) - Trying to connect to metastore with URI thrift://localhost:10001
        2013-11-11 14:56:25,631 WARN  hive.metastore (HiveMetaStoreClient.java:open(291)) - Failed to connect to the MetaStore Server...
        2013-11-11 14:56:25,631 INFO  hive.metastore (HiveMetaStoreClient.java:open(322)) - Waiting 1 seconds before next connection attempt.
        2013-11-11 14:56:26,631 INFO  hive.metastore (HiveMetaStoreClient.java:open(244)) - Trying to connect to metastore with URI thrift://localhost:10001
        2013-11-11 14:56:26,632 WARN  hive.metastore (HiveMetaStoreClient.java:open(291)) - Failed to connect to the MetaStore Server...
        2013-11-11 14:56:26,632 INFO  hive.metastore (HiveMetaStoreClient.java:open(322)) - Waiting 1 seconds before next connection attempt.
        2013-11-11 14:56:27,633 INFO  hive.metastore (HiveMetaStoreClient.java:open(244)) - Trying to connect to metastore with URI thrift://localhost:10001
        2013-11-11 14:56:27,634 WARN  hive.metastore (HiveMetaStoreClient.java:open(291)) - Failed to connect to the MetaStore Server...
        2013-11-11 14:56:27,634 INFO  hive.metastore (HiveMetaStoreClient.java:open(322)) - Waiting 1 seconds before next connection attempt.
        2013-11-11 14:56:29,693 INFO  hive.metastore (HiveMetaStoreClient.java:open(244)) - Trying to connect to metastore with URI thrift://localhost:10001
        2013-11-11 14:56:29,694 WARN  hive.metastore (HiveMetaStoreClient.java:open(291)) - Failed to connect to the MetaStore Server...
        2013-11-11 14:56:29,694 INFO  hive.metastore (HiveMetaStoreClient.java:open(322)) - Waiting 1 seconds before next connection attempt.
        2013-11-11 14:56:30,694 INFO  hive.metastore (HiveMetaStoreClient.java:open(244)) - Trying to connect to metastore with URI thrift://localhost:10001
        2013-11-11 14:56:30,695 WARN  hive.metastore (HiveMetaStoreClient.java:open(291)) - Failed to connect to the MetaStore Server...
        2013-11-11 14:56:30,695 INFO  hive.metastore (HiveMetaStoreClient.java:open(322)) - Waiting 1 seconds before next connection attempt.
        2013-11-11 14:56:31,695 INFO  hive.metastore (HiveMetaStoreClient.java:open(244)) - Trying to connect to metastore with URI thrift://localhost:10001
        2013-11-11 14:56:31,696 WARN  hive.metastore (HiveMetaStoreClient.java:open(291)) - Failed to connect to the MetaStore Server...
        2013-11-11 14:56:31,696 INFO  hive.metastore (HiveMetaStoreClient.java:open(322)) - Waiting 1 seconds before next connection attempt.
        2013. 11. 11 오후 2:56:32 org.jboss.netty.channel.DefaultChannelPipeline
        경고: An exception was thrown by a user handler while handling an exception event ([id: 0xbe6bfd26, /127.0.0.1:39838 => /127.0.0.1:26002] EXCEPTION: org.apache.tajo.rpc.RemoteCallException: Remote call error occurs when TajoMasterClientProtocolService.getTableListis called:)
        org.apache.tajo.rpc.RemoteException: org.apache.tajo.rpc.RemoteCallException: Remote call error occurs when TajoMasterClientProtocolService.getTableListis called:
        	at org.apache.tajo.rpc.BlockingRpcServer$ServerHandler.exceptionCaught(BlockingRpcServer.java:117)
        	at org.jboss.netty.handler.codec.oneone.OneToOneDecoder.handleUpstream(OneToOneDecoder.java:60)
        	at org.jboss.netty.handler.codec.frame.FrameDecoder.exceptionCaught(FrameDecoder.java:377)
        	at org.jboss.netty.channel.Channels.fireExceptionCaught(Channels.java:525)
        	at org.jboss.netty.channel.AbstractChannelSink.exceptionCaught(AbstractChannelSink.java:48)
        	at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:296)
        	at org.jboss.netty.handler.codec.oneone.OneToOneDecoder.handleUpstream(OneToOneDecoder.java:70)
        	at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:296)
        	at org.jboss.netty.handler.codec.frame.FrameDecoder.unfoldAndFireMessageReceived(FrameDecoder.java:462)
        	at org.jboss.netty.handler.codec.frame.FrameDecoder.callDecode(FrameDecoder.java:443)
        	at org.jboss.netty.handler.codec.frame.FrameDecoder.messageReceived(FrameDecoder.java:303)
        	at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:268)
        	at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:255)
        	at org.jboss.netty.channel.socket.nio.NioWorker.read(NioWorker.java:88)
        	at org.jboss.netty.channel.socket.nio.AbstractNioWorker.process(AbstractNioWorker.java:109)
        	at org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:312)
        	at org.jboss.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:90)
        	at org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:178)
        	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1146)
        	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        	at java.lang.Thread.run(Thread.java:679)
        Caused by: org.apache.tajo.rpc.RemoteCallException: Remote call error occurs when TajoMasterClientProtocolService.getTableListis called:
        	at org.apache.tajo.rpc.BlockingRpcServer$ServerHandler.messageReceived(BlockingRpcServer.java:94)
        	... 16 more
        Caused by: org.apache.tajo.rpc.RetriesExhaustedException: Failed after attempts=3, exceptions:
        java.lang.NullPointerException
        java.lang.NullPointerException
        java.lang.NullPointerException
        
        	at org.apache.tajo.rpc.ServerCallable.withRetries(ServerCallable.java:91)
        	at org.apache.tajo.catalog.AbstractCatalogClient.getAllTableNames(AbstractCatalogClient.java:73)
        	at org.apache.tajo.master.TajoMasterClientService$TajoMasterClientProtocolServiceHandler.getTableList(TajoMasterClientService.java:273)
        	at org.apache.tajo.ipc.TajoMasterClientProtocol$TajoMasterClientProtocolService$2.callBlockingMethod(TajoMasterClientProtocol.java:244)
        	at org.apache.tajo.rpc.BlockingRpcServer$ServerHandler.messageReceived(BlockingRpcServer.java:91)
        	... 16 more
        2013-11-11 14:56:33,317 ERROR hive.metastore (HiveMetaStoreClient.java:close(346)) - Unable to shutdown local metastore client
        org.apache.thrift.transport.TTransportException: Cannot write to null outputStream
        	at org.apache.thrift.transport.TIOStreamTransport.write(TIOStreamTransport.java:142)
        	at org.apache.thrift.protocol.TBinaryProtocol.writeI32(TBinaryProtocol.java:163)
        	at org.apache.thrift.protocol.TBinaryProtocol.writeMessageBegin(TBinaryProtocol.java:91)
        	at org.apache.thrift.TServiceClient.sendBase(TServiceClient.java:62)
        	at com.facebook.fb303.FacebookService$Client.send_shutdown(FacebookService.java:431)
        	at com.facebook.fb303.FacebookService$Client.shutdown(FacebookService.java:425)
        	at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.close(HiveMetaStoreClient.java:343)
        	at org.apache.hcatalog.common.HiveClientCache$CacheableHiveMetaStoreClient.tearDown(HiveClientCache.java:316)
        	at org.apache.hcatalog.common.HiveClientCache$CacheableHiveMetaStoreClient.finalize(HiveClientCache.java:331)
        	at java.lang.ref.Finalizer.invokeFinalizeMethod(Native Method)
        	at java.lang.ref.Finalizer.runFinalizer(Finalizer.java:101)
        	at java.lang.ref.Finalizer.access$100(Finalizer.java:32)
        	at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:190)
        
        Show
        Jihoon Son added a comment - Thanks for your patch. After applying your patch and adding the above catalog configuration, I met the following errors. tajo> \d 2013-11-11 14:56:32,710 INFO rpc.RpcConnectionPool (RpcConnectionPool.java:closeConnection(78)) - CloseConnection [class org.apache.tajo.ipc.TajoMasterClientProtocol,localhost/127.0.0.1:26002, false ] 2013-11-11 14:56:32,713 ERROR rpc.RpcProtos (BlockingRpcClient.java:exceptionCaught(205)) - RPC Exception:Exception [org.apache.tajo.ipc.TajoMasterClientProtocol(127.0.0.1:26002)]: org.apache.tajo.rpc.RetriesExhaustedException: Failed after attempts=3, exceptions: java.lang.NullPointerException java.lang.NullPointerException java.lang.NullPointerException at org.apache.tajo.rpc.ServerCallable.withRetries(ServerCallable.java:91) at org.apache.tajo.catalog.AbstractCatalogClient.getAllTableNames(AbstractCatalogClient.java:73) at org.apache.tajo.master.TajoMasterClientService$TajoMasterClientProtocolServiceHandler.getTableList(TajoMasterClientService.java:273) at org.apache.tajo.ipc.TajoMasterClientProtocol$TajoMasterClientProtocolService$2.callBlockingMethod(TajoMasterClientProtocol.java:244) at org.apache.tajo.rpc.BlockingRpcServer$ServerHandler.messageReceived(BlockingRpcServer.java:91) at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:296) at org.jboss.netty.handler.codec.oneone.OneToOneDecoder.handleUpstream(OneToOneDecoder.java:70) at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:296) at org.jboss.netty.handler.codec.frame.FrameDecoder.unfoldAndFireMessageReceived(FrameDecoder.java:462) at org.jboss.netty.handler.codec.frame.FrameDecoder.callDecode(FrameDecoder.java:443) at org.jboss.netty.handler.codec.frame.FrameDecoder.messageReceived(FrameDecoder.java:303) at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:268) at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:255) at org.jboss.netty.channel.socket.nio.NioWorker.read(NioWorker.java:88) at org.jboss.netty.channel.socket.nio.AbstractNioWorker.process(AbstractNioWorker.java:109) at org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:312) at org.jboss.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:90) at org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:178) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1146) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang. Thread .run( Thread .java:679) The master log is as follows. 2013-11-11 14:55:36,538 INFO rm.TajoWorkerResourceManager (TajoWorkerResourceManager.java:run(209)) - WorkerMonitor start 2013-11-11 14:55:36,540 INFO rm.TajoWorkerResourceManager (TajoWorkerResourceManager.java:run(278)) - WorkerResourceAllocationThread start SLF4J: Class path contains multiple SLF4J bindings. SLF4J: Found binding in [jar:file:{TAJO_HOME}/lib/slf4j-log4j12-1.6.1.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: Found binding in [jar:file:{HADOOP_HOME}/share/hadoop/common/lib/slf4j-log4j12-1.6.1.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: See http: //www.slf4j.org/codes.html#multiple_bindings for an explanation. 2013-11-11 14:55:41,584 INFO mortbay.log (NativeMethodAccessorImpl.java:invoke0(?)) - Logging to org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog 2013-11-11 14:55:41,626 INFO webapp.HttpServer (HttpServer.java:start(333)) - Jetty bound to port 26080 2013-11-11 14:55:41,626 INFO mortbay.log (NativeMethodAccessorImpl.java:invoke0(?)) - jetty-6.1.14 2013-11-11 14:55:45,426 INFO mortbay.log (NativeMethodAccessorImpl.java:invoke0(?)) - Started SelectChannelConnector@0.0.0.0:26080 2013-11-11 14:55:45,436 INFO master.TajoMaster (TajoMaster.java:checkAndInitializeSystemDirectories(201)) - Tajo Root Directory: hdfs: //localhost:7020/tajo 2013-11-11 14:55:46,031 INFO master.TajoMaster (TajoMaster.java:checkAndInitializeSystemDirectories(206)) - FileSystem (hdfs: //localhost:7020) is initialized. 2013-11-11 14:55:46,093 INFO master.TajoMaster (TajoMaster.java:checkAndInitializeSystemDirectories(226)) - Tajo Warehouse dir: hdfs: //localhost:7020/tajo/warehouse 2013-11-11 14:55:46,095 INFO master.TajoMaster (TajoMaster.java:checkAndInitializeSystemDirectories(235)) - Staging dir: hdfs: //localhost:7020/tajo/warehouse 2013-11-11 14:55:46,102 WARN storage.AbstractStorageManager (AbstractStorageManager.java:<init>(85)) - does not support block metadata. ('dfs.datanode.hdfs-blocks-metadata.enabled') 2013-11-11 14:55:46,135 INFO service.AbstractService (AbstractService.java:init(81)) - Service:Dispatcher is inited. 2013-11-11 14:55:46,138 INFO catalog.CatalogServer (CatalogServer.java:init(108)) - Catalog Store Class : org.apache.tajo.catalog.store.HCatalogStore 2013-11-11 14:55:46,141 INFO service.AbstractService (AbstractService.java:init(81)) - Service:org.apache.tajo.catalog.CatalogServer is inited. 2013-11-11 14:55:46,141 INFO service.AbstractService (AbstractService.java:init(81)) - Service:org.apache.tajo.master.GlobalEngine is inited. 2013-11-11 14:55:46,142 INFO event.AsyncDispatcher (AsyncDispatcher.java:register(153)) - Registering class org.apache.tajo.master.querymaster.QueryJobEvent$Type for class org.apache.tajo.master.querymaster.QueryJobManager$QueryJobManagerEventHandler 2013-11-11 14:55:46,143 INFO service.AbstractService (AbstractService.java:init(81)) - Service:Dispatcher is inited. 2013-11-11 14:55:46,143 INFO service.AbstractService (AbstractService.java:init(81)) - Service:org.apache.tajo.master.querymaster.QueryJobManager is inited. 2013-11-11 14:55:46,143 INFO service.AbstractService (AbstractService.java:init(81)) - Service:org.apache.tajo.master.TajoMasterClientService is inited. 2013-11-11 14:55:46,143 INFO service.AbstractService (AbstractService.java:init(81)) - Service:org.apache.tajo.master.TajoMasterService is inited. 2013-11-11 14:55:46,143 INFO service.AbstractService (AbstractService.java:init(81)) - Service:org.apache.tajo.master.TajoMaster is inited. 2013-11-11 14:55:46,143 INFO master.TajoMaster (TajoMaster.java:init(177)) - Tajo Master is initialized. 2013-11-11 14:55:46,143 INFO master.TajoMaster (TajoMaster.java:start(412)) - TajoMaster startup 2013-11-11 14:55:46,143 INFO service.AbstractService (AbstractService.java:start(94)) - Service:Dispatcher is started. 2013-11-11 14:55:46,201 INFO rpc.NettyServerBase (NettyServerBase.java:start(106)) - Rpc (CatalogProtocol) listens on /127.0.0.1:26005 2013-11-11 14:55:46,201 INFO catalog.CatalogServer (CatalogServer.java:start(156)) - Catalog Server startup (127.0.0.1:26005) 2013-11-11 14:55:46,201 INFO service.AbstractService (AbstractService.java:start(94)) - Service:org.apache.tajo.catalog.CatalogServer is started. 2013-11-11 14:55:46,247 INFO service.AbstractService (AbstractService.java:start(94)) - Service:org.apache.tajo.master.GlobalEngine is started. 2013-11-11 14:55:46,247 INFO service.AbstractService (AbstractService.java:start(94)) - Service:Dispatcher is started. 2013-11-11 14:55:46,247 INFO service.AbstractService (AbstractService.java:start(94)) - Service:org.apache.tajo.master.querymaster.QueryJobManager is started. 2013-11-11 14:55:46,256 INFO rpc.NettyServerBase (NettyServerBase.java:start(106)) - Rpc (TajoMasterClientProtocol) listens on /127.0.0.1:26002 2013-11-11 14:55:46,256 INFO master.TajoMasterClientService (TajoMasterClientService.java:start(90)) - Instantiated TajoMasterClientService at /127.0.0.1:26002 2013-11-11 14:55:46,257 INFO service.AbstractService (AbstractService.java:start(94)) - Service:org.apache.tajo.master.TajoMasterClientService is started. 2013-11-11 14:55:46,265 INFO rpc.NettyServerBase (NettyServerBase.java:start(106)) - Rpc (TajoMasterProtocol) listens on /127.0.0.1:26001 2013-11-11 14:55:46,265 INFO master.TajoMasterService (TajoMasterService.java:start(72)) - Instantiated TajoMasterService at /127.0.0.1:26001 2013-11-11 14:55:46,266 INFO service.AbstractService (AbstractService.java:start(94)) - Service:org.apache.tajo.master.TajoMasterService is started. 2013-11-11 14:55:46,266 INFO service.AbstractService (AbstractService.java:start(94)) - Service:org.apache.tajo.master.TajoMaster is started. 2013-11-11 14:55:46,318 WARN conf.Configuration (Configuration.java:warnOnceIfDeprecated(817)) - fs. default .name is deprecated. Instead, use fs.defaultFS 2013-11-11 14:55:49,754 INFO rm.TajoWorkerResourceManager (TajoWorkerResourceManager.java:workerHeartbeat(487)) - TajoWorker:host:localhost, port=28093,28091,28092,50461, slots=1024:1:1, used=0:0:0 added in live TajoWorker list 2013-11-11 14:56:22,035 INFO hive.metastore (HiveMetaStoreClient.java:open(244)) - Trying to connect to metastore with URI thrift: //localhost:10001 2013-11-11 14:56:22,049 WARN hive.metastore (HiveMetaStoreClient.java:open(291)) - Failed to connect to the MetaStore Server... 2013-11-11 14:56:22,049 INFO hive.metastore (HiveMetaStoreClient.java:open(322)) - Waiting 1 seconds before next connection attempt. 2013-11-11 14:56:23,049 INFO hive.metastore (HiveMetaStoreClient.java:open(244)) - Trying to connect to metastore with URI thrift: //localhost:10001 2013-11-11 14:56:23,050 WARN hive.metastore (HiveMetaStoreClient.java:open(291)) - Failed to connect to the MetaStore Server... 2013-11-11 14:56:23,050 INFO hive.metastore (HiveMetaStoreClient.java:open(322)) - Waiting 1 seconds before next connection attempt. 2013-11-11 14:56:24,051 INFO hive.metastore (HiveMetaStoreClient.java:open(244)) - Trying to connect to metastore with URI thrift: //localhost:10001 2013-11-11 14:56:24,052 WARN hive.metastore (HiveMetaStoreClient.java:open(291)) - Failed to connect to the MetaStore Server... 2013-11-11 14:56:24,052 INFO hive.metastore (HiveMetaStoreClient.java:open(322)) - Waiting 1 seconds before next connection attempt. 2013-11-11 14:56:25,631 INFO hive.metastore (HiveMetaStoreClient.java:open(244)) - Trying to connect to metastore with URI thrift: //localhost:10001 2013-11-11 14:56:25,631 WARN hive.metastore (HiveMetaStoreClient.java:open(291)) - Failed to connect to the MetaStore Server... 2013-11-11 14:56:25,631 INFO hive.metastore (HiveMetaStoreClient.java:open(322)) - Waiting 1 seconds before next connection attempt. 2013-11-11 14:56:26,631 INFO hive.metastore (HiveMetaStoreClient.java:open(244)) - Trying to connect to metastore with URI thrift: //localhost:10001 2013-11-11 14:56:26,632 WARN hive.metastore (HiveMetaStoreClient.java:open(291)) - Failed to connect to the MetaStore Server... 2013-11-11 14:56:26,632 INFO hive.metastore (HiveMetaStoreClient.java:open(322)) - Waiting 1 seconds before next connection attempt. 2013-11-11 14:56:27,633 INFO hive.metastore (HiveMetaStoreClient.java:open(244)) - Trying to connect to metastore with URI thrift: //localhost:10001 2013-11-11 14:56:27,634 WARN hive.metastore (HiveMetaStoreClient.java:open(291)) - Failed to connect to the MetaStore Server... 2013-11-11 14:56:27,634 INFO hive.metastore (HiveMetaStoreClient.java:open(322)) - Waiting 1 seconds before next connection attempt. 2013-11-11 14:56:29,693 INFO hive.metastore (HiveMetaStoreClient.java:open(244)) - Trying to connect to metastore with URI thrift: //localhost:10001 2013-11-11 14:56:29,694 WARN hive.metastore (HiveMetaStoreClient.java:open(291)) - Failed to connect to the MetaStore Server... 2013-11-11 14:56:29,694 INFO hive.metastore (HiveMetaStoreClient.java:open(322)) - Waiting 1 seconds before next connection attempt. 2013-11-11 14:56:30,694 INFO hive.metastore (HiveMetaStoreClient.java:open(244)) - Trying to connect to metastore with URI thrift: //localhost:10001 2013-11-11 14:56:30,695 WARN hive.metastore (HiveMetaStoreClient.java:open(291)) - Failed to connect to the MetaStore Server... 2013-11-11 14:56:30,695 INFO hive.metastore (HiveMetaStoreClient.java:open(322)) - Waiting 1 seconds before next connection attempt. 2013-11-11 14:56:31,695 INFO hive.metastore (HiveMetaStoreClient.java:open(244)) - Trying to connect to metastore with URI thrift: //localhost:10001 2013-11-11 14:56:31,696 WARN hive.metastore (HiveMetaStoreClient.java:open(291)) - Failed to connect to the MetaStore Server... 2013-11-11 14:56:31,696 INFO hive.metastore (HiveMetaStoreClient.java:open(322)) - Waiting 1 seconds before next connection attempt. 2013. 11. 11 오후 2:56:32 org.jboss.netty.channel.DefaultChannelPipeline 경고: An exception was thrown by a user handler while handling an exception event ([id: 0xbe6bfd26, /127.0.0.1:39838 => /127.0.0.1:26002] EXCEPTION: org.apache.tajo.rpc.RemoteCallException: Remote call error occurs when TajoMasterClientProtocolService.getTableListis called:) org.apache.tajo.rpc.RemoteException: org.apache.tajo.rpc.RemoteCallException: Remote call error occurs when TajoMasterClientProtocolService.getTableListis called: at org.apache.tajo.rpc.BlockingRpcServer$ServerHandler.exceptionCaught(BlockingRpcServer.java:117) at org.jboss.netty.handler.codec.oneone.OneToOneDecoder.handleUpstream(OneToOneDecoder.java:60) at org.jboss.netty.handler.codec.frame.FrameDecoder.exceptionCaught(FrameDecoder.java:377) at org.jboss.netty.channel.Channels.fireExceptionCaught(Channels.java:525) at org.jboss.netty.channel.AbstractChannelSink.exceptionCaught(AbstractChannelSink.java:48) at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:296) at org.jboss.netty.handler.codec.oneone.OneToOneDecoder.handleUpstream(OneToOneDecoder.java:70) at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:296) at org.jboss.netty.handler.codec.frame.FrameDecoder.unfoldAndFireMessageReceived(FrameDecoder.java:462) at org.jboss.netty.handler.codec.frame.FrameDecoder.callDecode(FrameDecoder.java:443) at org.jboss.netty.handler.codec.frame.FrameDecoder.messageReceived(FrameDecoder.java:303) at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:268) at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:255) at org.jboss.netty.channel.socket.nio.NioWorker.read(NioWorker.java:88) at org.jboss.netty.channel.socket.nio.AbstractNioWorker.process(AbstractNioWorker.java:109) at org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:312) at org.jboss.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:90) at org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:178) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1146) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang. Thread .run( Thread .java:679) Caused by: org.apache.tajo.rpc.RemoteCallException: Remote call error occurs when TajoMasterClientProtocolService.getTableListis called: at org.apache.tajo.rpc.BlockingRpcServer$ServerHandler.messageReceived(BlockingRpcServer.java:94) ... 16 more Caused by: org.apache.tajo.rpc.RetriesExhaustedException: Failed after attempts=3, exceptions: java.lang.NullPointerException java.lang.NullPointerException java.lang.NullPointerException at org.apache.tajo.rpc.ServerCallable.withRetries(ServerCallable.java:91) at org.apache.tajo.catalog.AbstractCatalogClient.getAllTableNames(AbstractCatalogClient.java:73) at org.apache.tajo.master.TajoMasterClientService$TajoMasterClientProtocolServiceHandler.getTableList(TajoMasterClientService.java:273) at org.apache.tajo.ipc.TajoMasterClientProtocol$TajoMasterClientProtocolService$2.callBlockingMethod(TajoMasterClientProtocol.java:244) at org.apache.tajo.rpc.BlockingRpcServer$ServerHandler.messageReceived(BlockingRpcServer.java:91) ... 16 more 2013-11-11 14:56:33,317 ERROR hive.metastore (HiveMetaStoreClient.java:close(346)) - Unable to shutdown local metastore client org.apache.thrift.transport.TTransportException: Cannot write to null outputStream at org.apache.thrift.transport.TIOStreamTransport.write(TIOStreamTransport.java:142) at org.apache.thrift.protocol.TBinaryProtocol.writeI32(TBinaryProtocol.java:163) at org.apache.thrift.protocol.TBinaryProtocol.writeMessageBegin(TBinaryProtocol.java:91) at org.apache.thrift.TServiceClient.sendBase(TServiceClient.java:62) at com.facebook.fb303.FacebookService$Client.send_shutdown(FacebookService.java:431) at com.facebook.fb303.FacebookService$Client.shutdown(FacebookService.java:425) at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.close(HiveMetaStoreClient.java:343) at org.apache.hcatalog.common.HiveClientCache$CacheableHiveMetaStoreClient.tearDown(HiveClientCache.java:316) at org.apache.hcatalog.common.HiveClientCache$CacheableHiveMetaStoreClient.finalize(HiveClientCache.java:331) at java.lang.ref.Finalizer.invokeFinalizeMethod(Native Method) at java.lang.ref.Finalizer.runFinalizer(Finalizer.java:101) at java.lang.ref.Finalizer.access$100(Finalizer.java:32) at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:190)
        Hide
        Jaehwa Jung added a comment -

        I updated the design document.

        Show
        Jaehwa Jung added a comment - I updated the design document.
        Hide
        Jaehwa Jung added a comment -

        I'm so glad to upload this patch.

        From now on, Tajo can connect HiveMetastore and can get lots of data stored on HDFS through HiveMetaStore. Of course, there were some limitation as follows:

        • current version support just text file format.
        • current version support just uncompressed type.

        But, I'll implement unsupported functions time soon.

        If you want to use HiveMetaStore as CatalogServer, you have to start up HiveMetaStore server as follows:

        $HIVE_HOME/bin/hive --service metastore
        

        Or you can start up HiveMetaStore server through HCatalog as follows:

        $HCATALOG_HOME/sbin/hcat_server.sh start
        

        If you start up HCatalog server, HCatalogServer start up HiveMetaStore server actually.

        And then, you have to update catalog-site.xml as follows:

          <property>
            <name>tajo.catalog.store.class</name>
            <value>org.apache.tajo.catalog.store.HCatalogStore</value>
          </property>
          <property>
            <name>tajo.catalog.uri</name>
            <value>thrift://localhost:10001</value>
          </property>
        

        'tajo.catalog.url' is HiveMetaStore sever uri. You must specify your address on this property.

        And you can build Tajo in accordance with your hive version. If you want to build Tajo in accordance with apache hive 0.11.0, you can build as follows:

        mvn clean install -Phive-0.11.0
        

        Or if you want to build Tajo in accordance with hive-0.10.0 for cdh4.3.0, you can build as follows:

        mvm clean install -Phive-0.10.0-cdh4.3.0
        

        In other version, you must modify pom.xml file which located at '$TAJO_HOME/tajo-catalog/tajo-catalog-server'.

        Show
        Jaehwa Jung added a comment - I'm so glad to upload this patch. From now on, Tajo can connect HiveMetastore and can get lots of data stored on HDFS through HiveMetaStore. Of course, there were some limitation as follows: current version support just text file format. current version support just uncompressed type. But, I'll implement unsupported functions time soon. If you want to use HiveMetaStore as CatalogServer, you have to start up HiveMetaStore server as follows: $HIVE_HOME/bin/hive --service metastore Or you can start up HiveMetaStore server through HCatalog as follows: $HCATALOG_HOME/sbin/hcat_server.sh start If you start up HCatalog server, HCatalogServer start up HiveMetaStore server actually. And then, you have to update catalog-site.xml as follows: <property> <name> tajo.catalog.store.class </name> <value> org.apache.tajo.catalog.store.HCatalogStore </value> </property> <property> <name> tajo.catalog.uri </name> <value> thrift://localhost:10001 </value> </property> 'tajo.catalog.url' is HiveMetaStore sever uri. You must specify your address on this property. And you can build Tajo in accordance with your hive version. If you want to build Tajo in accordance with apache hive 0.11.0, you can build as follows: mvn clean install -Phive-0.11.0 Or if you want to build Tajo in accordance with hive-0.10.0 for cdh4.3.0, you can build as follows: mvm clean install -Phive-0.10.0-cdh4.3.0 In other version, you must modify pom.xml file which located at '$TAJO_HOME/tajo-catalog/tajo-catalog-server'.
        Hide
        Jaehwa Jung added a comment -

        I considered various ways to integrate Tajo Catalog with Hive Metastore. Finally, I chose HCatalog to resolve it. I attached the design document. Please, check it.

        Show
        Jaehwa Jung added a comment - I considered various ways to integrate Tajo Catalog with Hive Metastore. Finally, I chose HCatalog to resolve it. I attached the design document. Please, check it.
        Hide
        Jinho Kim added a comment -

        ++1 for this feature!!!

        Show
        Jinho Kim added a comment - ++1 for this feature!!!
        Hide
        Hyunsik Choi added a comment -

        ++1 for this feature.

        Show
        Hyunsik Choi added a comment - ++1 for this feature.

          People

          • Assignee:
            Jaehwa Jung
            Reporter:
            Jaehwa Jung
          • Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development