Uploaded image for project: 'Tajo'
  1. Tajo
  2. TAJO-995

HiveMetaStoreClient wrapper should retry the connection

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 0.9.0
    • Fix Version/s: 0.9.0
    • Component/s: Catalog
    • Labels:
      None

      Description

      We should change to RetryingMetaStoreClient
      refer to https://issues.apache.org/jira/browse/HIVE-6390

      Caused by: com.google.protobuf.ServiceException: org.apache.tajo.catalog.exception.CatalogException: org.apache.thrift.transport.TTransportException: java.net.SocketTimeoutException: Read timed out
              at org.apache.tajo.catalog.CatalogServer$CatalogProtocolHandler.getTableDesc(CatalogServer.java:464)
              at org.apache.tajo.catalog.AbstractCatalogClient$11.call(AbstractCatalogClient.java:233)
              at org.apache.tajo.catalog.AbstractCatalogClient$11.call(AbstractCatalogClient.java:226)
              at org.apache.tajo.rpc.ServerCallable.withRetries(ServerCallable.java:95)
              ... 65 more
      Caused by: org.apache.tajo.catalog.exception.CatalogException: org.apache.thrift.transport.TTransportException: java.net.SocketTimeoutException: Read timed out
              at org.apache.tajo.catalog.store.HCatalogStore.existTable(HCatalogStore.java:93)
              at org.apache.tajo.catalog.CatalogServer$CatalogProtocolHandler.getTableDesc(CatalogServer.java:453)
              ... 68 more
      Caused by: org.apache.thrift.transport.TTransportException: java.net.SocketTimeoutException: Read timed out
              at org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:129)
              at org.apache.thrift.transport.TTransport.readAll(TTransport.java:84)
              at org.apache.thrift.protocol.TBinaryProtocol.readAll(TBinaryProtocol.java:378)
              at org.apache.thrift.protocol.TBinaryProtocol.readByte(TBinaryProtocol.java:264)
              at org.apache.thrift.protocol.TBinaryProtocol.readFieldBegin(TBinaryProtocol.java:228)
              at org.apache.hadoop.hive.metastore.api.FieldSchema$FieldSchemaStandardScheme.read(FieldSchema.java:483)
              at org.apache.hadoop.hive.metastore.api.FieldSchema$FieldSchemaStandardScheme.read(FieldSchema.java:476)
              at org.apache.hadoop.hive.metastore.api.FieldSchema.read(FieldSchema.java:410)
              at org.apache.hadoop.hive.metastore.api.StorageDescriptor$StorageDescriptorStandardScheme.read(StorageDescriptor.java:1309)
              at org.apache.hadoop.hive.metastore.api.StorageDescriptor$StorageDescriptorStandardScheme.read(StorageDescriptor.java:1288)
              at org.apache.hadoop.hive.metastore.api.StorageDescriptor.read(StorageDescriptor.java:1150)
              at org.apache.hadoop.hive.metastore.api.Table$TableStandardScheme.read(Table.java:1393)
              at org.apache.hadoop.hive.metastore.api.Table$TableStandardScheme.read(Table.java:1330)
              at org.apache.hadoop.hive.metastore.api.Table.read(Table.java:1186)
              at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$get_table_result$get_table_resultStandardScheme.read(ThriftHiveMetastore.java:27204)
              at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$get_table_result$get_table_resultStandardScheme.read(ThriftHiveMetastore.java:27189)
              at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$get_table_result.read(ThriftHiveMetastore.java:27120)
              at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:78)
              at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.recv_get_table(ThriftHiveMetastore.java:940)
              at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.get_table(ThriftHiveMetastore.java:926)
              at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.getTable(HiveMetaStoreClient.java:898)
             at org.apache.hcatalog.common.HCatUtil.getTable(HCatUtil.java:194)
      

        Activity

        Hide
        hudson Hudson added a comment -

        SUCCESS: Integrated in Tajo-block_iteration-branch-build #15 (See https://builds.apache.org/job/Tajo-block_iteration-branch-build/15/)
        TAJO-995: HiveMetaStoreClient wrapper should retry the connection (jinossy: rev 7c98709f0fcb06dfb675acae3d6489a6126f55b5)

        • tajo-catalog/tajo-catalog-drivers/tajo-hcatalog/src/main/java/org/apache/tajo/catalog/store/HCatalogUtil.java
        • tajo-catalog/tajo-catalog-drivers/tajo-hcatalog/src/main/java/org/apache/tajo/catalog/store/HCatalogStore.java
        • tajo-catalog/tajo-catalog-drivers/tajo-hcatalog/src/main/java/org/apache/tajo/catalog/store/HCatalogStoreClientPool.java
        Show
        hudson Hudson added a comment - SUCCESS: Integrated in Tajo-block_iteration-branch-build #15 (See https://builds.apache.org/job/Tajo-block_iteration-branch-build/15/ ) TAJO-995 : HiveMetaStoreClient wrapper should retry the connection (jinossy: rev 7c98709f0fcb06dfb675acae3d6489a6126f55b5) tajo-catalog/tajo-catalog-drivers/tajo-hcatalog/src/main/java/org/apache/tajo/catalog/store/HCatalogUtil.java tajo-catalog/tajo-catalog-drivers/tajo-hcatalog/src/main/java/org/apache/tajo/catalog/store/HCatalogStore.java tajo-catalog/tajo-catalog-drivers/tajo-hcatalog/src/main/java/org/apache/tajo/catalog/store/HCatalogStoreClientPool.java
        Hide
        hudson Hudson added a comment -

        SUCCESS: Integrated in Tajo-master-CODEGEN-build #32 (See https://builds.apache.org/job/Tajo-master-CODEGEN-build/32/)
        TAJO-995: HiveMetaStoreClient wrapper should retry the connection (jinossy: rev 7c98709f0fcb06dfb675acae3d6489a6126f55b5)

        • tajo-catalog/tajo-catalog-drivers/tajo-hcatalog/src/main/java/org/apache/tajo/catalog/store/HCatalogUtil.java
        • tajo-catalog/tajo-catalog-drivers/tajo-hcatalog/src/main/java/org/apache/tajo/catalog/store/HCatalogStoreClientPool.java
        • tajo-catalog/tajo-catalog-drivers/tajo-hcatalog/src/main/java/org/apache/tajo/catalog/store/HCatalogStore.java
        Show
        hudson Hudson added a comment - SUCCESS: Integrated in Tajo-master-CODEGEN-build #32 (See https://builds.apache.org/job/Tajo-master-CODEGEN-build/32/ ) TAJO-995 : HiveMetaStoreClient wrapper should retry the connection (jinossy: rev 7c98709f0fcb06dfb675acae3d6489a6126f55b5) tajo-catalog/tajo-catalog-drivers/tajo-hcatalog/src/main/java/org/apache/tajo/catalog/store/HCatalogUtil.java tajo-catalog/tajo-catalog-drivers/tajo-hcatalog/src/main/java/org/apache/tajo/catalog/store/HCatalogStoreClientPool.java tajo-catalog/tajo-catalog-drivers/tajo-hcatalog/src/main/java/org/apache/tajo/catalog/store/HCatalogStore.java
        Hide
        hudson Hudson added a comment -

        SUCCESS: Integrated in Tajo-master-build #390 (See https://builds.apache.org/job/Tajo-master-build/390/)
        TAJO-995: HiveMetaStoreClient wrapper should retry the connection (jinossy: rev 7c98709f0fcb06dfb675acae3d6489a6126f55b5)

        • tajo-catalog/tajo-catalog-drivers/tajo-hcatalog/src/main/java/org/apache/tajo/catalog/store/HCatalogStoreClientPool.java
        • tajo-catalog/tajo-catalog-drivers/tajo-hcatalog/src/main/java/org/apache/tajo/catalog/store/HCatalogUtil.java
        • tajo-catalog/tajo-catalog-drivers/tajo-hcatalog/src/main/java/org/apache/tajo/catalog/store/HCatalogStore.java
        Show
        hudson Hudson added a comment - SUCCESS: Integrated in Tajo-master-build #390 (See https://builds.apache.org/job/Tajo-master-build/390/ ) TAJO-995 : HiveMetaStoreClient wrapper should retry the connection (jinossy: rev 7c98709f0fcb06dfb675acae3d6489a6126f55b5) tajo-catalog/tajo-catalog-drivers/tajo-hcatalog/src/main/java/org/apache/tajo/catalog/store/HCatalogStoreClientPool.java tajo-catalog/tajo-catalog-drivers/tajo-hcatalog/src/main/java/org/apache/tajo/catalog/store/HCatalogUtil.java tajo-catalog/tajo-catalog-drivers/tajo-hcatalog/src/main/java/org/apache/tajo/catalog/store/HCatalogStore.java
        Hide
        hudson Hudson added a comment -

        SUCCESS: Integrated in Tajo-master-build #332 (See https://builds.apache.org/job/Tajo-master-build/332/)
        TAJO-995: HiveMetaStoreClient wrapper should retry the connection. (jinho) (jinossy: rev fbf7166897b3b25d8319198216dc4ca529e677ce)

        • tajo-catalog/tajo-catalog-drivers/tajo-hcatalog/src/main/java/org/apache/tajo/catalog/store/HCatalogStore.java
        • CHANGES
        • tajo-catalog/tajo-catalog-drivers/tajo-hcatalog/src/main/java/org/apache/tajo/catalog/store/HCatalogStoreClientPool.java
        • tajo-catalog/tajo-catalog-drivers/tajo-hcatalog/src/main/java/org/apache/tajo/catalog/store/HCatalogUtil.java
        Show
        hudson Hudson added a comment - SUCCESS: Integrated in Tajo-master-build #332 (See https://builds.apache.org/job/Tajo-master-build/332/ ) TAJO-995 : HiveMetaStoreClient wrapper should retry the connection. (jinho) (jinossy: rev fbf7166897b3b25d8319198216dc4ca529e677ce) tajo-catalog/tajo-catalog-drivers/tajo-hcatalog/src/main/java/org/apache/tajo/catalog/store/HCatalogStore.java CHANGES tajo-catalog/tajo-catalog-drivers/tajo-hcatalog/src/main/java/org/apache/tajo/catalog/store/HCatalogStoreClientPool.java tajo-catalog/tajo-catalog-drivers/tajo-hcatalog/src/main/java/org/apache/tajo/catalog/store/HCatalogUtil.java
        Hide
        githubbot ASF GitHub Bot added a comment -

        Github user jinossy commented on the pull request:

        https://github.com/apache/tajo/pull/108#issuecomment-51745394

        committed it
        Thank you for the review!!

        Show
        githubbot ASF GitHub Bot added a comment - Github user jinossy commented on the pull request: https://github.com/apache/tajo/pull/108#issuecomment-51745394 committed it Thank you for the review!!
        Hide
        githubbot ASF GitHub Bot added a comment -

        Github user asfgit closed the pull request at:

        https://github.com/apache/tajo/pull/108

        Show
        githubbot ASF GitHub Bot added a comment - Github user asfgit closed the pull request at: https://github.com/apache/tajo/pull/108
        Hide
        githubbot ASF GitHub Bot added a comment -

        Github user blrunner commented on the pull request:

        https://github.com/apache/tajo/pull/108#issuecomment-51554198

        Hi @jinossy

        +1

        Thanks your contribution, it's really necessary patch to tajo.
        It looks good overall and 'mvn clean install -Phcatalog-0.12.0 -Dtajo.catalog.store.class=org.apache.tajo.catalog.store.HCatalogStore' finished successfully.

        Push it.

        Show
        githubbot ASF GitHub Bot added a comment - Github user blrunner commented on the pull request: https://github.com/apache/tajo/pull/108#issuecomment-51554198 Hi @jinossy +1 Thanks your contribution, it's really necessary patch to tajo. It looks good overall and 'mvn clean install -Phcatalog-0.12.0 -Dtajo.catalog.store.class=org.apache.tajo.catalog.store.HCatalogStore' finished successfully. Push it.
        Hide
        githubbot ASF GitHub Bot added a comment -

        GitHub user jinossy opened a pull request:

        https://github.com/apache/tajo/pull/108

        TAJO-995: HiveMetaStoreClient wrapper should retry the connection

        You can merge this pull request into a Git repository by running:

        $ git pull https://github.com/jinossy/tajo TAJO-995

        Alternatively you can review and apply these changes as the patch at:

        https://github.com/apache/tajo/pull/108.patch

        To close this pull request, make a commit to your master/trunk branch
        with (at least) the following in the commit message:

        This closes #108


        commit 7c98709f0fcb06dfb675acae3d6489a6126f55b5
        Author: jinossy <jinossy@gmail.com>
        Date: 2014-08-06T08:43:35Z

        TAJO-995: HiveMetaStoreClient wrapper should retry the connection


        Show
        githubbot ASF GitHub Bot added a comment - GitHub user jinossy opened a pull request: https://github.com/apache/tajo/pull/108 TAJO-995 : HiveMetaStoreClient wrapper should retry the connection You can merge this pull request into a Git repository by running: $ git pull https://github.com/jinossy/tajo TAJO-995 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/tajo/pull/108.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #108 commit 7c98709f0fcb06dfb675acae3d6489a6126f55b5 Author: jinossy <jinossy@gmail.com> Date: 2014-08-06T08:43:35Z TAJO-995 : HiveMetaStoreClient wrapper should retry the connection

          People

          • Assignee:
            jhkim Jinho Kim
            Reporter:
            jhkim Jinho Kim
          • Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development