Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-18021

Add more info in timed out RetriesExhaustedException for read replica client get processing,

Details

    • Improvement
    • Status: Resolved
    • Minor
    • Resolution: Fixed
    • 2.0.0
    • 1.4.0, 2.0.0
    • Client
    • Reviewed

    Description

      Right now, when the client does not receive results from replica servers within configured timeout period, the client does not print out info which helps to understand/identify the cause. Please see
      https://github.com/apache/hbase/blob/master/hbase-client/src/main/java/org/apache/hadoop/hbase/client/RpcRetryingCallerWithReadReplicas.java#L212

      More info needs to be filled in the exception so it helps to pinpoint the root cause quickly.

      Attachments

        1. HBASE-18021-master-001.patch
          1 kB
          Hua Xiang
        2. HBASE-18021-master-002.patch
          1 kB
          Hua Xiang
        3. HBASE-18021-master-003.patch
          1 kB
          Hua Xiang

        Issue Links

          Activity

            zyork Zach York added a comment -

            Minor: While you are fixing this, please fix the capitalization.

            throw new RetriesExhaustedException("timed out after "

            throw new RetriesExhaustedException("Timed out after "

            Otherwise LGTM!

            zyork Zach York added a comment - Minor: While you are fixing this, please fix the capitalization. throw new RetriesExhaustedException("timed out after " throw new RetriesExhaustedException("Timed out after " Otherwise LGTM!
            huaxiang Hua Xiang added a comment -

            Thanks zyork, will upload a new patch with fixed capitalization.

            huaxiang Hua Xiang added a comment - Thanks zyork , will upload a new patch with fixed capitalization.
            huaxiang Hua Xiang added a comment -

            V2 addressed Zach's comments.

            huaxiang Hua Xiang added a comment - V2 addressed Zach's comments.
            zyork Zach York added a comment -

            +1 LGTM

            zyork Zach York added a comment - +1 LGTM
            hadoopqa Hadoop QA added a comment -
            -1 overall



            Vote Subsystem Runtime Comment
            0 reexec 0m 31s Docker mode activated.
            +1 hbaseanti 0m 0s Patch does not have any anti-patterns.
            +1 @author 0m 0s The patch does not contain any @author tags.
            -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
            +1 mvninstall 6m 6s master passed
            +1 compile 0m 31s master passed
            +1 checkstyle 0m 43s master passed
            +1 mvneclipse 0m 16s master passed
            +1 findbugs 1m 48s master passed
            +1 javadoc 0m 30s master passed
            +1 mvninstall 0m 33s the patch passed
            +1 compile 0m 29s the patch passed
            +1 javac 0m 29s the patch passed
            +1 checkstyle 0m 43s the patch passed
            +1 mvneclipse 0m 15s the patch passed
            -1 whitespace 0m 0s The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix.
            +1 hadoopcheck 55m 49s Patch does not cause any errors with Hadoop 2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha2.
            +1 findbugs 2m 24s the patch passed
            +1 javadoc 0m 35s the patch passed
            +1 unit 3m 46s hbase-client in the patch passed.
            +1 asflicense 0m 11s The patch does not generate ASF License warnings.
            75m 45s



            Subsystem Report/Notes
            Docker Client=17.03.0-ce Server=17.03.0-ce Image:yetus/hbase:757bf37
            JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12867199/HBASE-18021-master-001.patch
            JIRA Issue HBASE-18021
            Optional Tests asflicense javac javadoc unit findbugs hadoopcheck hbaseanti checkstyle compile
            uname Linux 4ac427450f00 4.8.3-std-1 #1 SMP Fri Oct 21 11:15:43 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
            Build tool maven
            Personality /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
            git revision master / b67f6fe
            Default Java 1.8.0_131
            findbugs v3.0.0
            whitespace https://builds.apache.org/job/PreCommit-HBASE-Build/6742/artifact/patchprocess/whitespace-eol.txt
            Test Results https://builds.apache.org/job/PreCommit-HBASE-Build/6742/testReport/
            modules C: hbase-client U: hbase-client
            Console output https://builds.apache.org/job/PreCommit-HBASE-Build/6742/console
            Powered by Apache Yetus 0.3.0 http://yetus.apache.org

            This message was automatically generated.

            hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 31s Docker mode activated. +1 hbaseanti 0m 0s Patch does not have any anti-patterns. +1 @author 0m 0s The patch does not contain any @author tags. -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 mvninstall 6m 6s master passed +1 compile 0m 31s master passed +1 checkstyle 0m 43s master passed +1 mvneclipse 0m 16s master passed +1 findbugs 1m 48s master passed +1 javadoc 0m 30s master passed +1 mvninstall 0m 33s the patch passed +1 compile 0m 29s the patch passed +1 javac 0m 29s the patch passed +1 checkstyle 0m 43s the patch passed +1 mvneclipse 0m 15s the patch passed -1 whitespace 0m 0s The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix. +1 hadoopcheck 55m 49s Patch does not cause any errors with Hadoop 2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha2. +1 findbugs 2m 24s the patch passed +1 javadoc 0m 35s the patch passed +1 unit 3m 46s hbase-client in the patch passed. +1 asflicense 0m 11s The patch does not generate ASF License warnings. 75m 45s Subsystem Report/Notes Docker Client=17.03.0-ce Server=17.03.0-ce Image:yetus/hbase:757bf37 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12867199/HBASE-18021-master-001.patch JIRA Issue HBASE-18021 Optional Tests asflicense javac javadoc unit findbugs hadoopcheck hbaseanti checkstyle compile uname Linux 4ac427450f00 4.8.3-std-1 #1 SMP Fri Oct 21 11:15:43 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh git revision master / b67f6fe Default Java 1.8.0_131 findbugs v3.0.0 whitespace https://builds.apache.org/job/PreCommit-HBASE-Build/6742/artifact/patchprocess/whitespace-eol.txt Test Results https://builds.apache.org/job/PreCommit-HBASE-Build/6742/testReport/ modules C: hbase-client U: hbase-client Console output https://builds.apache.org/job/PreCommit-HBASE-Build/6742/console Powered by Apache Yetus 0.3.0 http://yetus.apache.org This message was automatically generated.
            hadoopqa Hadoop QA added a comment -
            -1 overall



            Vote Subsystem Runtime Comment
            0 reexec 0m 30s Docker mode activated.
            +1 hbaseanti 0m 0s Patch does not have any anti-patterns.
            +1 @author 0m 0s The patch does not contain any @author tags.
            -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
            +1 mvninstall 6m 54s master passed
            +1 compile 0m 33s master passed
            +1 checkstyle 0m 47s master passed
            +1 mvneclipse 0m 20s master passed
            +1 findbugs 2m 7s master passed
            +1 javadoc 0m 36s master passed
            +1 mvninstall 0m 37s the patch passed
            +1 compile 0m 33s the patch passed
            +1 javac 0m 33s the patch passed
            +1 checkstyle 0m 47s the patch passed
            +1 mvneclipse 0m 16s the patch passed
            -1 whitespace 0m 0s The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix.
            +1 hadoopcheck 54m 44s Patch does not cause any errors with Hadoop 2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha2.
            +1 findbugs 2m 10s the patch passed
            +1 javadoc 0m 29s the patch passed
            +1 unit 3m 17s hbase-client in the patch passed.
            +1 asflicense 0m 11s The patch does not generate ASF License warnings.
            75m 24s



            Subsystem Report/Notes
            Docker Client=17.03.0-ce Server=17.03.0-ce Image:yetus/hbase:757bf37
            JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12867212/HBASE-18021-master-002.patch
            JIRA Issue HBASE-18021
            Optional Tests asflicense javac javadoc unit findbugs hadoopcheck hbaseanti checkstyle compile
            uname Linux 3ec6213de168 4.8.3-std-1 #1 SMP Fri Oct 21 11:15:43 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
            Build tool maven
            Personality /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build@2/component/dev-support/hbase-personality.sh
            git revision master / b67f6fe
            Default Java 1.8.0_131
            findbugs v3.0.0
            whitespace https://builds.apache.org/job/PreCommit-HBASE-Build/6743/artifact/patchprocess/whitespace-eol.txt
            Test Results https://builds.apache.org/job/PreCommit-HBASE-Build/6743/testReport/
            modules C: hbase-client U: hbase-client
            Console output https://builds.apache.org/job/PreCommit-HBASE-Build/6743/console
            Powered by Apache Yetus 0.3.0 http://yetus.apache.org

            This message was automatically generated.

            hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 30s Docker mode activated. +1 hbaseanti 0m 0s Patch does not have any anti-patterns. +1 @author 0m 0s The patch does not contain any @author tags. -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 mvninstall 6m 54s master passed +1 compile 0m 33s master passed +1 checkstyle 0m 47s master passed +1 mvneclipse 0m 20s master passed +1 findbugs 2m 7s master passed +1 javadoc 0m 36s master passed +1 mvninstall 0m 37s the patch passed +1 compile 0m 33s the patch passed +1 javac 0m 33s the patch passed +1 checkstyle 0m 47s the patch passed +1 mvneclipse 0m 16s the patch passed -1 whitespace 0m 0s The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix. +1 hadoopcheck 54m 44s Patch does not cause any errors with Hadoop 2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha2. +1 findbugs 2m 10s the patch passed +1 javadoc 0m 29s the patch passed +1 unit 3m 17s hbase-client in the patch passed. +1 asflicense 0m 11s The patch does not generate ASF License warnings. 75m 24s Subsystem Report/Notes Docker Client=17.03.0-ce Server=17.03.0-ce Image:yetus/hbase:757bf37 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12867212/HBASE-18021-master-002.patch JIRA Issue HBASE-18021 Optional Tests asflicense javac javadoc unit findbugs hadoopcheck hbaseanti checkstyle compile uname Linux 3ec6213de168 4.8.3-std-1 #1 SMP Fri Oct 21 11:15:43 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build@2/component/dev-support/hbase-personality.sh git revision master / b67f6fe Default Java 1.8.0_131 findbugs v3.0.0 whitespace https://builds.apache.org/job/PreCommit-HBASE-Build/6743/artifact/patchprocess/whitespace-eol.txt Test Results https://builds.apache.org/job/PreCommit-HBASE-Build/6743/testReport/ modules C: hbase-client U: hbase-client Console output https://builds.apache.org/job/PreCommit-HBASE-Build/6743/console Powered by Apache Yetus 0.3.0 http://yetus.apache.org This message was automatically generated.
            huaxiang Hua Xiang added a comment -

            v3 addressing the whitespace at the end of the line warning.

            huaxiang Hua Xiang added a comment - v3 addressing the whitespace at the end of the line warning.
            hadoopqa Hadoop QA added a comment -
            -1 overall



            Vote Subsystem Runtime Comment
            0 reexec 14m 0s Docker mode activated.
            +1 hbaseanti 0m 0s Patch does not have any anti-patterns.
            +1 @author 0m 0s The patch does not contain any @author tags.
            -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
            +1 mvninstall 3m 54s master passed
            +1 compile 0m 17s master passed
            +1 checkstyle 0m 24s master passed
            +1 mvneclipse 0m 12s master passed
            +1 findbugs 1m 5s master passed
            +1 javadoc 0m 22s master passed
            +1 mvninstall 0m 24s the patch passed
            +1 compile 0m 18s the patch passed
            +1 javac 0m 18s the patch passed
            +1 checkstyle 0m 27s the patch passed
            +1 mvneclipse 0m 10s the patch passed
            +1 whitespace 0m 0s The patch has no whitespace issues.
            +1 hadoopcheck 29m 21s Patch does not cause any errors with Hadoop 2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha2.
            +1 findbugs 1m 0s the patch passed
            +1 javadoc 0m 17s the patch passed
            +1 unit 2m 40s hbase-client in the patch passed.
            +1 asflicense 0m 11s The patch does not generate ASF License warnings.
            55m 29s



            Subsystem Report/Notes
            Docker Client=1.10.1 Server=1.10.1 Image:yetus/hbase:757bf37
            JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12867366/HBASE-18021-master-003.patch
            JIRA Issue HBASE-18021
            Optional Tests asflicense javac javadoc unit findbugs hadoopcheck hbaseanti checkstyle compile
            uname Linux 0295fb61a410 3.19.0-25-generic #26~14.04.1-Ubuntu SMP Fri Jul 24 21:16:20 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux
            Build tool maven
            Personality /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
            git revision master / c5cc81d
            Default Java 1.8.0_131
            findbugs v3.0.0
            Test Results https://builds.apache.org/job/PreCommit-HBASE-Build/6752/testReport/
            modules C: hbase-client U: hbase-client
            Console output https://builds.apache.org/job/PreCommit-HBASE-Build/6752/console
            Powered by Apache Yetus 0.3.0 http://yetus.apache.org

            This message was automatically generated.

            hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 14m 0s Docker mode activated. +1 hbaseanti 0m 0s Patch does not have any anti-patterns. +1 @author 0m 0s The patch does not contain any @author tags. -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 mvninstall 3m 54s master passed +1 compile 0m 17s master passed +1 checkstyle 0m 24s master passed +1 mvneclipse 0m 12s master passed +1 findbugs 1m 5s master passed +1 javadoc 0m 22s master passed +1 mvninstall 0m 24s the patch passed +1 compile 0m 18s the patch passed +1 javac 0m 18s the patch passed +1 checkstyle 0m 27s the patch passed +1 mvneclipse 0m 10s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 hadoopcheck 29m 21s Patch does not cause any errors with Hadoop 2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha2. +1 findbugs 1m 0s the patch passed +1 javadoc 0m 17s the patch passed +1 unit 2m 40s hbase-client in the patch passed. +1 asflicense 0m 11s The patch does not generate ASF License warnings. 55m 29s Subsystem Report/Notes Docker Client=1.10.1 Server=1.10.1 Image:yetus/hbase:757bf37 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12867366/HBASE-18021-master-003.patch JIRA Issue HBASE-18021 Optional Tests asflicense javac javadoc unit findbugs hadoopcheck hbaseanti checkstyle compile uname Linux 0295fb61a410 3.19.0-25-generic #26~14.04.1-Ubuntu SMP Fri Jul 24 21:16:20 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh git revision master / c5cc81d Default Java 1.8.0_131 findbugs v3.0.0 Test Results https://builds.apache.org/job/PreCommit-HBASE-Build/6752/testReport/ modules C: hbase-client U: hbase-client Console output https://builds.apache.org/job/PreCommit-HBASE-Build/6752/console Powered by Apache Yetus 0.3.0 http://yetus.apache.org This message was automatically generated.
            huaxiang Hua Xiang added a comment -

            enis, can you help to review the patch? The info added is helpful to debug timeout issues with read replica, we just applied it to one case we are working on and it helped. zyork helped to review the patch but I guess it needs a committer's review to commit. Sorry if I am wrong zyork, thanks!

            huaxiang Hua Xiang added a comment - enis , can you help to review the patch? The info added is helpful to debug timeout issues with read replica, we just applied it to one case we are working on and it helped. zyork helped to review the patch but I guess it needs a committer's review to commit. Sorry if I am wrong zyork , thanks!
            stack Michael Stack added a comment -

            Pushed to branch-1 and master. Thanks for the patch huaxiang

            stack Michael Stack added a comment - Pushed to branch-1 and master. Thanks for the patch huaxiang
            huaxiang Hua Xiang added a comment -

            Thanks Stack!

            huaxiang Hua Xiang added a comment - Thanks Stack!
            hudson Hudson added a comment -

            SUCCESS: Integrated in Jenkins build HBase-Trunk_matrix #2994 (See https://builds.apache.org/job/HBase-Trunk_matrix/2994/)
            HBASE-18021 Add more info in timed out RetriesExhaustedException for (stack: rev d64acfd3045027e219438ff6c45fd72f0ea5140f)

            • (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/client/RpcRetryingCallerWithReadReplicas.java
            hudson Hudson added a comment - SUCCESS: Integrated in Jenkins build HBase-Trunk_matrix #2994 (See https://builds.apache.org/job/HBase-Trunk_matrix/2994/ ) HBASE-18021 Add more info in timed out RetriesExhaustedException for (stack: rev d64acfd3045027e219438ff6c45fd72f0ea5140f) (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/client/RpcRetryingCallerWithReadReplicas.java
            hudson Hudson added a comment -

            SUCCESS: Integrated in Jenkins build HBase-1.4 #732 (See https://builds.apache.org/job/HBase-1.4/732/)
            HBASE-18021 Add more info in timed out RetriesExhaustedException for (stack: rev 51cb53776de413c3f2b4b481a0798f428680ac6b)

            • (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/client/RpcRetryingCallerWithReadReplicas.java
            hudson Hudson added a comment - SUCCESS: Integrated in Jenkins build HBase-1.4 #732 (See https://builds.apache.org/job/HBase-1.4/732/ ) HBASE-18021 Add more info in timed out RetriesExhaustedException for (stack: rev 51cb53776de413c3f2b4b481a0798f428680ac6b) (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/client/RpcRetryingCallerWithReadReplicas.java
            hudson Hudson added a comment -

            FAILURE: Integrated in Jenkins build HBase-HBASE-14614 #244 (See https://builds.apache.org/job/HBase-HBASE-14614/244/)
            HBASE-18021 Add more info in timed out RetriesExhaustedException for (stack: rev d64acfd3045027e219438ff6c45fd72f0ea5140f)

            • (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/client/RpcRetryingCallerWithReadReplicas.java
            hudson Hudson added a comment - FAILURE: Integrated in Jenkins build HBase- HBASE-14614 #244 (See https://builds.apache.org/job/HBase-HBASE-14614/244/ ) HBASE-18021 Add more info in timed out RetriesExhaustedException for (stack: rev d64acfd3045027e219438ff6c45fd72f0ea5140f) (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/client/RpcRetryingCallerWithReadReplicas.java

            People

              huaxiang Hua Xiang
              huaxiang Hua Xiang
              Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: