Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-8821

Explain message "Operation category X is not supported in state standby"

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.8.0, 3.0.0-alpha1
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      There is one message specifically that causes many users to question the health of their HDFS cluster, namely "Operation category READ/WRITE is not supported in state standby".

      HDFS-3447 is an attempt to lower the logging severity for StandbyException related messages but it is not resolved yet. So this jira is an attempt to explain this particular message so it appears less scary.

      The text is question 3.17 in the Hadoop Wiki FAQ
      ref: https://wiki.apache.org/hadoop/FAQ#What_does_the_message_.22Operation_category_READ.2FWRITE_is_not_supported_in_state_standby.22_mean.3F

      1. HDFS-8821-1.patch
        0.9 kB
        Gautam Gopalakrishnan
      2. HDFS-8821-2.patch
        0.8 kB
        Gautam Gopalakrishnan

        Activity

        Hide
        ggop Gautam Gopalakrishnan added a comment -

        Trivial patch attached

        Show
        ggop Gautam Gopalakrishnan added a comment - Trivial patch attached
        Hide
        ggop Gautam Gopalakrishnan added a comment -

        New patch with the s.apache.org link to the question itself.

        Show
        ggop Gautam Gopalakrishnan added a comment - New patch with the s.apache.org link to the question itself.
        Hide
        hadoopqa Hadoop QA added a comment -



        -1 overall



        Vote Subsystem Runtime Comment
        -1 pre-patch 15m 12s Findbugs (version ) appears to be broken on trunk.
        +1 @author 0m 0s The patch does not contain any @author tags.
        -1 tests included 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
        +1 javac 7m 33s There were no new javac warning messages.
        +1 javadoc 9m 38s There were no new javadoc warning messages.
        +1 release audit 0m 22s The applied patch does not increase the total number of release audit warnings.
        +1 checkstyle 0m 36s There were no new checkstyle issues.
        +1 whitespace 0m 0s The patch has no lines that end in whitespace.
        +1 install 1m 31s mvn install still works.
        +1 eclipse:eclipse 0m 33s The patch built with eclipse:eclipse.
        +1 findbugs 2m 30s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
        +1 native 3m 3s Pre-build of native portion
        -1 hdfs tests 161m 14s Tests failed in hadoop-hdfs.
            202m 16s  



        Reason Tests
        Failed unit tests hadoop.hdfs.TestDistributedFileSystem



        Subsystem Report/Notes
        Patch URL http://issues.apache.org/jira/secure/attachment/12747168/HDFS-8821-1.patch
        Optional Tests javadoc javac unit findbugs checkstyle
        git revision trunk / adcf5dd
        hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/11837/artifact/patchprocess/testrun_hadoop-hdfs.txt
        Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/11837/testReport/
        Java 1.7.0_55
        uname Linux asf909.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/11837/console

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment -1 pre-patch 15m 12s Findbugs (version ) appears to be broken on trunk. +1 @author 0m 0s The patch does not contain any @author tags. -1 tests included 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac 7m 33s There were no new javac warning messages. +1 javadoc 9m 38s There were no new javadoc warning messages. +1 release audit 0m 22s The applied patch does not increase the total number of release audit warnings. +1 checkstyle 0m 36s There were no new checkstyle issues. +1 whitespace 0m 0s The patch has no lines that end in whitespace. +1 install 1m 31s mvn install still works. +1 eclipse:eclipse 0m 33s The patch built with eclipse:eclipse. +1 findbugs 2m 30s The patch does not introduce any new Findbugs (version 3.0.0) warnings. +1 native 3m 3s Pre-build of native portion -1 hdfs tests 161m 14s Tests failed in hadoop-hdfs.     202m 16s   Reason Tests Failed unit tests hadoop.hdfs.TestDistributedFileSystem Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12747168/HDFS-8821-1.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / adcf5dd hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/11837/artifact/patchprocess/testrun_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/11837/testReport/ Java 1.7.0_55 uname Linux asf909.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-HDFS-Build/11837/console This message was automatically generated.
        Hide
        hadoopqa Hadoop QA added a comment -



        -1 overall



        Vote Subsystem Runtime Comment
        0 pre-patch 17m 30s Pre-patch trunk compilation is healthy.
        +1 @author 0m 0s The patch does not contain any @author tags.
        -1 tests included 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
        +1 javac 7m 41s There were no new javac warning messages.
        +1 javadoc 9m 41s There were no new javadoc warning messages.
        +1 release audit 0m 23s The applied patch does not increase the total number of release audit warnings.
        +1 checkstyle 1m 26s There were no new checkstyle issues.
        +1 whitespace 0m 0s The patch has no lines that end in whitespace.
        +1 install 1m 23s mvn install still works.
        +1 eclipse:eclipse 0m 33s The patch built with eclipse:eclipse.
        +1 findbugs 2m 31s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
        +1 native 3m 3s Pre-build of native portion
        -1 hdfs tests 160m 35s Tests failed in hadoop-hdfs.
            204m 50s  



        Reason Tests
        Failed unit tests hadoop.hdfs.TestDistributedFileSystem
          hadoop.hdfs.server.namenode.ha.TestStandbyIsHot



        Subsystem Report/Notes
        Patch URL http://issues.apache.org/jira/secure/attachment/12747174/HDFS-8821-2.patch
        Optional Tests javadoc javac unit findbugs checkstyle
        git revision trunk / adcf5dd
        hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/11839/artifact/patchprocess/testrun_hadoop-hdfs.txt
        Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/11839/testReport/
        Java 1.7.0_55
        uname Linux asf900.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/11839/console

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 17m 30s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. -1 tests included 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac 7m 41s There were no new javac warning messages. +1 javadoc 9m 41s There were no new javadoc warning messages. +1 release audit 0m 23s The applied patch does not increase the total number of release audit warnings. +1 checkstyle 1m 26s There were no new checkstyle issues. +1 whitespace 0m 0s The patch has no lines that end in whitespace. +1 install 1m 23s mvn install still works. +1 eclipse:eclipse 0m 33s The patch built with eclipse:eclipse. +1 findbugs 2m 31s The patch does not introduce any new Findbugs (version 3.0.0) warnings. +1 native 3m 3s Pre-build of native portion -1 hdfs tests 160m 35s Tests failed in hadoop-hdfs.     204m 50s   Reason Tests Failed unit tests hadoop.hdfs.TestDistributedFileSystem   hadoop.hdfs.server.namenode.ha.TestStandbyIsHot Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12747174/HDFS-8821-2.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / adcf5dd hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/11839/artifact/patchprocess/testrun_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/11839/testReport/ Java 1.7.0_55 uname Linux asf900.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-HDFS-Build/11839/console This message was automatically generated.
        Hide
        qwertymaniac Harsh J added a comment -

        +1 looks good to me. This should help avoid the continual confusion people new to HDFS HA appear to have (from experience) about whether that error is to be taken seriously or not. Thank you for writing up the wiki entry too!

        hadoop.hdfs.server.namenode.ha.TestStandbyIsHot

        Test name appears relevant but the failing test does not (fails at counting proper under-replicated blocks value). I'll still test this again manually before committing (by Tuesday EOD).

        Show
        qwertymaniac Harsh J added a comment - +1 looks good to me. This should help avoid the continual confusion people new to HDFS HA appear to have (from experience) about whether that error is to be taken seriously or not. Thank you for writing up the wiki entry too! hadoop.hdfs.server.namenode.ha.TestStandbyIsHot Test name appears relevant but the failing test does not (fails at counting proper under-replicated blocks value). I'll still test this again manually before committing (by Tuesday EOD).
        Hide
        ggop Gautam Gopalakrishnan added a comment -

        Thanks Harsh, I didn't see any failures locally.

        $ mvn test -DTestStandbyIsHot
        ...
        Running org.apache.hadoop.hdfs.server.namenode.ha.TestStandbyIsHot
        Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 20.369 sec - in org.apache.hadoop.hdfs.server.namenode.ha.TestStandbyIsHot
        
        Show
        ggop Gautam Gopalakrishnan added a comment - Thanks Harsh, I didn't see any failures locally. $ mvn test -DTestStandbyIsHot ... Running org.apache.hadoop.hdfs.server.namenode.ha.TestStandbyIsHot Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 20.369 sec - in org.apache.hadoop.hdfs.server.namenode.ha.TestStandbyIsHot
        Hide
        qwertymaniac Harsh J added a comment -

        Thanks Gautam, test passed locally for me as well. Seems unrelated.

        I've pushed this into trunk and branch-2; thank you for the continued contributions!

        Show
        qwertymaniac Harsh J added a comment - Thanks Gautam, test passed locally for me as well. Seems unrelated. I've pushed this into trunk and branch-2; thank you for the continued contributions!
        Hide
        hudson Hudson added a comment -

        SUCCESS: Integrated in Hadoop-trunk-Commit #8247 (See https://builds.apache.org/job/Hadoop-trunk-Commit/8247/)
        HDFS-8821. Explain message "Operation category X is not supported in state standby". Contributed by Gautam Gopalakrishnan. (harsh: rev c5caa25b8f2953e2b7a9d2c9dcbdbf1fed95c10b)

        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/ha/StandbyState.java
        Show
        hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-trunk-Commit #8247 (See https://builds.apache.org/job/Hadoop-trunk-Commit/8247/ ) HDFS-8821 . Explain message "Operation category X is not supported in state standby". Contributed by Gautam Gopalakrishnan. (harsh: rev c5caa25b8f2953e2b7a9d2c9dcbdbf1fed95c10b) hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/ha/StandbyState.java
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #273 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/273/)
        HDFS-8821. Explain message "Operation category X is not supported in state standby". Contributed by Gautam Gopalakrishnan. (harsh: rev c5caa25b8f2953e2b7a9d2c9dcbdbf1fed95c10b)

        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/ha/StandbyState.java
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #273 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/273/ ) HDFS-8821 . Explain message "Operation category X is not supported in state standby". Contributed by Gautam Gopalakrishnan. (harsh: rev c5caa25b8f2953e2b7a9d2c9dcbdbf1fed95c10b) hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/ha/StandbyState.java
        Hide
        hudson Hudson added a comment -

        SUCCESS: Integrated in Hadoop-Yarn-trunk #1003 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/1003/)
        HDFS-8821. Explain message "Operation category X is not supported in state standby". Contributed by Gautam Gopalakrishnan. (harsh: rev c5caa25b8f2953e2b7a9d2c9dcbdbf1fed95c10b)

        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/ha/StandbyState.java
        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        Show
        hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-Yarn-trunk #1003 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/1003/ ) HDFS-8821 . Explain message "Operation category X is not supported in state standby". Contributed by Gautam Gopalakrishnan. (harsh: rev c5caa25b8f2953e2b7a9d2c9dcbdbf1fed95c10b) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/ha/StandbyState.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #262 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/262/)
        HDFS-8821. Explain message "Operation category X is not supported in state standby". Contributed by Gautam Gopalakrishnan. (harsh: rev c5caa25b8f2953e2b7a9d2c9dcbdbf1fed95c10b)

        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/ha/StandbyState.java
        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #262 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/262/ ) HDFS-8821 . Explain message "Operation category X is not supported in state standby". Contributed by Gautam Gopalakrishnan. (harsh: rev c5caa25b8f2953e2b7a9d2c9dcbdbf1fed95c10b) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/ha/StandbyState.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Mapreduce-trunk #2219 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2219/)
        HDFS-8821. Explain message "Operation category X is not supported in state standby". Contributed by Gautam Gopalakrishnan. (harsh: rev c5caa25b8f2953e2b7a9d2c9dcbdbf1fed95c10b)

        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/ha/StandbyState.java
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk #2219 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2219/ ) HDFS-8821 . Explain message "Operation category X is not supported in state standby". Contributed by Gautam Gopalakrishnan. (harsh: rev c5caa25b8f2953e2b7a9d2c9dcbdbf1fed95c10b) hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/ha/StandbyState.java
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Hdfs-trunk #2200 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2200/)
        HDFS-8821. Explain message "Operation category X is not supported in state standby". Contributed by Gautam Gopalakrishnan. (harsh: rev c5caa25b8f2953e2b7a9d2c9dcbdbf1fed95c10b)

        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/ha/StandbyState.java
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk #2200 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2200/ ) HDFS-8821 . Explain message "Operation category X is not supported in state standby". Contributed by Gautam Gopalakrishnan. (harsh: rev c5caa25b8f2953e2b7a9d2c9dcbdbf1fed95c10b) hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/ha/StandbyState.java
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #270 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/270/)
        HDFS-8821. Explain message "Operation category X is not supported in state standby". Contributed by Gautam Gopalakrishnan. (harsh: rev c5caa25b8f2953e2b7a9d2c9dcbdbf1fed95c10b)

        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/ha/StandbyState.java
        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #270 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/270/ ) HDFS-8821 . Explain message "Operation category X is not supported in state standby". Contributed by Gautam Gopalakrishnan. (harsh: rev c5caa25b8f2953e2b7a9d2c9dcbdbf1fed95c10b) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/ha/StandbyState.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        Hide
        hudson Hudson added a comment -

        SUCCESS: Integrated in Hadoop-trunk-Commit #10014 (See https://builds.apache.org/job/Hadoop-trunk-Commit/10014/)
        HDFS-8821. Stop tracking CHANGES.txt in the HDFS-1312 feature branch. (arp: rev 4b93ddae07ba4332f40f896542ee2c6d7bf899ed)

        • hadoop-hdfs-project/hadoop-hdfs/pom.xml
        • hadoop-hdfs-project/hadoop-hdfs/HDFS-1312_CHANGES.txt
        Show
        hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-trunk-Commit #10014 (See https://builds.apache.org/job/Hadoop-trunk-Commit/10014/ ) HDFS-8821 . Stop tracking CHANGES.txt in the HDFS-1312 feature branch. (arp: rev 4b93ddae07ba4332f40f896542ee2c6d7bf899ed) hadoop-hdfs-project/hadoop-hdfs/pom.xml hadoop-hdfs-project/hadoop-hdfs/ HDFS-1312 _CHANGES.txt

          People

          • Assignee:
            ggop Gautam Gopalakrishnan
            Reporter:
            ggop Gautam Gopalakrishnan
          • Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development