Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: 2.2.0
    • Fix Version/s: 3.0.0-alpha1
    • Component/s: None
    • Labels:
    • Target Version/s:
    • Hadoop Flags:
      Incompatible change, Reviewed
    • Release Note:
      fsck does not print out dots for progress reporting by default. To print out dots, you should specify '-showprogress' option.

      Description

      this patch turns the dots during fsck off by default and provides an option to turn them back on if you have a fetish for millions and millions of dots on your terminal. i haven't done any benchmarks, but i suspect fsck is now 300% faster to boot.

      1. HDFS-2538.1.patch
        4 kB
        Mohammad Kamrul Islam
      2. HDFS-2538.2.patch
        4 kB
        Mohammad Kamrul Islam
      3. HDFS-2538.3.patch
        5 kB
        Mohammad Kamrul Islam
      4. HDFS-2538-branch-0.20-security-204.patch
        3 kB
        Allen Wittenauer
      5. HDFS-2538-branch-0.20-security-204.patch
        4 kB
        Allen Wittenauer
      6. HDFS-2538-branch-1.0.patch
        2 kB
        Allen Wittenauer
      7. HDFS-2538-branch-2.7.patch
        5 kB
        Íñigo Goiri

        Issue Links

          Activity

          Hide
          aw Allen Wittenauer added a comment -

          woops this included another patch. let's try that again.

          Show
          aw Allen Wittenauer added a comment - woops this included another patch. let's try that again.
          Hide
          tlipcon Todd Lipcon added a comment -

          Given we know how many files/blocks/whatevers we have in the namespace before we even start the operation, maybe we can just make the dot frequency relative to that? eg print a dot for every 1% or 5% so you just get a line or two, but still have some indication of progress?

          Show
          tlipcon Todd Lipcon added a comment - Given we know how many files/blocks/whatevers we have in the namespace before we even start the operation, maybe we can just make the dot frequency relative to that? eg print a dot for every 1% or 5% so you just get a line or two, but still have some indication of progress?
          Hide
          kihwal Kihwal Lee added a comment -

          I like the patch very much, but at the same time the part I like the most may need some changes.

          • Instead of -annoying, -verbose or -progress would be nice.
          • At the same time, can we change %100 to something bigger?
          Show
          kihwal Kihwal Lee added a comment - I like the patch very much, but at the same time the part I like the most may need some changes. Instead of -annoying , -verbose or -progress would be nice. At the same time, can we change %100 to something bigger?
          Hide
          aw Allen Wittenauer added a comment -

          Updated for 1.0 and fixes a slight output bug.

          Show
          aw Allen Wittenauer added a comment - Updated for 1.0 and fixes a slight output bug.
          Hide
          stevel@apache.org Steve Loughran added a comment -

          think annoying should be the default for compatibility, then you could make the parameter -quiet

          Show
          stevel@apache.org Steve Loughran added a comment - think annoying should be the default for compatibility, then you could make the parameter -quiet
          Hide
          adi2 Andy Isaacson added a comment -

          The option needs to be "verbose"/"quiet" or "progress"/"noprogress" rather than "annoying". (Although I find it annoying too.)

          I think, regrettably, that this counts as an incompatible change, if the default behavior is changed. I'd like to see progress turned off for trunk, with a -progress option to turn it back on; and progress on by default for stable branches, with a -noprogress option to turn it off.

          The branch-1 patch is missing the DFSck.java change to add "-annoying".

          Show
          adi2 Andy Isaacson added a comment - The option needs to be "verbose"/"quiet" or "progress"/"noprogress" rather than "annoying". (Although I find it annoying too.) I think, regrettably, that this counts as an incompatible change, if the default behavior is changed. I'd like to see progress turned off for trunk, with a -progress option to turn it back on; and progress on by default for stable branches, with a -noprogress option to turn it off. The branch-1 patch is missing the DFSck.java change to add "-annoying".
          Hide
          aw Allen Wittenauer added a comment -

          FWIW, I don't really plan on making a new version of this patch. I have had it checked into my tree for almost a year now. Given how easy it is to do merges now that Apache trees are available via git, I really have no inclination to spend the effort to get it committed. If someone else wants to spend the cycles, fine whatever.

          Show
          aw Allen Wittenauer added a comment - FWIW, I don't really plan on making a new version of this patch. I have had it checked into my tree for almost a year now. Given how easy it is to do merges now that Apache trees are available via git, I really have no inclination to spend the effort to get it committed. If someone else wants to spend the cycles, fine whatever.
          Hide
          kamrul Mohammad Kamrul Islam added a comment -

          This is based on original patch provided by Allen Wittenauer.
          This is for only Hadoop 2.3+.

          Show
          kamrul Mohammad Kamrul Islam added a comment - This is based on original patch provided by Allen Wittenauer . This is for only Hadoop 2.3+.
          Hide
          ajisakaa Akira Ajisaka added a comment -

          +1, LGTM.

          Show
          ajisakaa Akira Ajisaka added a comment - +1, LGTM.
          Hide
          hadoopqa Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12630407/HDFS-2538.1.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          -1 tests included. The patch doesn't appear to include any new or modified tests.
          Please justify why no new tests are needed for this patch.
          Also please list what manual steps were performed to verify this patch.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. There were no new javadoc warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 core tests. The patch failed these unit tests in hadoop-hdfs-project/hadoop-hdfs:

          org.apache.hadoop.hdfs.server.balancer.TestBalancerWithNodeGroup

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/6211//testReport/
          Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/6211//console

          This message is automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12630407/HDFS-2538.1.patch against trunk revision . +1 @author . The patch does not contain any @author tags. -1 tests included . The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . There were no new javadoc warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. -1 core tests . The patch failed these unit tests in hadoop-hdfs-project/hadoop-hdfs: org.apache.hadoop.hdfs.server.balancer.TestBalancerWithNodeGroup +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/6211//testReport/ Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/6211//console This message is automatically generated.
          Hide
          ajisakaa Akira Ajisaka added a comment -

          The test failure is not related to the patch.

          Show
          ajisakaa Akira Ajisaka added a comment - The test failure is not related to the patch.
          Hide
          jghoman Jakob Homan added a comment -

          Line: 46: Need a dash in the option.

          Also, this does need to have the dots on by default for compatibility. Option should be -quiet or -noprogress to suppress them.

          Show
          jghoman Jakob Homan added a comment - Line: 46: Need a dash in the option. Also, this does need to have the dots on by default for compatibility. Option should be -quiet or -noprogress to suppress them.
          Hide
          aw Allen Wittenauer added a comment -

          Also, this does need to have the dots on by default for compatibility.

          I humbly and respectfully disagree. The key question is compatibility against what? We're talking about command line output that I suspect the vast majority of people are filtering out and has no stability label that I'm aware of. What could possibly break by removing the dots?

          Show
          aw Allen Wittenauer added a comment - Also, this does need to have the dots on by default for compatibility. I humbly and respectfully disagree. The key question is compatibility against what? We're talking about command line output that I suspect the vast majority of people are filtering out and has no stability label that I'm aware of. What could possibly break by removing the dots?
          Hide
          kamrul Mohammad Kamrul Islam added a comment -

          Addressed Jakob's comment.

          Show
          kamrul Mohammad Kamrul Islam added a comment - Addressed Jakob's comment.
          Hide
          hadoopqa Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12630797/HDFS-2538.2.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          -1 tests included. The patch doesn't appear to include any new or modified tests.
          Please justify why no new tests are needed for this patch.
          Also please list what manual steps were performed to verify this patch.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. There were no new javadoc warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 core tests. The patch failed these unit tests in hadoop-hdfs-project/hadoop-hdfs:

          org.apache.hadoop.hdfs.server.namenode.TestCacheDirectives

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/6220//testReport/
          Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/6220//console

          This message is automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12630797/HDFS-2538.2.patch against trunk revision . +1 @author . The patch does not contain any @author tags. -1 tests included . The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . There were no new javadoc warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. -1 core tests . The patch failed these unit tests in hadoop-hdfs-project/hadoop-hdfs: org.apache.hadoop.hdfs.server.namenode.TestCacheDirectives +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/6220//testReport/ Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/6220//console This message is automatically generated.
          Hide
          ajisakaa Akira Ajisaka added a comment -

          The test failure is not related to the latest patch.

          What could possibly break by removing the dots?

          Unfortunately, users may use the scripts to run fsck which assume that dots are output in the console.

          Mohammad Kamrul Islam, can you please update the document also? The document is in hadoop-common-project/hadoop-common/src/site/apt/CommandsManual.apt.vm. I'm +1 once it is addressed.

          Show
          ajisakaa Akira Ajisaka added a comment - The test failure is not related to the latest patch. What could possibly break by removing the dots? Unfortunately, users may use the scripts to run fsck which assume that dots are output in the console. Mohammad Kamrul Islam , can you please update the document also? The document is in hadoop-common-project/hadoop-common/src/site/apt/CommandsManual.apt.vm. I'm +1 once it is addressed.
          Hide
          jghoman Jakob Homan added a comment -

          I'm willing to do the silent default and mark as incompatible on trunk, but not back to a branch. If the consensus is to do so, we can patch trunk and leave the branch patch for those who wish it.

          Show
          jghoman Jakob Homan added a comment - I'm willing to do the silent default and mark as incompatible on trunk, but not back to a branch. If the consensus is to do so, we can patch trunk and leave the branch patch for those who wish it.
          Hide
          ajisakaa Akira Ajisaka added a comment -

          I agree. I suppose it would be better to do the silent default and mark as incompatible on trunk, and add -quiet option in branch-2.

          Show
          ajisakaa Akira Ajisaka added a comment - I agree. I suppose it would be better to do the silent default and mark as incompatible on trunk, and add -quiet option in branch-2.
          Hide
          jghoman Jakob Homan added a comment -

          No, I don't want extra options and differing capabilities running around. If I commit an incompatible change on trunk, I don't want to apply a different patch to the branch.

          Show
          jghoman Jakob Homan added a comment - No, I don't want extra options and differing capabilities running around. If I commit an incompatible change on trunk, I don't want to apply a different patch to the branch.
          Hide
          ajisakaa Akira Ajisaka added a comment -

          If I commit an incompatible change on trunk, I don't want to apply a different patch to the branch.

          I understand that. Thank you, Jakob Homan.

          Show
          ajisakaa Akira Ajisaka added a comment - If I commit an incompatible change on trunk, I don't want to apply a different patch to the branch. I understand that. Thank you, Jakob Homan .
          Hide
          kamrul Mohammad Kamrul Islam added a comment -

          Uploaded patch based on consensus approach.
          This is only for trunk and backward incompatible.

          Show
          kamrul Mohammad Kamrul Islam added a comment - Uploaded patch based on consensus approach. This is only for trunk and backward incompatible.
          Hide
          jghoman Jakob Homan added a comment -

          Anyone have concerns over the plan to make this backwards incompatible, apply to trunk and not apply to branch? That's my intention. Speak now, yadda, yadda.

          Show
          jghoman Jakob Homan added a comment - Anyone have concerns over the plan to make this backwards incompatible, apply to trunk and not apply to branch? That's my intention. Speak now, yadda, yadda.
          Hide
          hadoopqa Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12631611/HDFS-2538.3.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          -1 tests included. The patch doesn't appear to include any new or modified tests.
          Please justify why no new tests are needed for this patch.
          Also please list what manual steps were performed to verify this patch.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. There were no new javadoc warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 core tests. The patch failed these unit tests in hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs:

          org.apache.hadoop.hdfs.server.namenode.ha.TestHASafeMode

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/6260//testReport/
          Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/6260//console

          This message is automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12631611/HDFS-2538.3.patch against trunk revision . +1 @author . The patch does not contain any @author tags. -1 tests included . The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . There were no new javadoc warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. -1 core tests . The patch failed these unit tests in hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs: org.apache.hadoop.hdfs.server.namenode.ha.TestHASafeMode +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/6260//testReport/ Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/6260//console This message is automatically generated.
          Hide
          aw Allen Wittenauer added a comment -

          As long as the patch goes in with the default off, it's fine.

          Show
          aw Allen Wittenauer added a comment - As long as the patch goes in with the default off, it's fine.
          Hide
          ajisakaa Akira Ajisaka added a comment -

          Thanks for the update, Mohammad Kamrul Islam.
          I'm okay with the default silent in trunk. I want the default off and mark as incompatible in branch-2 also. I think it would be better if users can disable dots in the next or after the next release.

          Show
          ajisakaa Akira Ajisaka added a comment - Thanks for the update, Mohammad Kamrul Islam . I'm okay with the default silent in trunk. I want the default off and mark as incompatible in branch-2 also. I think it would be better if users can disable dots in the next or after the next release.
          Hide
          aw Allen Wittenauer added a comment -

          Committed. Thank your for contribution for the betterment of society.

          Show
          aw Allen Wittenauer added a comment - Committed. Thank your for contribution for the betterment of society.
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-trunk-Commit #5895 (See https://builds.apache.org/job/Hadoop-trunk-Commit/5895/)
          HDFS-2538. option to disable fsck dots. Contributed by Mohammad Kamrul Islam. (aw: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1611220)

          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/site/apt/CommandsManual.apt.vm
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NamenodeFsck.java
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/DFSck.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-trunk-Commit #5895 (See https://builds.apache.org/job/Hadoop-trunk-Commit/5895/ ) HDFS-2538 . option to disable fsck dots. Contributed by Mohammad Kamrul Islam. (aw: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1611220 ) /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/site/apt/CommandsManual.apt.vm /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NamenodeFsck.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/DFSck.java
          Hide
          aw Allen Wittenauer added a comment -

          Failure came from YARN-2219. Nothing to worry about.

          Show
          aw Allen Wittenauer added a comment - Failure came from YARN-2219 . Nothing to worry about.
          Hide
          ajisakaa Akira Ajisaka added a comment -

          Since this change is an incompatible change, adding a release note.

          Show
          ajisakaa Akira Ajisaka added a comment - Since this change is an incompatible change, adding a release note.
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Yarn-trunk #615 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/615/)
          HDFS-2538. option to disable fsck dots. Contributed by Mohammad Kamrul Islam. (aw: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1611220)

          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/site/apt/CommandsManual.apt.vm
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NamenodeFsck.java
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/DFSck.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk #615 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/615/ ) HDFS-2538 . option to disable fsck dots. Contributed by Mohammad Kamrul Islam. (aw: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1611220 ) /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/site/apt/CommandsManual.apt.vm /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NamenodeFsck.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/DFSck.java
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Mapreduce-trunk #1834 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1834/)
          HDFS-2538. option to disable fsck dots. Contributed by Mohammad Kamrul Islam. (aw: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1611220)

          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/site/apt/CommandsManual.apt.vm
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NamenodeFsck.java
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/DFSck.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk #1834 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1834/ ) HDFS-2538 . option to disable fsck dots. Contributed by Mohammad Kamrul Islam. (aw: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1611220 ) /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/site/apt/CommandsManual.apt.vm /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NamenodeFsck.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/DFSck.java
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Hdfs-trunk #1807 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1807/)
          HDFS-2538. option to disable fsck dots. Contributed by Mohammad Kamrul Islam. (aw: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1611220)

          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/site/apt/CommandsManual.apt.vm
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NamenodeFsck.java
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/DFSck.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk #1807 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1807/ ) HDFS-2538 . option to disable fsck dots. Contributed by Mohammad Kamrul Islam. (aw: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1611220 ) /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/site/apt/CommandsManual.apt.vm /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NamenodeFsck.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/DFSck.java
          Hide
          ajisakaa Akira Ajisaka added a comment -

          After HDFS-2538, fsck can fail by SocketTimeoutException. This issue is tracked by HDFS-7175.

          Show
          ajisakaa Akira Ajisaka added a comment - After HDFS-2538 , fsck can fail by SocketTimeoutException. This issue is tracked by HDFS-7175 .
          Hide
          brahmareddy Brahma Reddy Battula added a comment -

          Konstantin Shvachko IMO we no need to backport this, since this incompatible change (even there is open broken jira HDFS-7175).

          As per versioning rule here,I think, we should not add incompatible changes to dot and minor releases..?

          Show
          brahmareddy Brahma Reddy Battula added a comment - Konstantin Shvachko IMO we no need to backport this, since this incompatible change (even there is open broken jira HDFS-7175 ). As per versioning rule here ,I think, we should not add incompatible changes to dot and minor releases..?
          Hide
          elgoiri Íñigo Goiri added a comment -

          Backporting to 2.7.4.

          Show
          elgoiri Íñigo Goiri added a comment - Backporting to 2.7.4.
          Hide
          elgoiri Íñigo Goiri added a comment -

          Backport for 2.7.4.

          Show
          elgoiri Íñigo Goiri added a comment - Backport for 2.7.4.
          Hide
          elgoiri Íñigo Goiri added a comment -

          Brahma Reddy Battula mentions, this is an incompatible change and there was some controversy.
          Konstantin Shvachko, feel free to ignore and close the JIRA again but remove the 2.7.4 flag.

          Show
          elgoiri Íñigo Goiri added a comment - Brahma Reddy Battula mentions, this is an incompatible change and there was some controversy. Konstantin Shvachko , feel free to ignore and close the JIRA again but remove the 2.7.4 flag.
          Hide
          shv Konstantin Shvachko added a comment -

          Resolving it back. The incompatibility concern is valid. I am still thinking about if we can / should include it. Sorry for confusion.
          Thanks for the patch Íñigo Goiri.

          Show
          shv Konstantin Shvachko added a comment - Resolving it back. The incompatibility concern is valid. I am still thinking about if we can / should include it. Sorry for confusion. Thanks for the patch Íñigo Goiri .
          Hide
          brahmareddy Brahma Reddy Battula added a comment -

          shall we remove 2.7.4 from target version and releaseblocker from label..?

          Show
          brahmareddy Brahma Reddy Battula added a comment - shall we remove 2.7.4 from target version and releaseblocker from label..?
          Hide
          ajisakaa Akira Ajisaka added a comment - - edited

          shall we remove 2.7.4 from target version and releaseblocker from label..?

          I think yes. What do you think Konstantin Shvachko?

          Show
          ajisakaa Akira Ajisaka added a comment - - edited shall we remove 2.7.4 from target version and releaseblocker from label..? I think yes. What do you think Konstantin Shvachko ?
          Hide
          shv Konstantin Shvachko added a comment -

          Hey guys,
          I think if we reverse the default to print with dots, this is not incompatible. So let's keep it for now.
          Let me check with Carl Steinbach if he remembers the context behind HDFS-7175 before final decision.

          Show
          shv Konstantin Shvachko added a comment - Hey guys, I think if we reverse the default to print with dots, this is not incompatible. So let's keep it for now. Let me check with Carl Steinbach if he remembers the context behind HDFS-7175 before final decision.
          Hide
          shv Konstantin Shvachko added a comment -

          Talked to people who were involved with this locally back then and recently.
          The bottom line for me is that SREs still run fsck with -showprogress option with this patch applied. So what's the point.
          Will not commit to 2.7.4.

          Show
          shv Konstantin Shvachko added a comment - Talked to people who were involved with this locally back then and recently. The bottom line for me is that SREs still run fsck with -showprogress option with this patch applied. So what's the point. Will not commit to 2.7.4.

            People

            • Assignee:
              kamrul Mohammad Kamrul Islam
              Reporter:
              aw Allen Wittenauer
            • Votes:
              0 Vote for this issue
              Watchers:
              21 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development