Hadoop HDFS
  1. Hadoop HDFS
  2. HDFS-5863 Improve OfflineImageViewer
  3. HDFS-5865

Update OfflineImageViewer document

        Details

        • Type: Sub-task Sub-task
        • Status: Closed
        • Priority: Minor Minor
        • Resolution: Fixed
        • Affects Version/s: 2.4.0
        • Fix Version/s: 2.5.0
        • Component/s: documentation
        • Labels:
        • Target Version/s:
        • Hadoop Flags:
          Reviewed

          Description

          OfflineImageViewer is renewed to handle the new format of fsimage by HDFS-5698 (fsimage in protobuf).

          We should document the followings:

          • The tool can handle the layout version of Hadoop 2.4 and up. (If you want to handle the older version, you can use OfflineImageViewer of Hadoop 2.3)
          • Delimited, Indented, and Ls processor were removed.
          • A new Web processor, which supersedes the Ls processor, was added.
          1. Text File
            HDFS-5865.2.patch
            23 kB
            Akira Ajisaka
          2. Text File
            HDFS-5865.patch
            18 kB
            Akira Ajisaka

            Activity

            Ascending order - Click to sort in descending order
            Hide
            Permalink
            Akira Ajisaka added a comment -

            Attaching a patch.

            Show
            Akira Ajisaka added a comment - Attaching a patch.
            Hide
            Permalink
            Hadoop QA added a comment -

            -1 overall. Here are the results of testing the latest attachment
            http://issues.apache.org/jira/secure/attachment/12630161/HDFS-5865.patch
            against trunk revision .

            +1 @author. The patch does not contain any @author tags.

            +0 tests included. The patch appears to be a documentation patch that doesn't require tests.

            +1 javac. The applied patch does not increase the total number of javac compiler warnings.

            +1 javadoc. There were no new javadoc warning messages.

            +1 eclipse:eclipse. The patch built with eclipse:eclipse.

            +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

            +1 release audit. The applied patch does not increase the total number of release audit warnings.

            -1 core tests. The patch failed these unit tests in hadoop-hdfs-project/hadoop-hdfs:

            org.apache.hadoop.fs.loadGenerator.TestLoadGenerator

            +1 contrib tests. The patch passed contrib unit tests.

            Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/6197//testReport/
            Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/6197//console

            This message is automatically generated.

            Show
            Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12630161/HDFS-5865.patch against trunk revision . +1 @author . The patch does not contain any @author tags. +0 tests included . The patch appears to be a documentation patch that doesn't require tests. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . There were no new javadoc warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. -1 core tests . The patch failed these unit tests in hadoop-hdfs-project/hadoop-hdfs: org.apache.hadoop.fs.loadGenerator.TestLoadGenerator +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/6197//testReport/ Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/6197//console This message is automatically generated.
            Hide
            Permalink
            Akira Ajisaka added a comment -

            Haohui Mai, would you please review the patch for the 2.4 release?
            Also, we should also update the document for branch-2 later.

            Show
            Akira Ajisaka added a comment - Haohui Mai , would you please review the patch for the 2.4 release? Also, we should also update the document for branch-2 later.
            Hide
            Permalink
            Haohui Mai added a comment -

            Downgrading to minor since documentation issues block no functionalities.

            By the way, since several major changes (e.g., HDFS-5978 and HDFS-6164) have been committed in to branch-2. Can you please provide a patch of branch-2 instead of targeting 2.4?

            Show
            Haohui Mai added a comment - Downgrading to minor since documentation issues block no functionalities. By the way, since several major changes (e.g., HDFS-5978 and HDFS-6164 ) have been committed in to branch-2. Can you please provide a patch of branch-2 instead of targeting 2.4?
            Hide
            Permalink
            Akira Ajisaka added a comment -

            I can provide a patch of branch-2, but I think 2.4 is the first. We should document the deprecated options and backward incompatibility because major changes have been committed in 2.4 branch also.

            Show
            Akira Ajisaka added a comment - I can provide a patch of branch-2, but I think 2.4 is the first. We should document the deprecated options and backward incompatibility because major changes have been committed in 2.4 branch also.
            Hide
            Permalink
            Akira Ajisaka added a comment -

            Since 2.4.0-rc0 will come soon, I'll update the description and create a patch for branch-2.

            Show
            Akira Ajisaka added a comment - Since 2.4.0-rc0 will come soon, I'll update the description and create a patch for branch-2.
            Hide
            Permalink
            Haohui Mai added a comment - - edited 
            +| <<<-i>>>\u007c<<<--inputFile>>> <input file> | Specify the input fsimage file
+| <<<-o>>>\u007c<<<--outputFile>>> <output file> | Specify the output filename,
+| <<<-p>>>\u007c<<<--processor>>> <processor> | Specify the image processor to
+| <<<-h>>>\u007c<<<--help>>>| Display the tool usage and help information and
...

            why using \u007c instead of |, which is in the old document?

            There are several paragraphs that are specific to the ls output processor haven't been cleaned up yet. For example:

               This will create a file named fsimage.txt in the current directory
   using the Ls output processor. For very large image files, this process
   may take several minutes.

   One can specify which output processor via the command-line switch -p.
   For instance:
            Show
            Haohui Mai added a comment - - edited +| <<<-i>>>\u007c<<<--inputFile>>> <input file> | Specify the input fsimage file +| <<<-o>>>\u007c<<<--outputFile>>> <output file> | Specify the output filename, +| <<<-p>>>\u007c<<<--processor>>> <processor> | Specify the image processor to +| <<<-h>>>\u007c<<<--help>>>| Display the tool usage and help information and ... why using \u007c instead of | , which is in the old document? There are several paragraphs that are specific to the ls output processor haven't been cleaned up yet. For example: This will create a file named fsimage.txt in the current directory using the Ls output processor. For very large image files, this process may take several minutes. One can specify which output processor via the command-line switch -p. For instance:
            Hide
            Permalink
            Akira Ajisaka added a comment -

            Thanks for the review, Haohui.

            why using \u007c instead of |, which is in the old document?

            I thought | was not available in a cell, but I confirmed it was available. I'll use | in the next patch.

            There are several paragraphs that are specific to the ls output processor haven't been cleaned up yet.

            Ls was not cleaned up yet in the patch because this patch was for branch-2.4. I'll update the patch.

            Show
            Akira Ajisaka added a comment - Thanks for the review, Haohui. why using \u007c instead of |, which is in the old document? I thought | was not available in a cell, but I confirmed it was available. I'll use | in the next patch. There are several paragraphs that are specific to the ls output processor haven't been cleaned up yet. Ls was not cleaned up yet in the patch because this patch was for branch-2.4. I'll update the patch.
            Hide
            Permalink
            Akira Ajisaka added a comment -

            Attaching a patch to add the description of the Web processor.

            Show
            Akira Ajisaka added a comment - Attaching a patch to add the description of the Web processor.
            Hide
            Permalink
            Hadoop QA added a comment -

            +1 overall. Here are the results of testing the latest attachment
            http://issues.apache.org/jira/secure/attachment/12641879/HDFS-5865.2.patch
            against trunk revision .

            +1 @author. The patch does not contain any @author tags.

            +0 tests included. The patch appears to be a documentation patch that doesn't require tests.

            +1 javac. The applied patch does not increase the total number of javac compiler warnings.

            +1 javadoc. There were no new javadoc warning messages.

            +1 eclipse:eclipse. The patch built with eclipse:eclipse.

            +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

            +1 release audit. The applied patch does not increase the total number of release audit warnings.

            +1 core tests. The patch passed unit tests in hadoop-hdfs-project/hadoop-hdfs.

            +1 contrib tests. The patch passed contrib unit tests.

            Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/6731//testReport/
            Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/6731//console

            This message is automatically generated.

            Show
            Hadoop QA added a comment - +1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12641879/HDFS-5865.2.patch against trunk revision . +1 @author . The patch does not contain any @author tags. +0 tests included . The patch appears to be a documentation patch that doesn't require tests. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . There were no new javadoc warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 core tests . The patch passed unit tests in hadoop-hdfs-project/hadoop-hdfs. +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/6731//testReport/ Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/6731//console This message is automatically generated.
            Hide
            Permalink
            Haohui Mai added a comment -

            +1

            Show
            Haohui Mai added a comment - +1
            Hide
            Permalink
            Haohui Mai added a comment -

            I've committed the patch to trunk and branch-2. Thanks Akira Ajisaka for the contribution.

            Show
            Haohui Mai added a comment - I've committed the patch to trunk and branch-2. Thanks Akira Ajisaka for the contribution.
            Hide
            Permalink
            Hudson added a comment -

            SUCCESS: Integrated in Hadoop-trunk-Commit #5574 (See https://builds.apache.org/job/Hadoop-trunk-Commit/5574/)
            HDFS-5865. Update OfflineImageViewer document. Contributed by Akira Ajisaka. (wheat9: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1590100)

            • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
            • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/site/apt/HdfsImageViewer.apt.vm
            Show
            Hudson added a comment - SUCCESS: Integrated in Hadoop-trunk-Commit #5574 (See https://builds.apache.org/job/Hadoop-trunk-Commit/5574/ ) HDFS-5865 . Update OfflineImageViewer document. Contributed by Akira Ajisaka. (wheat9: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1590100 ) /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/site/apt/HdfsImageViewer.apt.vm
            Hide
            Permalink
            Hudson added a comment -

            FAILURE: Integrated in Hadoop-Yarn-trunk #552 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/552/)
            HDFS-5865. Update OfflineImageViewer document. Contributed by Akira Ajisaka. (wheat9: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1590100)

            • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
            • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/site/apt/HdfsImageViewer.apt.vm
            Show
            Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk #552 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/552/ ) HDFS-5865 . Update OfflineImageViewer document. Contributed by Akira Ajisaka. (wheat9: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1590100 ) /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/site/apt/HdfsImageViewer.apt.vm
            Hide
            Permalink
            Hudson added a comment -

            SUCCESS: Integrated in Hadoop-Hdfs-trunk #1743 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1743/)
            HDFS-5865. Update OfflineImageViewer document. Contributed by Akira Ajisaka. (wheat9: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1590100)

            • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
            • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/site/apt/HdfsImageViewer.apt.vm
            Show
            Hudson added a comment - SUCCESS: Integrated in Hadoop-Hdfs-trunk #1743 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1743/ ) HDFS-5865 . Update OfflineImageViewer document. Contributed by Akira Ajisaka. (wheat9: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1590100 ) /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/site/apt/HdfsImageViewer.apt.vm
            Hide
            Permalink
            Hudson added a comment -

            FAILURE: Integrated in Hadoop-Mapreduce-trunk #1769 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1769/)
            HDFS-5865. Update OfflineImageViewer document. Contributed by Akira Ajisaka. (wheat9: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1590100)

            • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
            • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/site/apt/HdfsImageViewer.apt.vm
            Show
            Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk #1769 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1769/ ) HDFS-5865 . Update OfflineImageViewer document. Contributed by Akira Ajisaka. (wheat9: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1590100 ) /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/site/apt/HdfsImageViewer.apt.vm

              People

              • Assignee:
                Akira Ajisaka
                Reporter:
                Akira Ajisaka
              • Votes:
                0 Vote for this issue
                Watchers:
                6 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Development