Details
-
Type:
Sub-task
-
Status: Closed
-
Priority:
Minor
-
Resolution: Fixed
-
Affects Version/s: 2.4.0
-
Fix Version/s: 2.5.0
-
Component/s: documentation
-
Labels:
-
Target Version/s:
-
Hadoop Flags:Reviewed
Description
OfflineImageViewer is renewed to handle the new format of fsimage by HDFS-5698 (fsimage in protobuf).
We should document the followings:
- The tool can handle the layout version of Hadoop 2.4 and up. (If you want to handle the older version, you can use OfflineImageViewer of Hadoop 2.3)
- Delimited, Indented, and Ls processor were removed.
- A new Web processor, which supersedes the Ls processor, was added.
Activity
| Status | Resolved [ 5 ] | Closed [ 6 ] |
SUCCESS: Integrated in Hadoop-Hdfs-trunk #1743 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1743/)
HDFS-5865. Update OfflineImageViewer document. Contributed by Akira Ajisaka. (wheat9: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1590100)
- /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
- /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/site/apt/HdfsImageViewer.apt.vm
FAILURE: Integrated in Hadoop-Yarn-trunk #552 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/552/)
HDFS-5865. Update OfflineImageViewer document. Contributed by Akira Ajisaka. (wheat9: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1590100)
- /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
- /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/site/apt/HdfsImageViewer.apt.vm
SUCCESS: Integrated in Hadoop-trunk-Commit #5574 (See https://builds.apache.org/job/Hadoop-trunk-Commit/5574/)
HDFS-5865. Update OfflineImageViewer document. Contributed by Akira Ajisaka. (wheat9: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1590100)
- /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
- /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/site/apt/HdfsImageViewer.apt.vm
| Status | Patch Available [ 10002 ] | Resolved [ 5 ] |
| Hadoop Flags | Reviewed [ 10343 ] | |
| Fix Version/s | 2.5.0 [ 12326264 ] | |
| Resolution | Fixed [ 1 ] |
I've committed the patch to trunk and branch-2. Thanks Akira AJISAKA for the contribution.
+1
+1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12641879/HDFS-5865.2.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+0 tests included. The patch appears to be a documentation patch that doesn't require tests.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 javadoc. There were no new javadoc warning messages.
+1 eclipse:eclipse. The patch built with eclipse:eclipse.
+1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
+1 core tests. The patch passed unit tests in hadoop-hdfs-project/hadoop-hdfs.
+1 contrib tests. The patch passed contrib unit tests.
Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/6731//testReport/
Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/6731//console
This message is automatically generated.
| Description |
OfflineImageViewer is renewed to handle the new format of fsimage by We should document followings: * The tool can handle the layout version of Hadoop 2.4 and up. (If you want to handle the older version, you can use OfflineImageViewer of Hadoop 2.3) * Remove deprecated options such as Delimited and Indented processor. |
OfflineImageViewer is renewed to handle the new format of fsimage by We should document the followings: * The tool can handle the layout version of Hadoop 2.4 and up. (If you want to handle the older version, you can use OfflineImageViewer of Hadoop 2.3) * Delimited, Indented, and Ls processor were removed. * A new Web processor, which supersedes the Ls processor, was added. |
| Status | Open [ 1 ] | Patch Available [ 10002 ] |
| Attachment | HDFS-5865.2.patch [ 12641879 ] |
Attaching a patch to add the description of the Web processor.
Thanks for the review, Haohui.
why using \u007c instead of |, which is in the old document?
I thought | was not available in a cell, but I confirmed it was available. I'll use | in the next patch.
There are several paragraphs that are specific to the ls output processor haven't been cleaned up yet.
Ls was not cleaned up yet in the patch because this patch was for branch-2.4. I'll update the patch.
+| <<<-i>>>\u007c<<<--inputFile>>> <input file> | Specify the input fsimage file +| <<<-o>>>\u007c<<<--outputFile>>> <output file> | Specify the output filename, +| <<<-p>>>\u007c<<<--processor>>> <processor> | Specify the image processor to +| <<<-h>>>\u007c<<<--help>>>| Display the tool usage and help information and ...
why using \u007c instead of |, which is in the old document?
There are several paragraphs that are specific to the ls output processor haven't been cleaned up yet. For example:
This will create a file named fsimage.txt in the current directory using the Ls output processor. For very large image files, this process may take several minutes. One can specify which output processor via the command-line switch -p. For instance:
| Summary | Update OfflineImageViewer document for branch-2.4 | Update OfflineImageViewer document |
| Status | Patch Available [ 10002 ] | Open [ 1 ] |
Since 2.4.0-rc0 will come soon, I'll update the description and create a patch for branch-2.
I can provide a patch of branch-2, but I think 2.4 is the first. We should document the deprecated options and backward incompatibility because major changes have been committed in 2.4 branch also.
| Target Version/s | 2.4.0 [ 12326143 ] | 2.5.0 [ 12326264 ] |
| Priority | Blocker [ 1 ] | Minor [ 4 ] |
| Summary | Update OfflineImageViewer document | Update OfflineImageViewer document for branch-2.4 |
| Priority | Major [ 3 ] | Blocker [ 1 ] |
Haohui Mai, would you please review the patch for the 2.4 release?
Also, we should also update the document for branch-2 later.
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12630161/HDFS-5865.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+0 tests included. The patch appears to be a documentation patch that doesn't require tests.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 javadoc. There were no new javadoc warning messages.
+1 eclipse:eclipse. The patch built with eclipse:eclipse.
+1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
-1 core tests. The patch failed these unit tests in hadoop-hdfs-project/hadoop-hdfs:
org.apache.hadoop.fs.loadGenerator.TestLoadGenerator
+1 contrib tests. The patch passed contrib unit tests.
Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/6197//testReport/
Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/6197//console
This message is automatically generated.
| Status | Open [ 1 ] | Patch Available [ 10002 ] |
| Affects Version/s | 2.4.0 [ 12326143 ] | |
| Affects Version/s | 3.0.0 [ 12320356 ] | |
| Target Version/s | 3.0.0 [ 12320356 ] | 2.4.0 [ 12326143 ] |
| Priority | Minor [ 4 ] | Major [ 3 ] |
| Description | The Offline Image Viewer document describes "Currently valid options are {{Ls}}, {{XML}}, and {{Indented}}" in {{--processor}} option, but now valid options are {{Ls}}, {{XML}}, and {{FileDistribution}}. |
OfflineImageViewer is renewed to handle the new format of fsimage by We should document followings: * The tool can handle the layout version of Hadoop 2.4 and up. (If you want to handle the older version, you can use OfflineImageViewer of Hadoop 2.3) * Remove deprecated options such as Delimited and Indented processor. |
| Summary | Document 'FileDistribution' argument in 'hdfs oiv --processor' option | Update OfflineImageViewer document |
| Attachment | HDFS-5865.patch [ 12630161 ] |
Attaching a patch.
| Assignee | Akira AJISAKA [ ajisakaa ] |
| Field | Original Value | New Value |
|---|---|---|
| Summary | Document some arguments in 'hdfs oiv --processor' option | Document 'FileDistribution' argument in 'hdfs oiv --processor' option |
| Affects Version/s | 3.0.0 [ 12320356 ] | |
| Affects Version/s | 2.2.0 [ 12325049 ] | |
| Target Version/s | 2.4.0 [ 12326143 ] | 3.0.0 [ 12320356 ] |
| Priority | Major [ 3 ] | Minor [ 4 ] |
| Description | The Offline Image Viewer document now describes "Currently valid options are {{Ls}}, {{XML}}, and {{Indented}}" in {{--processor}} option, but there're more options such as {{Delimited}}, {{FileDistribution}}, and {{NameDistribution}}. | The Offline Image Viewer document describes "Currently valid options are {{Ls}}, {{XML}}, and {{Indented}}" in {{--processor}} option, but now valid options are {{Ls}}, {{XML}}, and {{FileDistribution}}. |
FAILURE: Integrated in Hadoop-Mapreduce-trunk #1769 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1769/)
HDFS-5865. Update OfflineImageViewer document. Contributed by Akira Ajisaka. (wheat9: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1590100)