Hadoop HDFS
  1. Hadoop HDFS
  2. HDFS-5863 Improve OfflineImageViewer
  3. HDFS-5975

Create an option to specify a file path for OfflineImageViewer

    Details

    • Type: Sub-task Sub-task
    • Status: Resolved
    • Priority: Minor Minor
    • Resolution: Invalid
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: tools
    • Labels:
      None
    • Target Version/s:

      Description

      The output of OfflineImageViewer becomes quite large if an input fsimage is large. I propose '-filePath' option to make the output smaller.

      The below command will output the ls -R of /user/root.

      hdfs oiv -i input -o output -p Ls -filePath /user/root
      
      1. HDFS-5975.patch
        8 kB
        Akira AJISAKA
      2. HDFS-5975.2.patch
        8 kB
        Akira AJISAKA

        Issue Links

          Activity

          Hide
          Haohui Mai added a comment -

          I think this feature has good practical impact, since the operator rarely needs to do a full lsr starting from the root directory. LsrPBImage should output on-demand.

          My suggestion is to push this idea one step further – is it possible to create a tool which takes the fsimage, and exposes the read-only version of WebHDFS API? You can imagine the tool looks very similar to jhat, except that it exposes the WebHDFS API.

          That way we can allow the operator to use the existing command-line tool, or even the web UI to debug the fsimage. It also allows the operator to interactively browsing the file system, figuring out what goes wrong.

          Show
          Haohui Mai added a comment - I think this feature has good practical impact, since the operator rarely needs to do a full lsr starting from the root directory. LsrPBImage should output on-demand. My suggestion is to push this idea one step further – is it possible to create a tool which takes the fsimage, and exposes the read-only version of WebHDFS API? You can imagine the tool looks very similar to jhat, except that it exposes the WebHDFS API. That way we can allow the operator to use the existing command-line tool, or even the web UI to debug the fsimage. It also allows the operator to interactively browsing the file system, figuring out what goes wrong.
          Hide
          Akira AJISAKA added a comment -

          That's a good idea! I'll create another JIRA.

          Show
          Akira AJISAKA added a comment - That's a good idea! I'll create another JIRA.
          Hide
          Akira AJISAKA added a comment -

          Attaching a patch.

          Show
          Akira AJISAKA added a comment - Attaching a patch.
          Hide
          Hadoop QA added a comment -

          +1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12629977/HDFS-5975.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 1 new or modified test files.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. There were no new javadoc warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed unit tests in hadoop-hdfs-project/hadoop-hdfs.

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/6188//testReport/
          Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/6188//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - +1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12629977/HDFS-5975.patch against trunk revision . +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 1 new or modified test files. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . There were no new javadoc warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 core tests . The patch passed unit tests in hadoop-hdfs-project/hadoop-hdfs. +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/6188//testReport/ Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/6188//console This message is automatically generated.
          Hide
          Akira AJISAKA added a comment -

          Rebased the patch for the latest trunk.

          Show
          Akira AJISAKA added a comment - Rebased the patch for the latest trunk.
          Hide
          Hadoop QA added a comment -

          +1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12630351/HDFS-5975.2.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 1 new or modified test files.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. There were no new javadoc warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed unit tests in hadoop-hdfs-project/hadoop-hdfs.

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/6205//testReport/
          Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/6205//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - +1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12630351/HDFS-5975.2.patch against trunk revision . +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 1 new or modified test files. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . There were no new javadoc warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 core tests . The patch passed unit tests in hadoop-hdfs-project/hadoop-hdfs. +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/6205//testReport/ Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/6205//console This message is automatically generated.
          Hide
          Akira AJISAKA added a comment -

          Haohui Mai, cloud you please review the latest patch?

          Show
          Akira AJISAKA added a comment - Haohui Mai , cloud you please review the latest patch?
          Hide
          Akira AJISAKA added a comment -

          Lsr processor was removed by HDFS-6164, so this issue is now invalid.

          Show
          Akira AJISAKA added a comment - Lsr processor was removed by HDFS-6164 , so this issue is now invalid.

            People

            • Assignee:
              Unassigned
              Reporter:
              Akira AJISAKA
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development