Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-5952

Create a tool to run data analysis on the PB format fsimage

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Patch Available
    • Major
    • Resolution: Unresolved
    • 2.6.0
    • None
    • tools
    • Bring back the delimited processor and resolve memory problem about offline image viewer on PB-based fsimage.

    Description

      Delimited processor in OfflineImageViewer is not supported after HDFS-5698 was merged.
      The motivation of delimited processor is to run data analysis on the fsimage, therefore, there might be more values to create a tool for Hive or Pig that reads the PB format fsimage directly.

      Attachments

        1. HDFS-5952.patch
          16 kB
          Hao Chen

        Issue Links

          is blocked by

          Bug - A problem which impairs or prevents the functions of the product. HDFS-6914 Resolve huge memory consumption Issue with OIV processing PB-based fsimages

          • Major - Major loss of function.
          • Patch Available
          is related to

          Sub-task - The sub-task of the issue HDFS-6673 Add delimited format support to PB OIV tool

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Closed

          Activity

            People

              Unassigned Unassigned
              aajisaka Akira Ajisaka
              Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

                Created:
                Updated: