Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-1338

Improve TestDFSIO

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Open
    • Major
    • Resolution: Unresolved
    • None
    • None
    • test
    • None

    Description

      Currently the read test in TestDFSIO benchmark just opens a large side file and measures the read performance. The MR scheduler has no opportunity to do any optimization for the TestDFSIO MR application. The side-effect of this is that it is very hard to do any meaningful analysis of the results of the benchmark i.e. to check if node-local or rack-local or off-switch read performance improved/degraded.

      Attachments

        1. TestDFSIOparser_y0.20.patch
          4 kB
          Tsz-wo Sze

        Issue Links

          relates to

          Bug - A problem which impairs or prevents the functions of the product. MAPREDUCE-2023 TestDFSIO read test may not read specified bytes.

          • Major - Major loss of function.
          • Closed

          Activity

            People

              Unassigned Unassigned
              acmurthy Arun Murthy
              Votes:
              0 Vote for this issue
              Watchers:
              14 Start watching this issue

              Dates

                Created:
                Updated: