Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-6931

Remove TestDFSIO "Total Throughput" calculation

    XMLWordPrintableJSON

Details

    • Reviewed

    Description

      The new "Total Throughput" line added in https://issues.apache.org/jira/browse/HDFS-9153 is currently calculated as toMB(size) / ((float)execTime) and claims to be in units of "MB/s", but execTime is in milliseconds; thus, the reported number is 1/1000x the actual value:

          String resultLines[] = {
              "----- TestDFSIO ----- : " + testType,
              "            Date & time: " + new Date(System.currentTimeMillis()),
              "        Number of files: " + tasks,
              " Total MBytes processed: " + df.format(toMB(size)),
              "      Throughput mb/sec: " + df.format(size * 1000.0 / (time * MEGA)),
              "Total Throughput mb/sec: " + df.format(toMB(size) / ((float)execTime)),
              " Average IO rate mb/sec: " + df.format(med),
              "  IO rate std deviation: " + df.format(stdDev),
              "     Test exec time sec: " + df.format((float)execTime / 1000),
              "" };
      

      The different calculated fields can also use toMB and a shared milliseconds-to-seconds conversion to make it easier to keep units consistent.

      Attachments

        1. MAPREDUCE-6931-001.patch
          3 kB
          Konstantin Shvachko

        Issue Links

          Activity

            People

              dennishuo Dennis Huo
              dennishuo Dennis Huo
              Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: