Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-6931

Remove TestDFSIO "Total Throughput" calculation

    Details

    • Hadoop Flags:
      Reviewed

      Description

      The new "Total Throughput" line added in https://issues.apache.org/jira/browse/HDFS-9153 is currently calculated as toMB(size) / ((float)execTime) and claims to be in units of "MB/s", but execTime is in milliseconds; thus, the reported number is 1/1000x the actual value:

          String resultLines[] = {
              "----- TestDFSIO ----- : " + testType,
              "            Date & time: " + new Date(System.currentTimeMillis()),
              "        Number of files: " + tasks,
              " Total MBytes processed: " + df.format(toMB(size)),
              "      Throughput mb/sec: " + df.format(size * 1000.0 / (time * MEGA)),
              "Total Throughput mb/sec: " + df.format(toMB(size) / ((float)execTime)),
              " Average IO rate mb/sec: " + df.format(med),
              "  IO rate std deviation: " + df.format(stdDev),
              "     Test exec time sec: " + df.format((float)execTime / 1000),
              "" };
      

      The different calculated fields can also use toMB and a shared milliseconds-to-seconds conversion to make it easier to keep units consistent.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                dennishuo Dennis Huo
                Reporter:
                dennishuo Dennis Huo
              • Votes:
                0 Vote for this issue
                Watchers:
                7 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: