Details

    • Type: Test Test
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.2.0
    • Component/s: fs
    • Labels:
      None

      Description

      DFS i/o benchmark is a map-reduce based test that measures performance of the cluster for reads and writes.
      This is an evolved version of HADOOP-72, and HADOOP-95 test.

      This test writes into or reads from a specified number of files.
      File size is specified as a parameter to the test.
      Each file is processed in a separate map task.
      The unique reducer then collects stats.
      Finally, the following information is displayed

      1. read or write test
      2. date and time the test finished
      3. number of files processed
      4. total number of bytes processed
      5. throughput in mb/sec (total number of bytes / sum of processing times)
      6. average i/o rate in mb/sec per file
      7. standard i/o rate deviation

      I included the test into the AllTestDriver.

      1. IOBenchmark.patch
        23 kB
        Konstantin Shvachko

        Issue Links

          Activity

          Konstantin Shvachko created issue -
          Konstantin Shvachko made changes -
          Field Original Value New Value
          Attachment IOBenchmark.patch [ 12326226 ]
          Hide
          Doug Cutting added a comment -

          I just committed this. Thanks, Konstantin.

          Show
          Doug Cutting added a comment - I just committed this. Thanks, Konstantin.
          Doug Cutting made changes -
          Fix Version/s 0.2 [ 12310813 ]
          Resolution Fixed [ 1 ]
          Status Open [ 1 ] Resolved [ 5 ]
          Doug Cutting made changes -
          Status Resolved [ 5 ] Closed [ 6 ]
          Doug Cutting made changes -
          Workflow jira [ 12362013 ] no reopen closed [ 12373112 ]
          Doug Cutting made changes -
          Workflow no reopen closed [ 12373112 ] no-reopen-closed [ 12373448 ]
          Doug Cutting made changes -
          Workflow no-reopen-closed [ 12373448 ] no-reopen-closed, patch-avail [ 12377759 ]
          Konstantin Shvachko made changes -
          Link This issue is related to HDFS-708 [ HDFS-708 ]

            People

            • Assignee:
              Unassigned
              Reporter:
              Konstantin Shvachko
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development